Commit Graph
3 Commits
Author SHA1 Message Date
Brennan Benson 66dfdc456f feat(computer-use): support macOS middle click and stop the silent left-click fallback (#14721)
* feat(computer-use): support macOS middle click and gate the AX click path

`--mouse-button middle` already validated end-to-end through the CLI, the
zod schema, and the provider validator, and both the Windows and Linux
providers honored it. Only the macOS provider rejected it outright with
"middle-click is not yet supported", so the flag was a dead end on the one
platform that has no fallback.

Two changes:

- Add `.middle` to the macOS button mapping. macOS has no dedicated middle
  event family, so it rides `otherMouseDown`/`otherMouseUp` with the button
  number carried by `mouseButton: .center`; that constructor argument is
  honored for exactly the `otherMouse*` types, so no extra field write is
  needed.
- Validate the requested button before the accessibility fast path, and skip
  that path for buttons it cannot express. Previously the raw string was read
  unvalidated, and `performClickAction` only special-cased `right`, so
  `click --mouse-button middle --element-index N` (no modifiers, count 1) fell
  through to `AXPress` — a left click — and reported success with
  `path: "accessibility"`. Any unrecognized button string did the same. This
  matches guards the Windows and Linux providers already had.

The button enum moves into `OrcaComputerUseMacOSCore` so it is unit-testable;
`main.swift` keeps only the CoreGraphics mapping.

Also documents `--mouse-button` in the computer-use skill guide, which never
mentioned the flag, so agents on Windows and Linux had no way to discover it.

* test(computer-use): cover macOS middle click in the real-desktop e2e suite

* test(computer-use): prove macOS middle-click delivery
2026-08-15 00:41:45 -07:00
Neil d0f341ad69 fix(computer-use): make modifier clicks interruption-safe (#11451)
* fix(computer-use): make modifier clicks interruption-safe

* fix(computer-use): pace modified Windows multiclicks

* fix(computer-use): address modifier safety review
2026-07-29 18:29:10 -07:00
Brennan Benson 31f643ca42 Add version-matched skill guides to the CLI (#8624)
* Add version-matched bundled skill guides

* Clarify skill freshness rollout PRs

* Add canonical skills show alias

* fix(skills): address guide review feedback

* fix(skills): make guide commands cross-platform

* fix(skills): apply the ORCA convention to the emulator guides

Review follow-up: the emulator guides still instructed literal
`orca emulator ...` in sh fences with no Linux disambiguation, so on
unmanaged Linux they could launch the GNOME screen reader — the exact
failure the executable-selection preamble prevents. Both emulator
guides now carry the preamble and ORCA placeholder across fences,
tables, and prose, and the cross-platform safety test covers all four
converted guides. Also replaces computer-use's "unless a block names a
shell" carve-out, which contradicted its own POSIX example, with the
unconditional placeholder rule.
2026-07-14 02:17:55 -07:00