Commit Graph
6 Commits
Author SHA1 Message Date
Jinwoo Hong b44ef1e59d fix(skills): narrow computer-use discovery boundary (#17736)
* fix(skills): narrow computer-use discovery boundary

* chore: remove merge-formatting noise

* fix(skills): name browser page automation surfaces
2026-08-31 18:57:52 -04:00
Brennan Benson 2a760e310b fix(computer): report unasserted accessibility actions (#15028)
* fix(computer): report unasserted accessibility actions

* fix(computer): fail closed on missing action metadata

* Fix merged tab search test fixture
2026-08-18 11:29:26 -07:00
Brennan Benson fc8b92e507 docs(computer): explain screenshot file requirements (#15054)
* docs(computer): clarify screenshot output requirements

* fix(cli): do not advertise an unshipped --probe flag

The capabilities help line referenced --probe, which does not exist yet;
it ships in a later change. Advertising it here would be false until then.

* fix(cli): align computer-use screenshot guidance

* docs(computer): document inline screenshot fallback

* docs(computer): keep screenshot summary accurate

* docs(computer): keep screenshot guidance general
2026-08-18 01:18:56 -07:00
Brennan Benson 66dfdc456f feat(computer-use): support macOS middle click and stop the silent left-click fallback (#14721)
* feat(computer-use): support macOS middle click and gate the AX click path

`--mouse-button middle` already validated end-to-end through the CLI, the
zod schema, and the provider validator, and both the Windows and Linux
providers honored it. Only the macOS provider rejected it outright with
"middle-click is not yet supported", so the flag was a dead end on the one
platform that has no fallback.

Two changes:

- Add `.middle` to the macOS button mapping. macOS has no dedicated middle
  event family, so it rides `otherMouseDown`/`otherMouseUp` with the button
  number carried by `mouseButton: .center`; that constructor argument is
  honored for exactly the `otherMouse*` types, so no extra field write is
  needed.
- Validate the requested button before the accessibility fast path, and skip
  that path for buttons it cannot express. Previously the raw string was read
  unvalidated, and `performClickAction` only special-cased `right`, so
  `click --mouse-button middle --element-index N` (no modifiers, count 1) fell
  through to `AXPress` — a left click — and reported success with
  `path: "accessibility"`. Any unrecognized button string did the same. This
  matches guards the Windows and Linux providers already had.

The button enum moves into `OrcaComputerUseMacOSCore` so it is unit-testable;
`main.swift` keeps only the CoreGraphics mapping.

Also documents `--mouse-button` in the computer-use skill guide, which never
mentioned the flag, so agents on Windows and Linux had no way to discover it.

* test(computer-use): cover macOS middle click in the real-desktop e2e suite

* test(computer-use): prove macOS middle-click delivery
2026-08-15 00:41:45 -07:00
Neil d0f341ad69 fix(computer-use): make modifier clicks interruption-safe (#11451)
* fix(computer-use): make modifier clicks interruption-safe

* fix(computer-use): pace modified Windows multiclicks

* fix(computer-use): address modifier safety review
2026-07-29 18:29:10 -07:00
Brennan Benson 31f643ca42 Add version-matched skill guides to the CLI (#8624)
* Add version-matched bundled skill guides

* Clarify skill freshness rollout PRs

* Add canonical skills show alias

* fix(skills): address guide review feedback

* fix(skills): make guide commands cross-platform

* fix(skills): apply the ORCA convention to the emulator guides

Review follow-up: the emulator guides still instructed literal
`orca emulator ...` in sh fences with no Linux disambiguation, so on
unmanaged Linux they could launch the GNOME screen reader — the exact
failure the executable-selection preamble prevents. Both emulator
guides now carry the preamble and ORCA placeholder across fences,
tables, and prose, and the cross-platform safety test covers all four
converted guides. Also replaces computer-use's "unless a block names a
shell" carve-out, which contradicted its own POSIX example, with the
unconditional placeholder rule.
2026-07-14 02:17:55 -07:00