* feat(computer-use): support macOS middle click and gate the AX click path
`--mouse-button middle` already validated end-to-end through the CLI, the
zod schema, and the provider validator, and both the Windows and Linux
providers honored it. Only the macOS provider rejected it outright with
"middle-click is not yet supported", so the flag was a dead end on the one
platform that has no fallback.
Two changes:
- Add `.middle` to the macOS button mapping. macOS has no dedicated middle
event family, so it rides `otherMouseDown`/`otherMouseUp` with the button
number carried by `mouseButton: .center`; that constructor argument is
honored for exactly the `otherMouse*` types, so no extra field write is
needed.
- Validate the requested button before the accessibility fast path, and skip
that path for buttons it cannot express. Previously the raw string was read
unvalidated, and `performClickAction` only special-cased `right`, so
`click --mouse-button middle --element-index N` (no modifiers, count 1) fell
through to `AXPress` — a left click — and reported success with
`path: "accessibility"`. Any unrecognized button string did the same. This
matches guards the Windows and Linux providers already had.
The button enum moves into `OrcaComputerUseMacOSCore` so it is unit-testable;
`main.swift` keeps only the CoreGraphics mapping.
Also documents `--mouse-button` in the computer-use skill guide, which never
mentioned the flag, so agents on Windows and Linux had no way to discover it.
* test(computer-use): cover macOS middle click in the real-desktop e2e suite
* test(computer-use): prove macOS middle-click delivery
Mouse events posted with CGEventPostToPid reach the target app with no
window association, so AppKit never routes the press to a view: hover
states fire but the control is never activated, and the mouseUp is
dropped outright when posted back-to-back. Post click events to the HID
event tap instead (as keyboard synthesis already does), pace them, and
stamp mouseEventClickState so multi-clicks register.
Synthetic clicks now also report verification unverified/synthetic_input
from the helper itself, matching the other synthetic actions.
* perf(computer): add mac helper owner-loss benchmark
Measure the release helper's resident memory before and after its owner-session deadline. Record exact revisions, per-trial RSS, retained state, and clean-exit latency so lifecycle reclamation is reproducible.
* fix(computer): reap mac helper after client loss
Bind the detached macOS helper lifetime to authenticated socket ownership. Reap the helper after its final authenticated client disconnects, and add a startup deadline for sessions that never authenticate.
* test(computer): harden owner benchmark cleanup
* test(computer): make owner benchmark cleanup failure-safe
* test(computer): close remaining owner cleanup races
## Summary
- stop the macOS helper from unlinking caller-supplied socket and token paths
- keep parent-owned token cleanup tied to the helper startup that created it
- fail closed for non-socket socket-path collisions and cover helper cleanup races with regression tests
## Verification
- swift test --package-path native/computer-use-macos
- pnpm exec vitest run --config config/vitest.config.ts src/main/computer/macos-native-provider-client.test.ts
- pnpm run typecheck:node
- pnpm lint
- pnpm run build:computer-macos
- Electron/helper startup smoke validation
* fix(computer-use): stop malformed numeric args from crashing the agent
requiredNumber only checked isFinite, and several request handlers cast the
resulting Double straight to Int/UInt32 (elementIndex, clickCount, pages,
from/toElementIndex, windowId, windowIndex). Int(Double) traps when the value
is outside the integer's representable range, so a single malformed request
such as `{"elementIndex": 1e300}` crashed the entire agent process, killing
all in-flight automation.
Add a bounded conversion helper in the Core library and route every untrusted
Double->integer cast through it. boundedInteger truncates toward zero like
Int(Double) but returns nil (via init(exactly:)) instead of trapping when the
value is non-finite or out of range; the request handlers then surface a clean
invalid_argument error (or resolve to nil for the optional window lookups).
The helper lives in the Core library because the test target cannot import the
executable target where the handlers live. A negative-control run confirms the
out-of-range test traps with the previous Int(Double) cast and passes with the
fix.
* review: close remaining numeric crash paths
- parse stale-element validation indexes through bounded conversion
- reject malformed window selectors instead of dropping the target window
- bound synthetic scroll wheel deltas and cover Int32 conversion
---------
Co-authored-by: Jinjing <6427696+AmethystLiang@users.noreply.github.com>
Neil
Jinwoo Hong
Brennan Benson
