Kelvin Amoaba 030c88deec feat(computer): return tree diffs after actions and add get-app-state --query (#36023) ...

* feat(computer): send tree diffs after actions and add get-app-state --query

Action results now carry only the accessibility-tree lines that changed since
the caller's last view of the same window (~ changed, + added, - removed as
index ranges, or an explicit no-change line), plus the focused element. The
runtime sidecar keeps the last tree per session/worktree and app, so all three
providers get it. The CLI opts in with treeDiff; older runtimes strip the
param and keep returning the full tree, and --full-tree opts out.

The macOS helper now keeps an element's index across snapshots of a window:
AX identity first, then the next unclaimed index with the same signature for
views that rebuild their AX objects, and never reuses a removed index.

get-app-state --query prints matching elements with their ancestors and
--max-elements caps the output, both in the CLI over the rendered tree.

Custom action names with raw newlines no longer split an element over lines.

* fix(computer): preserve parent changes in snapshot diffs

* fix(computer): refuse stale index reuse for lookalike controls

* fix(computer): print full trees in action replies

* fix(computer): keep controls located in snapshot replies

2026-10-29 23:05:41 +00:00

..

Sources

feat(computer): return tree diffs after actions and add get-app-state --query (#36023)

2026-10-29 23:05:41 +00:00

Tests/OrcaComputerUseMacOSTests

feat(computer): return tree diffs after actions and add get-app-state --query (#36023)

2026-10-29 23:05:41 +00:00

Package.swift

Add native computer-use automation (#1683)

2026-05-11 14:20:08 -07:00