6 Commits
Author SHA1 Message Date
Kelvin Amoaba c794f75e4e fix(computer-use): use UI Automation for Windows scroll, expand and slider actions (#41429)
* fix(computer-use): scroll, expand and set sliders on Windows without raising the window

* fix(computer-use): wait for Windows scroll and slider writes to settle before verifying

* fix(computer-use): reject NaN for Windows slider set-value

* test(computer-use): wait on the Windows fixture state with vi.waitFor and a schema

* test(computer-use): check activation per action against a holder window in the Windows fixture

* test(computer-use): log fixture window events for activation failures

* fix(computer-use): report foreground when a Windows app raises itself for a UI Automation action

WPF activates its own window for Scroll, ExpandCollapse, RangeValue and SelectionItem calls from any UIA client, so the runtime now compares the foreground window before and after a pattern action instead of assuming background. The Windows e2e checks delivery against the fixture's own activation count, with a holder window in another process.

* test(computer-use): skip an unstarted holder when closing the Windows fixture

* fix(computer-use): bound Windows pattern scroll paging and tighten slider read-back

* fix(computer-use): cap Windows slider read-back tolerance and run the UIA e2e only on CI
2026-11-12 23:17:10 +00:00
Kelvin Amoaba 41c8b81372 fix(computer): capture covered Windows windows, refuse elevated targets, time-box hung windows (#41468) 2026-11-12 20:40:24 +00:00
OrcaWinandOrcaWin bbe22681c2 test: remove notification, icon and native policy inventories (#41375)
* test: remove notification and native policy inventories

* test: remove icon projection samples and tray suite

---------

Co-authored-by: OrcaWin <293788423+OrcaWin@users.noreply.github.com>
2026-11-11 20:54:07 -08:00
OrcaWinandOrcaWin b9d91d7fa4 test: remove low-value renderer launch and Windows checks (#41305)
Co-authored-by: OrcaWin <293788423+OrcaWin@users.noreply.github.com>
2026-11-11 17:22:41 -08:00
Neil 65d30b156a fix(computer-use): only SW_RESTORE minimized windows on Windows (#41253) 2026-11-11 13:45:40 -08:00
Kelvin Amoaba 3b83d3d260 fix(computer): skip activation for accessibility clicks (#36006)
* fix(computer): leave Linux and Windows windows in the background for accessibility actions

Element clicks no longer restore and activate the target window when an
AT-SPI action or UIA Invoke/SelectionItem/Toggle pattern handles them; only
the pointer fallback activates. On Linux, type-text and paste-text insert
into the focused editable field through AT-SPI EditableText without needing
window focus, and fall back to key synthesis or the clipboard otherwise.

Refs #12412

* fix(computer): refuse pointer input after failed activation

* fix(computer): stop after unconfirmed text writes

* fix(computer): insert the full UTF-8 text on Linux

* test(computer): focus each Linux text-entry trial

* fix(computer): verify Linux text mutations before deletion

AT-SPI can acknowledge writes without changing text. Read back
mutation counts and caret state before reporting success.

* fix(computer): verify complete Linux text replacements

* fix(computer): decline unreadable Linux text selections
2026-10-29 22:17:13 +00:00