Coordinate, right, middle, modified and multi-clicks, clicks on elements with no
primary pattern, unscrollable wheel scrolls and drags raised the window and
moved the real cursor without being asked. They now refuse with
background_unavailable before anything is sent. Coordinate and fractional
scrolls go through ScrollPattern first. With --restore-window only the target
window is raised (never a cloaked one), with a UIA activation fallback and a
second attempt, points are validated first and the cursor is put back behind
the input. A failing UIA pattern or disabled element is an error instead of a
real click, and delivery is measured after the post-action observe.
* fix(computer-use): scroll, expand and set sliders on Windows without raising the window
* fix(computer-use): wait for Windows scroll and slider writes to settle before verifying
* fix(computer-use): reject NaN for Windows slider set-value
* test(computer-use): wait on the Windows fixture state with vi.waitFor and a schema
* test(computer-use): check activation per action against a holder window in the Windows fixture
* test(computer-use): log fixture window events for activation failures
* fix(computer-use): report foreground when a Windows app raises itself for a UI Automation action
WPF activates its own window for Scroll, ExpandCollapse, RangeValue and SelectionItem calls from any UIA client, so the runtime now compares the foreground window before and after a pattern action instead of assuming background. The Windows e2e checks delivery against the fixture's own activation count, with a holder window in another process.
* test(computer-use): skip an unstarted holder when closing the Windows fixture
* fix(computer-use): bound Windows pattern scroll paging and tighten slider read-back
* fix(computer-use): cap Windows slider read-back tolerance and run the UIA e2e only on CI
* fix(computer): leave Linux and Windows windows in the background for accessibility actions
Element clicks no longer restore and activate the target window when an
AT-SPI action or UIA Invoke/SelectionItem/Toggle pattern handles them; only
the pointer fallback activates. On Linux, type-text and paste-text insert
into the focused editable field through AT-SPI EditableText without needing
window focus, and fall back to key synthesis or the clipboard otherwise.
Refs #12412
* fix(computer): refuse pointer input after failed activation
* fix(computer): stop after unconfirmed text writes
* fix(computer): insert the full UTF-8 text on Linux
* test(computer): focus each Linux text-entry trial
* fix(computer): verify Linux text mutations before deletion
AT-SPI can acknowledge writes without changing text. Read back
mutation counts and caret state before reporting success.
* fix(computer): verify complete Linux text replacements
* fix(computer): decline unreadable Linux text selections
Kelvin Amoaba
OrcaWinandOrcaWin
Neil