Files
camoufox/docs/input-dispatch.md
T
Jake WriterandClaude Opus 5 e6288c8c57 fix(juggler): synthesized input carries what real mouse and keyboard input carries
- pointerType was "" for every Playwright mouse event: juggler dispatched with
  MOZ_SOURCE_UNKNOWN. It now passes MOZ_SOURCE_MOUSE (#776).
- keyboard.type() never pressed Shift: a shifted character now arrives
  bracketed by ShiftLeft keydown/keyup (location 1) with shiftKey set.
- After the pointer was parked off content, pointerover/enter re-entered with
  buttons=1 and pressure 0.5; the tracked position is now forgotten on park.
- A Windows identity gets contextmenu after mouseup with buttons=0, as Windows
  does; GTK/macOS keep it on press.
- Wheel events are sent as line deltas (DOMMouseScroll.detail 3 per notch
  instead of the pixel count).
- The browser rect is measured after the APZ flush await, so a chrome height
  change during the wait cannot put a y==0 dispatch one row above content.
- ci/run_sundial.py moves and clicks the mouse so input vectors have data.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-16 22:07:39 -06:00

89 lines
4.7 KiB
Markdown

# Synthesized input dispatch
Every synthesized mouse and wheel event in the parent process goes through
`additions/juggler/input/MouseDispatch.js`. `scripts/check-input-dispatch.py`
fails the build if anything else dispatches input or does browser-relative
coordinate arithmetic, and it runs on every pull request.
## The invariant
> A synthesized input event whose ack we await must reach the content
> renderer — and when it does not, we must stop waiting.
## Why it is worth a module and a lint
Four deadlocks shipped between 2026-04 and 2026-09, all the same failure:
| Date | Commit | Trigger |
|---|---|---|
| 2026-06-04 | `9270618` | `x == width` / `y == height` — the far edges |
| 2026-07-18 | `541ffca` | trajectory points, which bypassed the endpoint's guard (#225, #677) |
| 2026-07-24 | `16e5a13` | a zero-displacement move |
| 2026-09-03 | `014cc65` | `y == 0` — the near edge (#751, #752) |
Each was fixed by adding one more coordinate guard at one more call site. That
does not converge, for two reasons.
**The trigger set is not enumerable.** Whether relative `y == 0` reaches the
renderer is decided by `Math.round(boundingBox.top) < boundingBox.top` — a
rounding accident in the fractional height of browser chrome, which varies with
the *spoofed OS*: windows `51.4` → `51` deadlocks, macos `53.1` → `53`
deadlocks, linux `56.5` → `57` is fine. No review catches that, and no
hand-written list of coordinates contains it.
**Every miss costs the whole process.** `activateAndRun()`
(`TargetRegistry.js`) serializes input on a promise chain shared by every tab in
the process. It swallows errors to keep the chain running, but it cannot swallow
a callback that never returns. One unbounded `await` for an ack that will never
arrive wedges every later input event, in every tab, permanently — at 0% CPU,
with nothing in flight and no diagnostic.
`#677` is why review is not enough: restoring the humanize trajectory meant
writing a bounds check, and the one written was a copy of the pre-`#225` form,
reintroducing a fixed deadlock one day before it was re-fixed.
## How it is enforced
**One chokepoint.** `MouseDispatch` owns the relative→absolute conversion (with
the boundary snap), the in-viewport predicate, and the ack wait. Callers pass
relative coordinates and never see a bounding box.
**A fresh rect.** The browser rect is measured *after* the last `await` before
the dispatch (`apz-repaints-flushed`), never before it. The chrome can change
height during that wait — measured 2026-09-14: the nav-bar grew from 40 to
41 px about a second after startup when the (then forced) built-in theme was
applied — and a rect taken before it put a relative `y == 0` one row above the
content: the event reached the renderer as an exit event at client `y == -1`,
produced no ack, and the page saw no mousemove (~1 in 25 runs of
`near-edge-mouse-deadlock.py`). Both dispatch sites (`Page.dispatchMouseEvent`,
`Page.dispatchWheelEvent`) measure synchronously before dispatching.
**Bounded waits.** `sendAcked()` waits at most `kAckDeadlineMs` (5s) and then
drops the event with a warning naming the type, coordinate and browser rect. The
deadline is sized above the slowest *legitimate* ack, not near the typical one:
acks are p99 1ms on an idle page, but they are delivered from the content main
thread and inherit any block on it — a 3s synchronous script delayed one by
2849ms. `sendTrajectoryAcked()` abandons the rest of a curve after the first
undelivered point, so the ~90 bounded waits a curve can hold — the 1.5s default
humanize ceiling at 60Hz — cannot add up to an unbounded slot. It also refuses
to dispatch a point on the pixel the previous dispatch left the cursor on: a
zero-displacement move produces no `eMouseMove`, so it is never acked, and
that is the same deadlock reached from inside a curve rather than from
`Page.dispatchMouseEvent`.
`activateAndRun()` carries a 30s backstop for the other unbounded waits
reachable from the same slot (`apz-repaints-flushed`, `TabSwitchDone`, the drag
path's waits), none of which has failed yet.
**The static check.** `scripts/check-input-dispatch.py`, wired into
`.github/workflows/lint.yml`. Two exemptions, both content-process:
`PageAgent.js` (drag events, already content-relative, no ack) and
`FrameTree.js` (the ack *producer*).
**Boundary coverage.** `tests/patches/mouse-boundary-sweep.py` sweeps the whole
viewport ring across every spoofed OS with humanize on and off, asserting each
point is acked *and observed by the page*. Hand-picked coordinate lists are what
let each of the four bugs through: `humanize-edge-deadlock.py` probed only the
far edges, and `humanize-mouse-trajectory.py` pins `os="linux"` — the one
fingerprint immune to `#751`. `tests/patches/input-ack-backstop.py` covers the
bounded wait itself.