Files
orca/mobile/src
Jinwoo Hong 8f78c28248 fix(orchestration): fence worker release on mobile keystrokes (#19337)
* fix(orchestration): fence worker release on mobile keystrokes

A settled worker's terminal stayed ownership_state='owned' unless a takeover was
recorded, and the only recorder was orchestration.workerTerminalUserInput, which
only the desktop/web xterm input signal and the native-chat composer call. Mobile
input arrives as terminal.send / stream input frames instead of a report, so a
phone user typing in a settled worker's pane never fenced anything: worker-list
kept recommending release and worker-release closed the PTY under them.

Give the host one definition of "a human typed into this terminal" and route every
lane through it. The mobile input floor claim is that definition and already exists
on both byte lanes: it is taken only for deliberate phone input, never for the
emulator's own query replies, and never for an agent's `orca terminal send`, which
names itself a desktop client and so is indistinguishable from a keystroke at this
layer. Settling that claim after an accepted write now records the takeover through
the same code the RPC reporter uses, throttled to one write per pane per 30s so a
keystroke does not pay for an immediate transaction. The record lands on the runtime
that owns both the terminal and the orchestration database, so SSH-hosted and remote
workers behave exactly like local ones.

No mobile change: mobile already sends client.type (mobile/src/terminal/terminal-send-request.ts:24).

* fix(orchestration): ask the database, do not remember, whether a pane is fenced

The keystroke throttle armed on the attempt rather than on the outcome, so a
zero-row or thrown record poisoned the pane for 30s. A phone keystroke during
the worker-start readiness wait lands before prepareStartingWorkerAuthority
creates the owned resource; a real keystroke seconds later was then suppressed,
the worker settled, and workerRelease closed the terminal under the phone user.
A SQLITE_BUSY on the first write did the same, with no retry.

The cache was the defect, not its arming condition. Its precondition is the set
of owned resources on the pane, which changes underneath it, and any cache keyed
on ownership identity would have to read the database to learn that identity --
which is the whole question. So the input lane now asks: a read using the same
predicate the write uses answers "is anything still fenceable here?" without
taking BEGIN IMMEDIATE, and only then is the write attempted. Ordinary typing
costs a lookup instead of a write lock, a failed write is retried by the next
keystroke, and a takeover writes once per ownership epoch rather than once per
window, because the flip to user_owned removes the pane from the candidate set.
Sharing the predicate keeps the probe from drifting from the writer.

Adds the two escape cases as permanent regressions, drives the mocked send
through the real RuntimeTerminalWriter, and asserts a mobile takeover lifts the
settled-worker resume fence, which no test covered.

* refactor(orchestration): let the database dedupe the takeover, drop the read probe

The probe was meant to keep keystrokes off BEGIN IMMEDIATE, so it had to earn
that with a number. Measured against a real WAL database it costs more than the
write it avoids: at 25 live workers the probe is 0.19ms and the no-op write is
0.10ms, because the probe runs the same candidate selection with each statement
taking its own read snapshot instead of sharing the transaction's. It is a
compensating mechanism with negative value, so it goes, along with the database
method and the predicate extraction it needed.

owned -> user_owned is one-way and scoped to a resource, so the database is
already the dedupe: every deliberate human write attempts the transition, the
second attempt matches no row, and the fence sweep runs only on changed > 0.
Nothing is remembered between keystrokes, so no state can outlive the ownership
it described -- a keystroke before the worker's authority attaches, a write the
database refuses, and a re-dispatch onto the same pane all resolve against the
rows as they are at that instant. An attempt costs about 0.1ms at typical fleet
size and 0.34ms at 100 live workers, on mobile writes only.

Replaces the write-count test, which asserted the old mechanism, with the
invariant: many keystrokes settle into one takeover and one fence sweep. Adds
the re-dispatch case, where a pane's next worker is fenced on its own merits.

* refactor(terminal): name the provenance rule the takeover fence hangs off

The fence rode the mobile input floor claim, with only a comment tying the two
together. The floor is arbitration -- who may write next -- while the fence needs
provenance -- who produced the bytes. They agree today, so anyone reweighing the
floor would have moved the fence without noticing.

isDeliberateHumanInput states the provenance rule on its own terms, and both byte
lanes decide with it when they open a write: the claim carries the verdict beside
the handle, and settlement records the takeover only when a human produced the
bytes. No behavior change -- afterWrite is wired only where the predicate already
answers true -- and the rule is now pinned by its own cases, so a future
arbitration change has to answer this question again rather than inherit it.

* test(orchestration): prove the unary lane classifies a metadata-less phone

A phone build older than client.type is recognised only by its pane's mobile
driver, which the unary lane passes as the provenance evidence. Nothing proved
it did: replacing that argument with false left all 17 tests green while a
shipped phone silently stopped fencing worker release. The new case drives a
clientless send on a mobile-driven pane and fails under that mutation.

The stream lane now passes false outright. Its isMobile is read off the same
client object it carries, so the metadata-less phone cannot reach it, and
passing the flag suggested a legacy path that does not exist there.

Also states what the per-keystroke cost scales with. A pane owning no resource
misses the pane_key index and falls through to a scan of owned resources, so the
figure is tens of microseconds at realistic worker counts rather than a flat
0.1ms, and it grows with rows that are never released.

* fix(terminal): let provenance alone decide the takeover, on every accepted write

A phone older than client.type sends no client metadata, and both stream
initializers derive isMobile from that metadata alone, so such a subscription
reported false and took the stream lane's uninstrumented branch: provenance was
computed and then never consumed. Bytes from a real person landed through both
frame adapters and the resource stayed owned, so workerRelease closed the PTY
under them. The unary lane already fenced that population off the pane's mobile
driver, which is the host's standing reading of clientless input, so the two byte
lanes disagreed at the destructive boundary.

The predicate was still subordinate to floor plumbing: it could only be consulted
where a floor client id existed. Now the accepted-write callback attaches on both
lanes regardless of whether a floor was reserved, and humanInput alone decides
recording; a write holding no claim commits nothing. Arbitration keeps its own
condition around reserveWrite, where it belongs, and the unary lane's duplicate
outer provenance filter is gone. The stream lane reads clientless provenance from
the pane's driver, the same policy the unary lane uses.

The claim holder is now TerminalInputWrite, carrying the verdict beside an
optional floorClaim, so the structure says what the doc said: a write may fence
without holding the floor.

Regressions drive both real frame adapters, clientless direct delivery, and the
paired-web desktop negative. Metadata-only provenance fails 3 on the stream lane
and 1 on the unary lane; gating the callback on a reservation fails the same 3.

* fix(runtime): resolve retained handles before mobile input provenance

A renderer reload clears transient handles while retaining runtime-owned
PTY identities. Legacy mobile provenance saw no leaf, then sendTerminal
restored the same handle and delivered an unfenced key. Normalize through
getLivePtyForHandle at the shared live-leaf resolver entry so classification
and writes agree, preserving existing leaf generation/incarnation checks.

Caller audit:
- terminal-send-method: driver, query-reply authority, lock and floor checks
  now resolve the retained PTY before sending.
- terminal-input-delivery: legacy mobile classification and exact-PTY
  binding now see the same target as the writer; equality checks remain.
- terminal-multiplex-subscribe-resolution: retained PTYs resolve directly
  without a spurious missing-terminal wait.
- terminal-lifecycle-methods resize and terminal-viewport-methods display
  mode, restore-fit and updateViewport retain their original PTY target.
- inspectTerminalProcess: avoids false terminal_gone after reload while
  preserving provider inspection and incarnation fences.
- getLivePaneKeyForTerminalHandle and getOrchestrationDispatchAuthority:
  unaffected because both already call getLivePtyForHandle first.
No wire/schema changes, host fallback, process-death inference, or Git
workspace assumptions; SSH providers keep ownership of execution evidence.

Validation:
- Unmodified round-3 reviewer probe: reproduced 2/2 failures, then 2/2 pass.
- Unmodified round-2 reviewer probes: 13/13 pass.
- Checked-in takeover suites: 24/24 pass. Removing only the resolver call
  fails both new reload cases; source restored afterward.
- RPC orchestration + terminal, aggregate runtime handle registry,
  handle incarnation, mobile tab mount, stale geometry, and reload probe:
  2027 passed, 1 skipped (89 files).
- tc:node and check:code-quality:changed pass; background launch enabled.

* test(rpc): require unconditional terminal afterWrite callbacks

Update exact sendTerminal expectations for the round-2 accepted-write
contract. Preserve beforeWrite expectations, absence of reserveWrite,
byte payloads and call-count checks; require afterWrite to be a function.

Reproduced the requested two-file run: 5 failed, 31 passed. The full RPC
suite exposed the same stale shape in ACK budget/overflow, desktop resize
(including its later retry), and agent-prompt fallback assertions. Update
those too, for 11 assertions across six test files. No production changes.

Validation: ORCA_BACKGROUND_LAUNCH=1 full src/main/runtime/rpc suite:
264 files passed; 2292 tests passed, 1 skipped. Changed-code quality and
staged oxlint/React Doctor/oxfmt checks passed. Ran lint-staged --no-stash
manually to honor checkout safety rather than its default backup hook.

* fix(mobile): report worker takeover outside terminal byte delivery

New phones announce accepted real user input through the existing worker
report RPC, addressed by terminal handle. Share a per-client/per-handle
30-second gate with one bounded retry; report through the same RPC client
as the input. Cover live commits and dictation via their shared sender,
accessory keys, gestures, buffered submit, paste and accepted native chat.
Query replies, attachment heals, triage and diff-review sends do not report.
Phones predating this build do not fence release.

Remove byte provenance and takeover callbacks from host delivery. Restore
both lanes' pre-PR floor-claim plumbing and the original options assertions.
Keep the host recorder uncached with its conditional resume-fence sweep.
No DB schema or stream change; terminal is an optional report address.

Retain the shared resolver recovery independently of takeover: the new
SSH inspection test fails without it during renderer reload. Other callers
still benefit for subscription, resize, viewport and exact-PTY binding;
unary driver/lock checks see the retained PTY. Pane routing and dispatch
authority already recover through getLivePtyForHandle and are unaffected.
Existing leaf generation checks and first-PTY adoption remain unchanged.
No other input-plumbing hunk is retained relative to the PR base.

Replace byte-takeover tests with handle-addressed local/SSH report and
unknown-handle tests, plus real unary/stream writes asserting zero SQL
prepare/exec calls. Mobile send-site integration covers reports, exclusions,
rejected writes and gate counts. Desktop report tests are unchanged.
Register replacement coverage in the settled-worker release manifest.

Validation (all background): host/RPC/runtime 3541 passed, 2 skipped;
mobile session/terminal 2045 passed; node and mobile typechecks, changed
quality, mobile oxlint, reliability manifest and max-lines ratchet passed.
All five requested mutations fail assertions; resolver revert also fails
independent inspection. Staged checks run manually with --no-stash.
Final src diff against PR base: 5 files, +165/-13 (previously +839/-85).

* fix(runtime): allow the takeover report from mobile-scoped tokens

The mobile RPC allow-list gates every phone request before dispatch and the
reporter swallows a refusal, so without this entry every phone shipped
unfenced. Pin it beside the report tests, and pin the once-per-takeover
fence sweep the replaced byte-lane suite used to assert.

* fix(mobile): a no-op takeover report does not arm the gate; Stop reports too

A key during worker startup reports before the resource is owned; caching
that zero-change reply for 30 s suppressed the report that would have fenced
the worker once it attached. Native-chat Stop is deliberate input and now
reports on an accepted Escape.

* fix(mobile): takeover gate ignores the host answer, like desktop

Reopening the gate on a zero-change reply made every accepted key on an
ordinary terminal an RPC plus a host write transaction (round 6: 100 for
100). The startup window it closed is unreachable: the agent has no prompt
to accept input until after its resource row exists. Plain terminals now
pay one report per 30 s window; the native-chat Stop report stays.

Send-site fixture answers the report RPC with a changed count; the draft
test filters to terminal.send calls.

* docs(runtime): say why resolveLiveLeafForHandle re-links before lookup

* chore(i18n): regenerate the runtime-required catalog for the contrast floor strings

* test(orchestration): give the stopping-worker guard fixtures a Run

* test(orchestration): drop fence-sweep assertions retired by the settled-worker policy

* test(orchestration): pin the mid-boot phone takeover that #19608 makes possible

A handle-addressed report during the worker's tui-idle wait now finds the
custody row written at terminal creation, so it flips the pane to user_owned
and worker-release retains it instead of closing it under the user.
2026-09-08 14:48:57 -04:00
..