Commit Graph
4 Commits
Author SHA1 Message Date
Kelvin Amoaba ffaf887e78 feat(mobile): transcribe dictation on the phone, with the computer as fallback (#39386)
* feat(mobile): define the on-device speech contract for the native audio module and page bridge

* feat(mobile): dictate on the phone first and replay to the computer when the phone fails

* refactor(mobile): type the on-device speech module without a cast

* fix(mobile): point dictation's finish ref at the committed callback

* test(mobile): wait on the takeover request instead of a fixed sleep

* fix(mobile): release each dictation request's own share of a replayed backlog

* fix(mobile): let the computer's text replace the phone's after a fallback
2026-11-07 09:37:45 +00:00
OrcaWinandOrcaWin 7d6603aa17 [comment audit] Complete repository-wide comment cleanup (#33249)
* [comment audit] Align legacy failure-read counts with cached display

* [comment audit] Condense shared provider lifecycle and browser recovery comments [skip ci]

* [comment audit] Condense shared generation, path and preview comments [skip ci]

* [comment audit] Condense host, file and Git contract rationale [skip ci]

* [comment audit] Tighten Git lifecycle and provider contract comments [skip ci]

* [comment audit] Clarify shared host authority and launch isolation [skip ci]

* [comment audit] simplify local access mobile and native chat contracts [skip ci]

* [comment audit] simplify native chat presentation and state contracts [skip ci]

* [comment audit] simplify native runtime and provider contracts [skip ci]

* [comment audit] simplify migration and orchestration contracts [skip ci]

* [comment audit] simplify plugin trust and process capture rationale [skip ci]

* [comment audit] simplify terminal delivery and remote compatibility rationale [skip ci]

* [comment audit] clarify remote recovery and launch wire rationale [skip ci]

* [comment audit] shorten shared RPC compatibility and runtime evidence comments [skip ci]

* [comment audit] simplify secure storage shell startup and sidebar rationale [skip ci]

* [comment audit] clarify sidebar, skill and SSH invariants [skip ci]

* [comment audit] condense structured chat delivery and timing rationale [skip ci]

* [comment audit] condense terminal replay and telemetry invariants [skip ci]

* [comment audit] clarify launch permission and terminal delivery rationale [skip ci]

* [comment audit] clarify workspace ownership and layout contracts [skip ci]

* [comment audit] wait for fitted and painted SSH terminal links

* [comment audit] clarify layout persistence and session ownership [skip ci]

* [comment audit] clarify host routing and workspace lifecycle contracts [skip ci]

* [comment audit] clarify startup persistence and app lifetime comments [skip ci]

* [comment audit] clarify recovery and activity boundaries [skip ci]

* [comment audit] condense activity and artifact invariants [skip ci]

* [comment audit] condense automation authority contracts [skip ci]

* [comment audit] simplify automation scheduling and history contracts [skip ci]

* [comment audit] clarify browser chrome and annotation lifecycles [skip ci]

* [comment audit] clarify browser routing and retained guest ownership [skip ci]

* [comment audit] clarify browser recovery and document ownership comments [skip ci]

* [comment audit] restore hosted runners required by native IME fixtures

* [comment audit] shorten preview, palette and dashboard invariants [skip ci]

* [comment audit] clarify terminal preview, dashboard and retention comments [skip ci]

* [comment audit] clarify diff-note recovery and editor lifecycle comments [skip ci]

* [comment audit] shorten rich-text and combined-diff ownership comments [skip ci]

* [comment audit] shorten autosave, draft recovery and model ownership comments [skip ci]

* [comment audit] shorten save-queue, notebook and Markdown constraints [skip ci]

* [comment audit] shorten Monaco, PDF and rich Markdown safeguards [skip ci]

* [comment audit] clarify rich Markdown source and selection safeguards [skip ci]

* [comment audit] clarify editor loading and emulator stream lifetimes [skip ci]

* [comment audit] trim emulator and feature tour narration [skip ci]

* [comment audit] clarify floating workspace and GitHub cache ownership [skip ci]

* [comment audit] trim provider controls and Linear filter explanations [skip ci]

* [comment audit] clarify pairing and chat resume lifetimes [skip ci]

* [comment audit] clarify chat delivery and transcript lifetimes [skip ci]

* [comment audit] clarify session options and host-owned status [skip ci]

* [comment audit] clarify draft durability and live-status precedence [skip ci]

* [comment audit] Trim chat send and model ownership explanations [skip ci]

* [comment audit] Shorten transcript ownership and send recovery comments [skip ci]

* [comment audit] Simplify native chat composer and transcript hook comments [skip ci]

* [comment audit] Shorten native chat scroll and structured session explanations [skip ci]

* [comment audit] Trim workspace picker and onboarding commentary [skip ci]

* [comment audit] Simplify pet rendering, review cache and quick open comments [skip ci]

* [comment audit] Trim sidebar and session history explanations [skip ci]

* [comment audit] Condense session resume and review checks comments [skip ci]

* [comment audit] Shorten checks state and file explorer refresh comments [skip ci]

* [comment audit] Trim plugin bridge and source-control action rationale [skip ci]

* [comment audit] Trim source-control listing and review rationale [skip ci]

* [comment audit] Shorten sidebar lifecycle and explorer safeguards [skip ci]

* [comment audit] Condense Settings account and appearance rationale [skip ci]

* [comment audit] Trim pairing and repository Settings explanations [skip ci]

* [comment audit] Condense Settings search SSH and terminal safeguards [skip ci]

* [comment audit] Shorten Settings voice and provider ownership comments [skip ci]

* [comment audit] Condense Settings draft persistence and host-state rationale [skip ci]

* [comment audit] Trim Settings hooks and sidebar setup rationale [skip ci]

* [comment audit] Condense sidebar deletion and board-filter safeguards [skip ci]

* [comment audit] Condense worktree cards, host ownership and Markdown contracts [skip ci]

* [comment audit] Condense sidebar drag, folder and filtering contracts [skip ci]

* [comment audit] Condense sidebar parity, sleep and host ownership contracts [skip ci]

* [comment audit] Prune focus-read baseline after comment cleanup

* [comment audit] Clarify sidebar visibility, board drops and agent evidence [skip ci]

* [comment audit] Tighten sidebar listing, drag and reveal contracts [skip ci]

* [comment audit] Clarify viewport geometry and skill update safeguards [skip ci]

* [comment audit] Tighten skill inventory, usage and host status comments [skip ci]

* [comment audit] Clarify usage ownership and resource kill safeguards [skip ci]

* [comment audit] Tighten tab selection, rename and search contracts [skip ci]

* [comment audit] Tighten tab-strip geometry and drag ownership comments [skip ci]

* [comment audit] Tighten Tasks recovery, pagination and mutation comments [skip ci]

* [comment audit] Tighten provider loading and terminal recovery comments [skip ci]

* [comment audit] Tighten completion, cadence and PTY ownership comments [skip ci]

* [comment audit] Clarify terminal restore and parked-pane boundaries [skip ci]

* [comment audit] Tighten PTY startup and replay explanations [skip ci]

* [comment audit] Clarify PTY identity and delivery ownership [skip ci]

* [comment audit] Shorten remote reconnect and input replay rationale [skip ci]

* [comment audit] Condense terminal parking clipboard and file link comments [skip ci]

* [comment audit] Clarify IME ownership and trim trace history [skip ci]

* [comment audit] Shorten pane recovery and parked watcher explanations [skip ci]

* [comment audit] Condense snapshot replay and visibility recovery rationale [skip ci]

* [comment audit] Shorten terminal parking and keyboard ownership rationale [skip ci]

* [comment audit] Trim renderer lifecycle and menu rationale [skip ci]

* [comment audit] Condense workspace cleanup ownership and settlement notes [skip ci]

* [comment audit] Clarify renderer hook evidence and source ownership [skip ci]

* [comment audit] Tighten IPC ownership and reconnect explanations [skip ci]

* [comment audit] Trim renderer focus, paste and attention comments [skip ci]

* [comment audit] Shorten scroll, locale and agent lifecycle comments [skip ci]

* [comment audit] Trim agent launch, resume and ownership commentary [skip ci]

* [comment audit] Shorten account, browser and editor comments [skip ci]

* [comment audit] Clarify host-mirror waits, previews and input ownership [skip ci]

* [comment audit] Condense launch recovery and editor tokenization comments [skip ci]

* [comment audit] Condense workspace selection and terminal pane rationale [skip ci]

* [comment audit] Condense terminal fitting and rendering recovery comments [skip ci]

* [comment audit] Condense terminal input and host routing explanations [skip ci]

* [comment audit] Condense ownership and sleeping-session recovery rationale [skip ci]

* [comment audit] Condense structured launch and terminal link explanations [skip ci]

* [comment audit] Condense terminal ownership and workspace file routing rationale [skip ci]

* [comment audit] Condense session persistence and workspace activation rationale [skip ci]

* [comment audit] Condense workspace routing and browser ownership rationale [skip ci]

* [comment audit] Condense mirror recovery and layout authority rationale [skip ci]

* [comment audit] Condense runtime transport and file safety rationale [skip ci]

* [comment audit] Condense runtime graph and structured session rationale [skip ci]

* [comment audit] Condense browser creation and remote tab recovery rationale [skip ci]

* [comment audit] Condense snapshot authority and publisher recovery rationale [skip ci]

* [comment audit] Condense frame, startup and provider cache rationale [skip ci]

* [comment audit] Condense host twin and agent lifecycle rationale [skip ci]

* [comment audit] Condense browser authority and editor cleanup rationale [skip ci]

* [comment audit] Condense editor buffer identity and review cache rationale [skip ci]

* [comment audit] Condense provider lifetime and repository ownership rationale [skip ci]

* [comment audit] Condense reconnect, hydration and terminal retirement rationale [skip ci]

* [comment audit] Condense recovery, reconciliation and persistence rationale [skip ci]

* [comment audit] Condense cleanup ownership and listing admission rationale [skip ci]

* [comment audit] Condense teardown ordering and terminal ownership rationale [skip ci]

* [comment audit] Condense recovery ledger and web host-boundary rationale [skip ci]

* [comment audit] Condense web reconnect and credential lifecycle rationale [skip ci]

* [comment audit] Condense cloud rollout and assignment safety rationale [skip ci]

* [comment audit] Condense relay session and database recovery rationale [skip ci]

* [comment audit] Condense relay placement and protocol compatibility rationale [skip ci]

* [comment audit] Condense build and packaging constraints [skip ci]

* [comment audit] Condense CI gates and validation isolation rationale [skip ci]

* [comment audit] Condense catalog identity and browser validation rationale [skip ci]

* [comment audit] Condense mobile browser and frame-budget rationale [skip ci]

* [comment audit] Shorten mobile route and browser test explanations [skip ci]

* [comment audit] Condense editor and session seam test rationale [skip ci]

* [comment audit] Trim session and terminal closure history [skip ci]

* [comment audit] Shorten terminal browser and census explanations [skip ci]

* [comment audit] Preserve root-only E2E diagnostic baseline

* [comment audit] Condense mobile seam and HTML preview test rationale [skip ci]

* [comment audit] Condense preview instrumentation and render harness rationale [skip ci]

* [comment audit] Condense bundle contracts and native-build constraints [skip ci]

* [comment audit] Shorten native packaging and CI routing rationale [skip ci]

* [comment audit] Condense release tooling and benchmark constraints [skip ci]

* [comment audit] Condense packaging and platform gate rationale [skip ci]

* [comment audit] Shorten mobile route and history rationale [skip ci]

* [comment audit] Condense mobile browser and component rationale [skip ci]

* [comment audit] Shorten drawer and workspace-form rationale [skip ci]

* [comment audit] Trim mobile review and editor-document explanations [skip ci]

* [comment audit] Condense editor lifecycle and Markdown explanations [skip ci]

* [comment audit] Trim mobile file, diagnostics and dictation commentary [skip ci]

* [comment audit] Simplify mobile host-list and workspace-view explanations [skip ci]

* [comment audit] Clarify mobile shell lifetime and bridge delivery constraints [skip ci]

* [comment audit] Condense bridge stream, native request and audio rationale [skip ci]

* [comment audit] Reduce bridge schema and page-client commentary [skip ci]

* [comment audit] Tighten mobile page bridge contracts [skip ci]

* [comment audit] Clarify mobile cache and session constraints [skip ci]

* [comment audit] Condense mobile route and lifecycle rationale [skip ci]

* [comment audit] Tighten mobile shell hook timing rationale [skip ci]

* [comment audit] Clarify route handoff and notification compatibility [skip ci]

* [comment audit] Clarify push registration and dictation cleanup constraints [skip ci]

* [comment audit] Condense keyboard geometry and native media ownership rationale [skip ci]

* [comment audit] Tighten mobile chat scroll and composer concurrency rationale [skip ci]

* [comment audit] Condense mobile reply compatibility and upload ownership rationale [skip ci]

* [comment audit] Condense optimistic echo and scroll ownership rationale [skip ci]

* [comment audit] Tighten stream replay and queued session ownership rationale [skip ci]

* [comment audit] Condense mobile composer and reply-schema constraints [skip ci]

* [comment audit] Clarify mobile transcript recovery and session ownership [skip ci]

* [comment audit] Clarify mobile queue and source-control compatibility [skip ci]

* [comment audit] Condense storage and task launch constraints [skip ci]

* [comment audit] Clarify task schemas and terminal document ownership [skip ci]

* [comment audit] Tighten terminal input and viewport rationale [skip ci]

* [comment audit] Shorten terminal and RPC recorder contracts [skip ci]

* [comment audit] Condense recorder engine and connection contracts [skip ci]

* [comment audit] Shorten host credential and relay recovery comments [skip ci]

* [comment audit] Tighten pairing recovery and bundle transport rationale [skip ci]

* [comment audit] Clarify RPC delivery and workspace catalog boundaries [skip ci]

* [comment audit] Condense catalog and relay entry contracts [skip ci]

* [comment audit] Shorten shutdown and SSH relay delivery contracts [skip ci]

* [comment audit] Condense WSL relay and E2E regression rationale [skip ci]

* [comment audit] Shorten cross-version harness and renderer fixture explanations [skip ci]

* [comment audit] Condense paired-host and E2E lifecycle rationale [skip ci]

* [comment audit] Shorten layout oracle and regression explanations [skip ci]

* [comment audit] Condense paired recovery and ownership regression contracts [skip ci]

* [comment audit] Shorten restart, sidebar and SSH lifecycle contracts [skip ci]

* [comment audit] Condense terminal rendering and IME regression rationale [skip ci]

* [comment audit] Shorten terminal recovery and benchmark contracts [skip ci]

* [comment audit] Condense tooling and Windows lifecycle proof rationale [skip ci]

* [comment audit] Condense workflow routing and release rationale [skip ci]

* [comment audit] Condense mobile security and installer contracts [skip ci]

* [comment audit] Shorten native rationale and stylesheet workarounds [skip ci]

* [comment audit] Shorten Windows visibility probe rationale [skip ci]

* [comment audit] Condense embedded release and signing rationale [skip ci]

* [comment audit] Retain current main test behavior in final reconciliation [skip ci]

* [comment audit] Complete final comment review across current source inventory

* [comment audit] Handle consolidated audit reports before final CI

* [comment audit] Preserve credential storage census detail [skip ci]

* [comment audit] Scale broad changed E2E selections with verified timing shards

* [comment audit] Align E2E assertions with browser close and settled session metadata

* [comment audit] Match existing E2Es to current resource and windowing contracts

* [comment audit] Observe the shared server outage notice in reconnect coverage

* [comment audit] Make existing renderer probes measure settled tab and PTY work

* [comment audit] Align palette E2E selector with task search

* [comment audit] Refresh remaining palette search selectors

* [comment audit] Keep selector correction within selected audit specs

* [comment audit] Await emoji fixture settings inside renderer evaluation

* [comment audit] Cancel fixture fit retries between terminal tests

---------

Co-authored-by: OrcaWin <293788423+OrcaWin@users.noreply.github.com>
2026-10-25 04:05:46 -07:00
Jinwoo Hong 86b93e02a7 feat(mobile): the microphone owns the wake lock, and the stop reply carries the tail (OTA phase C, ruling 36) (#22072)
* feat(mobile): give the microphone its own screen lock (OTA phase C, ruling 36)

An open microphone holds the screen; a closed one gives it back. The lock
lives in the device-side capture on both hosts — the shell's
`native.audio.start|stop` handler and the native seam — so the page never
decides anything about the screen.

One tag per capture, minted by the module that owns the mic. Both captures
give it back on every close path: a stop, a page session ending with the
capture open, a device that would not begin, and an engine that throws after
the capture is open, which now ends the capture rather than leaving it live.

Claude-Session: https://claude.ai/code/session_01JNnE9qzUZMMnqpZWCqM3nb

* feat(mobile): carry the capture's tail on the stop reply (OTA phase C, ruling 36)

`native.audio.stop` drains what the ring still holds into its own reply, so
the page's `end()` is one verb: stop, hand the bytes on, done. The drain,
await and read-once-more ordering goes with it, and so do `ending`,
`reading` and `released` — three variables that existed only to order a last
read against the stop and to stop a refused read re-entering `end`.

The tail fields default rather than being required: the page updates over
the air and the shell does not, so a page this new can meet a shell that
answers `stopped` alone. That dictation loses its tail where a required
field would have lost it the stop.

The heap case from PR D's bot round cannot recur: `end` issues no read, and
a stop reply carries no interruption, so the lane that re-entered is gone.

Claude-Session: https://claude.ai/code/session_01JNnE9qzUZMMnqpZWCqM3nb

* refactor(mobile): drop the dictation finish id, which ordered nothing

`finishingIdRef` tracked the dictation a stop was finishing, and every state
it could name was already named: `cancel`, a disable, an unmount and a newer
start each bump the generation or clear the active id, so the finish guard
answered the same either way. Its one distinguishing arm released pending
audio bytes for a dictation whose budget `closeDictationAudio` had just
reset, and could subtract those bytes from a newer dictation's reserve.

`acceptingChunksRef` stays: it is what stops a late microphone event being
sent after the capture handed over its tail and before the finish goes out.
`pendingChunksRef` stays: `stop` awaits it so the finish cannot overtake the
last chunk send.

The finish guard is pinned by a case that cancels while the finish is in
flight; neutered, it reds.

Claude-Session: https://claude.ai/code/session_01JNnE9qzUZMMnqpZWCqM3nb

* feat(mobile): delete the page's wake-lock seam (OTA phase C, ruling 36)

The page never names the screen now. `native.wakelock.set` is gone with its
schemas, its shell server, its grant rows and its harness entry; so are the
page's keep-awake owner, the Android foreground re-acquire, and the
`DictationKeepAwakeDevice` the capture contract carried. One module holds
the screen — the device calls the microphone's capture makes — and both
device-side captures share its one tag, because there is one microphone.

Deleted: native-wakelock.ts (120), native-wakelock.test.ts (140),
mobile-dictation-keep-awake.ts (248), mobile-dictation-keep-awake.test.ts
(440), mobile-dictation-foreground-keep-awake.ts (78). With the tag pools
gone, the desktop-start flow has one stale check instead of two, no startup
budget to wait out and nothing to release.

A source-scanning census pins it: no module under mobile/src or mobile/app
but the one owner imports expo-keep-awake, and nothing anywhere names the
retired verb. It reports the file and line, and checks the owner does import
the package so the absence is the rule holding and not the match missing.

KNOWN RED, reported and not recorded over: 25 golden cases in the speech.*
families fail. The recorder adapter had to drop its keep-awake owner, which
moves `adapterSha256` for every golden that mounts it, and the deleted
owner's id minting shifts the deterministic random sequence, so the recorded
`dictationId` values move too. No speech.dictation.* param, reply or
operation changed. Awaiting the lead's call on a scoped re-record.

Claude-Session: https://claude.ai/code/session_01JNnE9qzUZMMnqpZWCqM3nb

* test(mobile): re-record the corpus after the wake-lock deletion

Baseline bumped to 491eb35b4d and the corpus recorded against it. Every
changed line in `goldens/` falls into six classes and nothing else:

  baseline                            1574 lines   787 goldens
  lockfileSha256                      1574 lines   787 goldens
  adapterSha256                         38 lines    19 goldens
  scenarioSha256                         8 lines     4 goldens
  dictationId shift                    728 lines     4 goldens
  keep-awake effects + renumbering      63 lines     2 goldens
                                      ----
                                      3985 lines, which is the whole diff

`recorderSha256` is untouched: no recorder module outside `adapters/` moved.

The id shift is attributable arithmetic, not a behaviour change. The
recording scheduler seeds `Math.random` with an LCG from seed 1; replaying
it gives draw 1 `8ig2henseon` and draw 2 `dakoxjr8wun`. The deleted
keep-awake owner minted its id from draw 1 during the hook's mount, so the
dictation id took draw 2. With the owner gone the dictation id takes draw 1,
which is why four scenario steps that pinned the literal value move with it.

`adapterSha256` covers `speech.setup-sheet` as well as the three dictation
families, because one adapter module hosts them all. The two goldens with
vanished effects also renumber the ordinals after them, which is what the
removal of an entry from a sequential counter does.

`lockfileSha256` is provenance that `compareGolden` copies from the actual
and never fails on. It moves in all 787 files because main's own
`mobile/pnpm-lock.yaml` has moved since the corpus was last recorded; this
branch does not touch it.

Claude-Session: https://claude.ai/code/session_01JNnE9qzUZMMnqpZWCqM3nb

* fix(mobile): drop the retired wake-lock grant C7.7's route row carried

The merge brought in the session route's manifest entry, which names all
four dictation grants including `native.wakelock.set`. This branch deleted
that verb, so the row granted a page something the shell no longer serves.
Ruling 32 item 6 said C7.7 takes the dictation grant from PR D's census in
this merge; this is that.

Nothing caught it automatically: the shell-side grant list is derived from
the verb tuple and is already three, and the closure census holds a route to
the grants it needs rather than refusing ones it does not.

Claude-Session: https://claude.ai/code/session_01JNnE9qzUZMMnqpZWCqM3nb

* test(mobile): repin the session-route closure at the measured 4,331

Measured on this merge rather than summed: 4,331 modules, 989 local.

Main is red here on its own pin. `3cfb070294` measures 4,333 / 991 against a
committed 4,330, three modules this branch never touched — measured in a
throwaway worktree detached at that commit, with the same generators run.
This merge measures 4,331 / 989, and diffing the two local lists gives the
difference exactly: `mobile-dictation-keep-awake.ts` and
`mobile-dictation-foreground-keep-awake.ts` leave, and nothing joins. So the
branch's own effect is the -2 ruling 36 implies, and repinning to the
measurement is also what takes main's closure test green again.

Claude-Session: https://claude.ai/code/session_01JNnE9qzUZMMnqpZWCqM3nb

* test(mobile): repin the session-route closure at 4,331 on the merge with #22067

Measured on this merge: 4,331 modules, 989 local, against main's freshly
repinned 4,333 / 991 at `3cfb070294`.

Both provenances kept. #22067 names the three `src/shared` modules #21924
pulled into the page closure, which is what made main's earlier 4,330 stale;
this branch's own -2 is the page's wake-tag owner and its Android foreground
re-acquire, deleted by ruling 36. Diffing the two local lists gives exactly
those two leaving and nothing joining, so the number is a reading rather
than 4,333 minus an argument.

Claude-Session: https://claude.ai/code/session_01JNnE9qzUZMMnqpZWCqM3nb

* fix(mobile): close the capture when the desktop start fails

The hook opens the microphone before it asks the desktop for a session, so a
refused session left the mic open — and, since the screen rides the mic, the
display held until the user cancelled, retried, or the screen unmounted. The
failure arm now runs the same `rollbackRecordingStart` the commit failure
does, because "undo the capture this start opened" is one thing and the hook
owns it; guarded like that arm, so a seam that throws on the way down cannot
take the desktop cancel with it.

Red-first on both hosts. Natively, a new test drives the real seam under the
engine and keep-awake mocks: the refusal used to leave `initialize` with no
`toggleRecording(false)` and a held screen. On the page, the mic control's
own test over the port pair saw `native.audio.start` with no
`native.audio.stop`. Two unit cases pin the call itself, including for a
start nobody will report.

Ruling 36's own words: mic closed means released. This closes the mic rather
than adding a release beside it.

Claude-Session: https://claude.ai/code/session_01JNnE9qzUZMMnqpZWCqM3nb

* test(mobile): re-record after the failed-start capture close

Baseline bumped to a1bdee9af9 and the corpus recorded against it. The whole
diff is two classes:

  baseline   1574 lines   787 goldens
  content      74 lines     1 golden

`lockfileSha256`, `adapterSha256`, `scenarioSha256` and `recorderSha256` do
not move: no lockfile, adapter, scenario or recorder module changed.

The one content golden is
`matrix-speech.dictation-start-speech.dictation.start-1`. Its failure
partitions now carry a `rollback-recording` effect at ordinal 3, which is
the capture being closed, and the `speech.dictation.cancel#1` entries after
it renumber from 3,4 to 3,4,4,5 — the pool holds one entry per distinct
content, so a partition whose ordinal moved stops sharing an entry with the
one it used to match. Removing the new effect and ignoring ordinals makes
the two recordings identical, checked by dereferencing every hash rather
than by reading the diff.

Claude-Session: https://claude.ai/code/session_01JNnE9qzUZMMnqpZWCqM3nb

* fix(mobile): only the owning start rolls its capture back

There is one capture seam and it carries no start identity, so round 1's
rollback let a stale start's rejection end a live newer dictation: A opens
the capture and waits on the desktop, the user cancels, B starts and is
recording, A's request finally rejects and ends B's microphone and hands
back B's screen. The rollback now runs only while this start is still the
current one, which is what `wasCurrent` on the line above already reads; a
stale failure still cancels its own desktop session and touches nothing
else. Past the generation the capture was either already ended by whatever
superseded this start, or belongs to the one that did.

Red-first on both hosts, driving that exact sequence rather than a spy: the
native test over the real seam saw the screen go `+ - + -`, and the page's
mic-control test over the port pair saw a fourth `native.audio.` verb after
B was recording. Both now end with B still holding what it took.

The mirror image is covered and now pinned at host level too: A's request
resolving late does not commit A over B, because the stale check after the
desktop start returns through `cancelStaleStart`, which cancels A's session
without touching the capture.

Claude-Session: https://claude.ai/code/session_01JNnE9qzUZMMnqpZWCqM3nb

* test(mobile): re-record after the stale start stopped rolling back

Baseline bumped to e17b2cf603. Two classes, the whole diff:

  baseline   1574 lines   787 goldens
  content      74 lines     1 golden

`lockfileSha256`, `adapterSha256`, `scenarioSha256` and `recorderSha256` do
not move.

The one content golden is
`matrix-speech.dictation-start-speech.dictation.start-1`, whose scenario is
the superseded start, so every partition in it is a stale one. The
`rollback-recording` effect round 1 put there is gone, and the
`speech.dictation.cancel#1` entries fold back from 4 to 2 as the ordinals
after it renumber — the pool holds one entry per distinct content, so
partitions whose ordinals agree again share an entry again. Dropping that
effect from the superseded partitions and ignoring ordinals makes the two
recordings identical, checked by dereferencing every hash.

`speech-desktop-start-recording-failed` is untouched: that start still owns
its capture at failure time, so it still rolls back.

Claude-Session: https://claude.ai/code/session_01JNnE9qzUZMMnqpZWCqM3nb
2026-09-21 18:57:55 -04:00
Jinwoo Hong a91ca8b19e feat(mobile): dictation on the OTA page over native audio verbs (Phase C, C7.10 PR D) (#21905)
* feat(mobile): serve dictation capture over four native audio verbs

The page owns dictation's state machine and speaks `speech.dictation.*` to
the desktop, where transcription runs; the microphone is the shell's. So the
shell gains `native.audio.start|read|stop` and `native.wakelock.set` — four
rows, four grants — and rings what the microphone produces at the page's own
pending-audio budget rather than pushing bytes the page would hand straight
back.

`native_audio_not_capturing` joins the refusal vocabulary: a read for a
capture this session does not have is the one refusal the page must tell from
a device that failed.

Claude-Session: https://claude.ai/code/session_01JNnE9qzUZMMnqpZWCqM3nb

* refactor(mobile): take dictation capture through one seam on both hosts

`use-mobile-dictation.ts` held the microphone and the wake tag directly, so
the page had a hook whose every device call was a stub answering denied. The
five calls move behind `src/platform/dictation-capture.ts`: natively the same
calls in the same order, on the page the shell's four verbs, with the drain
raising the events the engine emits.

The tag bookkeeping stays where it was and stops importing `expo-keep-awake`:
two calls come in through the seam and the pools, the queue, the timeouts and
the retries are the same on either host.

The chunk sender is untouched. A chunk carries raw PCM because that is what
the budget counts and what `speech.dictation.chunk` is built from, so the page
pays one decode of 32 KB a second rather than the flow carrying two shapes.

Claude-Session: https://claude.ai/code/session_01JNnE9qzUZMMnqpZWCqM3nb

* perf(mobile): drain the shell's audio ring on a 500 ms batch

One `speech.dictation.chunk` per native microphone event is 31.25 forwarded
requests a second, and each holds one of the bridge's 64 in-flight slots for a
whole desktop round trip. Measured over ten seconds against a two-second link:
63 in flight at the peak and one slot left for the rest of the page.

Drained every 500 ms instead: 5 in flight, 60 slots free, the same 42 KiB/s,
and 38 frames out and 34 back for the whole session. The frame cap was never
the bound — half a second of PCM is 3.3% of one.

Claude-Session: https://claude.ai/code/session_01JNnE9qzUZMMnqpZWCqM3nb

* test(mobile): derive dictation's grant rows from the route closure

Which routes need the four audio verbs is a census, not a hand list: the rule
reads each registered page route's own closure and holds its `grants` to what
that closure reaches. Vacuous on today's route list — the session route is the
only closure carrying the seam and is not registered yet — so a control runs
the same rule against the session module and names all four as missing.

The closure also records what the seam took off the page: `@orca/expo-two-way-
audio` and `expo-keep-awake` are gone from it entirely, and removing the web
file puts four of the vendored stub's modules back.

The mic control's render case found a real one. A start the shell refused
outright left the button on "Starting voice dictation" with no way back, which
is every tap on a route without the grants. It reports the refusal and returns
to idle.

Claude-Session: https://claude.ai/code/session_01JNnE9qzUZMMnqpZWCqM3nb

* test(mobile): keep the mic control's render case inside the tests typecheck

A `let` the renderer assigns inside a callback narrows to `never` afterwards,
and the mocked `Pressable` took `children` as `unknown`. Both are type-level
only, and the ratchet is the gate that notices: a test outside `tsc` can pin a
shape that stopped existing.

Claude-Session: https://claude.ai/code/session_01JNnE9qzUZMMnqpZWCqM3nb

* fix(mobile): hand over the audio still in the ring before stopping the shell

`end` cancelled the drain and stopped the capture without a last read, so up to
one drain interval of the utterance's tail — 16,000 bytes, the 400 ms a user is
still speaking as they lift the button — was discarded on every stop. The
reviewer's probe spoke 12,288 bytes in the last 400 ms and the page delivered
none of them. Natively that audio is already in the hook's hands, so this was a
page-only loss of the end of every sentence.

`end` is now asynchronous: it cancels the timer, waits for any read in flight,
reads once more, and only then stops the shell — stopping first takes the
capture away and the read after it is refused. `stop()` awaits it before it
stops accepting chunks and before it takes the pending set, or the tail would be
dropped one line later and `finish` could overtake the last send.

A release still skips the last read: the screen is going away and there is
nobody left to hand the tail to.

Claude-Session: https://claude.ai/code/session_01JNnE9qzUZMMnqpZWCqM3nb

* fix(mobile): release the page session's wake tag when the session ends

The wake-lock server held its tags per instance and a new one was built per page
session with nothing ever disposing it, so a tag a session took was never given
back and the screen stayed awake for the app's lifetime. The page is a document
that can navigate, fault or be swiped away mid-dictation, so nothing else was
ever going to call deactivate. Its own docstring claimed the opposite.

It now answers `{ serve, dispose }` and is disposed with the session exactly as
the microphone and the staged media handles are. `dispose` drops only what is
still held, so a tag the page already gave back is not deactivated twice.

Claude-Session: https://claude.ai/code/session_01JNnE9qzUZMMnqpZWCqM3nb

* refactor(mobile): drop the capability flag no screen reads

`canCaptureAudio` answered four grants to nobody and had no test. The fence that
actually holds is the per-verb `ungranted` check every member already makes
before a frame is sent, and the mic control's render case pins what a screen
does with it.

The surface's member list is pinned instead, so the next flag with nothing
behind it has to be added there on purpose.

Claude-Session: https://claude.ai/code/session_01JNnE9qzUZMMnqpZWCqM3nb

* test(mobile): re-measure the closure the dictation census records, and make its controls real

Three corrections to the census, all of them about the census lying rather than
the product being wrong.

The recorded number was four modules; measured on this head it is eight — five
from `@orca/expo-two-way-audio` and three from `expo-keep-awake` — for a net +7
once the local file that left is counted. The absolute closure counts are
provenance in the docstring and are not asserted, because every merge of main
moves them. The absence now has a precondition: both package names are resolved
from the install, so a substring matching nothing fails as a typo.

The case named "reads the census file" read no file. It reads the shell's own
verb table through `import()`, behind the closure guard, so a grant the shell has
no row for reds instead of agreeing with itself.

And the grant control re-implemented the rule's filter inline. Both the rule and
the control drive one function now, over the entry C7.7 would write if it copied
its neighbours' grants.

Claude-Session: https://claude.ai/code/session_01JNnE9qzUZMMnqpZWCqM3nb

* fix(mobile): serialise the shell's audio starts and stops

Two starts racing the OS permission prompt — a page reloaded while it is up,
which is the case the replacement rule exists for — both reached `listen()`, and
the second overwrote the first's handlers without removing them. The engine went
on calling into a capture nobody could read, and `dispose` freed one of the two.
A stop that overlapped a start found nothing to end and the start opened a
microphone behind it.

Starts and stops now run one at a time in the order the page asked for them, and
a start that comes back after the session ended opens nothing. Reads stay off the
queue: they must not wait behind an opening capture, and a read with no capture
is already a refusal.

Claude-Session: https://claude.ai/code/session_01JNnE9qzUZMMnqpZWCqM3nb

* fix(mobile): end a capture on the same two interruptions on both hosts

The page treated any interruption as the capture being taken away, while the
native seam has always gated on `began` and `blocked`. So an `ended` on its own —
the OS handing the session back after a notification chime — cancelled a live
dictation on the page and did nothing natively.

The rule is now one predicate beside the vocabulary it belongs to, read by all
three places that decide it: the shell, which stops filling its ring; the native
seam, which raises it off `onAudioInterruption`; and the page, which raises it off
a read reply. `recording` still ends the page's capture whatever the kind, because
a capture the shell no longer has is gone however it went.

The native half had no test of its own, which is why the drift was invisible. It
has one now: the five calls it makes, the chunk it hands over, the wake tag, and
which interruptions end it.

Claude-Session: https://claude.ai/code/session_01JNnE9qzUZMMnqpZWCqM3nb

* refactor(mobile): type the chunk sender as what its callers pass

The sender's parameter was the native `MicrophoneDataEvent` though both callers
hand it a `DictationCaptureChunk`. Structurally the event is the wider type, so
it accepted either and read `droppedBytes` off neither — a page whose audio the
shell's ring had dropped would have sent it as though nothing were missing, and
nothing would have failed to compile.

Typed as the chunk, with a compile fence beside the seam holding both directions:
a chunk is accepted, an event is refused, and a raw buffer is not a chunk. The
seam normalises the bytes, so the widening the sender did has no caller left.

Claude-Session: https://claude.ai/code/session_01JNnE9qzUZMMnqpZWCqM3nb

* test(mobile): re-record the session closure with dictation's page modules on it

The capture seam moves this count down rather than up. Measured at 2d697a4012
and at this head: modules 4,324 to 4,319 and local modules 974 to 977. Three
local modules join — the page's capture seam, its contract and the audio verb
shapes — and eight vendored ones leave, because the seam is what stops the page
importing a microphone it does not have: five of `@orca/expo-two-way-audio` and
three of `expo-keep-awake`, replaced by four verbs the shell answers.

Named beside the sentences already there, with the counterfactual that puts the
eight back pointing at the census that runs it.

Claude-Session: https://claude.ai/code/session_01JNnE9qzUZMMnqpZWCqM3nb

* test(mobile): re-record the session closure over C2.9's two bridge modules

C2.9 moved this pin to 4,326 by putting the page-route grants and the manifest
grammar behind `bridge-envelope.ts`, which every page closure reads. Dictation's
capture still moves it down from wherever it lands: measured at 5d13a70ea3 and
at this head, modules 4,326 to 4,321 and local modules 976 to 979.

Three local modules join — the page's capture seam, its contract and the audio
verb shapes — and eight vendored ones leave, five of `@orca/expo-two-way-audio`
and three of `expo-keep-awake`, replaced by four verbs the shell answers.
Recorded beside C2.9's sentences rather than in place of them.

Claude-Session: https://claude.ai/code/session_01JNnE9qzUZMMnqpZWCqM3nb

* perf(config): build each route closure once in the dictation census

Every `mobileWebAppRouteClosure` call is a full esbuild metafile build, and the
cases here ask about six route modules across nine of them — fifteen builds. The
CPU that cost tipped two timing-sensitive neighbours in this shard over: a
benchmark whose child has 100 ms to write a pid file, and a census globbing a
scratch tree another test was removing. Neither is reached by this file and both
are fragile without it; the added load was the difference. With the census
excluded the shard was green, with it three runs of three were red.

Memoised per route module, so six builds. The shard still fails intermittently
on this machine for its own reasons, but not because of this file.

Claude-Session: https://claude.ai/code/session_01JNnE9qzUZMMnqpZWCqM3nb

* fix(config): gate the dictation census's verb-table case on the mobile install

`import()` defers when a module loads, not what loading costs. Vite transforms
the file at that moment and resolves the nearest `tsconfig.json` for it, which is
`mobile/tsconfig.json`, which extends `expo/tsconfig.base.json` — absent on the
root-only shard, so the transform threw `TSConfckParseError` and reddened
`test / tests node 24 1/8`. The comment claiming the dynamic import avoided that
was wrong.

Gated on `mobileWebAppDependenciesPresent()`, the same guard the closure cases
use, which is the only thing that keeps a mobile module off that shard.

Reproduced both ways in this tree by moving `mobile/node_modules` aside: before,
1 failed with that error; after, 3 passed and 6 skipped; with the install back,
9 passed.

Claude-Session: https://claude.ai/code/session_01JNnE9qzUZMMnqpZWCqM3nb

* fix(mobile): make the page's capture end idempotent

The hook's interruption handler is `() => void cancel()`, and `cancel` reaches
`capture.end()` synchronously through `closeDictationAudio`. So the last read in
`end` could raise an interruption that called straight back into `end`, whose own
last read was refused for the same reason the first was — the shell has no
capture — and the recursion issued bridge reads until the page ran out of memory.
The pin crashed the test worker with `JavaScript heap out of memory` before the
fix.

A second `end` returns the first one's promise, assigned before anything can
await so a handler re-entering from inside the read finds it set. `begin` clears
it, because the seam is memoised per client and the next dictation on the same
screen has to be able to drain — pinned by a case that ends, starts again and
reads the new audio.

Claude-Session: https://claude.ai/code/session_01JNnE9qzUZMMnqpZWCqM3nb

* fix(mobile): give back a wake tag whose activation landed after dispose

`held.add(tag)` ran after the device call, so a `dispose()` falling between the
activate and its reply walked an empty set and the tag was recorded afterwards.
Nothing walks that set again, so the screen stayed awake for the app's lifetime —
and the page is a document that can be swiped away mid-dictation, which is
exactly when that window is open.

A tag that lands after the session ended is deactivated on the spot and reported
to the page as not held.

Claude-Session: https://claude.ai/code/session_01JNnE9qzUZMMnqpZWCqM3nb

* fix(mobile): swallow the native audio session's shutdown failures

The contract says `end` never rejects, and the native one did: it is `async`, so a
throwing JSI binding rejects rather than throws. Every caller reaches it as
`void capture.end()` inside a synchronous `try`, which cannot see a rejection — so
a device that would not stop recording left an unhandled rejection, and the
cleanup the `try` was written to protect was never what was at risk.

`release` was worse in kind: it runs bare in the unmount path, so a throwing
`tearDown` took the wake tag's release and the desktop's cancel with it.

Both log and continue. Pinned behaviourally against an engine that refuses to
stop and a tear-down that throws; the source case that claimed the hook's `try`
was the guard now says where the guard actually is.

Claude-Session: https://claude.ai/code/session_01JNnE9qzUZMMnqpZWCqM3nb

* test(mobile): pin the order stop() hands the tail over in

The tail fix depends on `await capture.end()` running before chunks stop being
accepted, and nothing held that: the source check only asserts `end()` precedes
`Promise.allSettled`, which both orders satisfy, so reversing the two lines left
the whole mobile suite green while the page silently dropped the end of every
sentence.

Driven against a capture whose `end()` delivers a chunk — what the page's seam
does and the device's never does, which is why only this case can tell the orders
apart. It asserts the tail reaches the desktop as `speech.dictation.chunk`, with
those bytes, before `finish`; reversed, no chunk is sent at all.

The mocked seam is one object for the module's life, because the hook keys its
teardown effect on the capture's identity: a seam returning a fresh object per
render cancels the dictation on every render. Both real seams are stable.

Claude-Session: https://claude.ai/code/session_01JNnE9qzUZMMnqpZWCqM3nb

* fix(mobile): keep a wake tag recorded until the device really drops it

`held` was cleared whether or not `deactivate` succeeded, so a tag the device
refused was forgotten. The page's owner queues exactly that failure for a retry
(`pendingCleanupTags`), and the retry arrives here as another `active: false` —
which a shell that had already forgotten the tag answered without calling
anything, leaving the native tag on for the life of the app.

The boundary: the set means "the device still has this tag", not "the page asked
for it". That keeps the reason the set exists — never call `deactivateKeepAwake`
for a tag this shell never took, since its failure would read to the page as a
wake lock it could not drop — while letting every retry for a live tag through.
Always reaching the device regardless of the set would have traded the second
property for the first.

Pinned with a device that refuses once and then accepts, and with a dispose whose
deactivation is refused.

Claude-Session: https://claude.ai/code/session_01JNnE9qzUZMMnqpZWCqM3nb

* fix(mobile): tear the audio device down for a start that lost the race

A start that came back after `dispose()` returned without ending the engine, and
the local `end()` is a no-op with no capture, so nothing tore down the session
`initialize()` had just brought up. Nobody else would: the dispose had already
run and no capture was ever recorded. The device's audio session stayed up for
the life of the app.

It ends the engine on that path now. Pinned on both places the race can be lost —
inside the permission prompt and inside the open itself — each asserting exactly
one teardown and no live listeners.

Claude-Session: https://claude.ai/code/session_01JNnE9qzUZMMnqpZWCqM3nb

* test(mobile): drive the stop-tail pin through the typed fake client

The hand-rolled client fake needed three type assertions to stand in for an
`RpcClient`, which the changed-code quality gate refuses on new lines. It drives
`createFakeRpcClient` instead — a real one — answering each request as the hook
makes it, and reads the chunk's base64 by narrowing rather than asserting.

Re-confirmed the pin still reds on the reversed order after the rewrite.

Claude-Session: https://claude.ai/code/session_01JNnE9qzUZMMnqpZWCqM3nb

* fix(mobile): clear the page capture's end when it settles

A finished `end` stayed in `ending` until `begin` cleared it, and `begin` is not
guaranteed to run between two ends: `open` starts the shell recording, and a start
that goes stale after the hook sets `activeIdRef` cleans up through `capture.end()`
without ever committing. The second end answered from the first one's settled
promise and never issued `native.audio.stop`, leaving the shell holding a live
microphone.

Cleared on settle instead, and only for its own flight. The re-entrancy the latch
was really for happens while the promise is still pending, so guarding the flight
is enough. `release` keeps its own flag rather than a settled `ending`, which now
clears itself and would unlatch it.

Claude-Session: https://claude.ai/code/session_01JNnE9qzUZMMnqpZWCqM3nb

* fix(mobile): serialise the shell's wake tags and record them before compensating

Two ways the device could end up holding a tag nothing would ever give back.

A release that arrived while its own activate was still in flight read `held`
before the activate had recorded anything, found nothing, deactivated nothing and
reported the tag off; then the activate landed and the device stayed on. `held` is
read and written across an await, so operations are now chained per tag. Per tag
rather than per server, so one hanging device call cannot hold up another
dictation.

And on the late-tag path, a compensating deactivate the device refused was
swallowed while `{ active: false }` was returned: the device still held the lock,
`held` lacked the tag, `dispose` had already walked the set, and the page believed
an activate had succeeded. The tag is now recorded as soon as `activate` resolves,
deleted only once the device has really dropped it, and a refusal rejects so the
caller's retry path runs.

Claude-Session: https://claude.ai/code/session_01JNnE9qzUZMMnqpZWCqM3nb

* fix(mobile): let dispose re-read the tag set once its turn comes

Queueing dispose behind each tag's own operations introduced a call the module
says it does not make: a release already in flight can give the tag back before
dispose runs, and deactivating an unheld tag is a native call whose failure would
read to the page as a lock it could not drop.

Re-reads the set when the queued action runs rather than trusting what it held
when dispose was called.

Claude-Session: https://claude.ai/code/session_01JNnE9qzUZMMnqpZWCqM3nb

* test(mobile): pin the dictation census to the routes C4.4 registered

Claude-Session: https://claude.ai/code/session_01JNnE9qzUZMMnqpZWCqM3nb
2026-09-21 06:42:30 -04:00