mirror of
https://github.com/stablyai/orca.git
synced 2026-09-22 08:02:28 +00:00
Jinwoo Hong
a91ca8b19e
* feat(mobile): serve dictation capture over four native audio verbs The page owns dictation's state machine and speaks `speech.dictation.*` to the desktop, where transcription runs; the microphone is the shell's. So the shell gains `native.audio.start|read|stop` and `native.wakelock.set` — four rows, four grants — and rings what the microphone produces at the page's own pending-audio budget rather than pushing bytes the page would hand straight back. `native_audio_not_capturing` joins the refusal vocabulary: a read for a capture this session does not have is the one refusal the page must tell from a device that failed. Claude-Session: https://claude.ai/code/session_01JNnE9qzUZMMnqpZWCqM3nb * refactor(mobile): take dictation capture through one seam on both hosts `use-mobile-dictation.ts` held the microphone and the wake tag directly, so the page had a hook whose every device call was a stub answering denied. The five calls move behind `src/platform/dictation-capture.ts`: natively the same calls in the same order, on the page the shell's four verbs, with the drain raising the events the engine emits. The tag bookkeeping stays where it was and stops importing `expo-keep-awake`: two calls come in through the seam and the pools, the queue, the timeouts and the retries are the same on either host. The chunk sender is untouched. A chunk carries raw PCM because that is what the budget counts and what `speech.dictation.chunk` is built from, so the page pays one decode of 32 KB a second rather than the flow carrying two shapes. Claude-Session: https://claude.ai/code/session_01JNnE9qzUZMMnqpZWCqM3nb * perf(mobile): drain the shell's audio ring on a 500 ms batch One `speech.dictation.chunk` per native microphone event is 31.25 forwarded requests a second, and each holds one of the bridge's 64 in-flight slots for a whole desktop round trip. Measured over ten seconds against a two-second link: 63 in flight at the peak and one slot left for the rest of the page. Drained every 500 ms instead: 5 in flight, 60 slots free, the same 42 KiB/s, and 38 frames out and 34 back for the whole session. The frame cap was never the bound — half a second of PCM is 3.3% of one. Claude-Session: https://claude.ai/code/session_01JNnE9qzUZMMnqpZWCqM3nb * test(mobile): derive dictation's grant rows from the route closure Which routes need the four audio verbs is a census, not a hand list: the rule reads each registered page route's own closure and holds its `grants` to what that closure reaches. Vacuous on today's route list — the session route is the only closure carrying the seam and is not registered yet — so a control runs the same rule against the session module and names all four as missing. The closure also records what the seam took off the page: `@orca/expo-two-way- audio` and `expo-keep-awake` are gone from it entirely, and removing the web file puts four of the vendored stub's modules back. The mic control's render case found a real one. A start the shell refused outright left the button on "Starting voice dictation" with no way back, which is every tap on a route without the grants. It reports the refusal and returns to idle. Claude-Session: https://claude.ai/code/session_01JNnE9qzUZMMnqpZWCqM3nb * test(mobile): keep the mic control's render case inside the tests typecheck A `let` the renderer assigns inside a callback narrows to `never` afterwards, and the mocked `Pressable` took `children` as `unknown`. Both are type-level only, and the ratchet is the gate that notices: a test outside `tsc` can pin a shape that stopped existing. Claude-Session: https://claude.ai/code/session_01JNnE9qzUZMMnqpZWCqM3nb * fix(mobile): hand over the audio still in the ring before stopping the shell `end` cancelled the drain and stopped the capture without a last read, so up to one drain interval of the utterance's tail — 16,000 bytes, the 400 ms a user is still speaking as they lift the button — was discarded on every stop. The reviewer's probe spoke 12,288 bytes in the last 400 ms and the page delivered none of them. Natively that audio is already in the hook's hands, so this was a page-only loss of the end of every sentence. `end` is now asynchronous: it cancels the timer, waits for any read in flight, reads once more, and only then stops the shell — stopping first takes the capture away and the read after it is refused. `stop()` awaits it before it stops accepting chunks and before it takes the pending set, or the tail would be dropped one line later and `finish` could overtake the last send. A release still skips the last read: the screen is going away and there is nobody left to hand the tail to. Claude-Session: https://claude.ai/code/session_01JNnE9qzUZMMnqpZWCqM3nb * fix(mobile): release the page session's wake tag when the session ends The wake-lock server held its tags per instance and a new one was built per page session with nothing ever disposing it, so a tag a session took was never given back and the screen stayed awake for the app's lifetime. The page is a document that can navigate, fault or be swiped away mid-dictation, so nothing else was ever going to call deactivate. Its own docstring claimed the opposite. It now answers `{ serve, dispose }` and is disposed with the session exactly as the microphone and the staged media handles are. `dispose` drops only what is still held, so a tag the page already gave back is not deactivated twice. Claude-Session: https://claude.ai/code/session_01JNnE9qzUZMMnqpZWCqM3nb * refactor(mobile): drop the capability flag no screen reads `canCaptureAudio` answered four grants to nobody and had no test. The fence that actually holds is the per-verb `ungranted` check every member already makes before a frame is sent, and the mic control's render case pins what a screen does with it. The surface's member list is pinned instead, so the next flag with nothing behind it has to be added there on purpose. Claude-Session: https://claude.ai/code/session_01JNnE9qzUZMMnqpZWCqM3nb * test(mobile): re-measure the closure the dictation census records, and make its controls real Three corrections to the census, all of them about the census lying rather than the product being wrong. The recorded number was four modules; measured on this head it is eight — five from `@orca/expo-two-way-audio` and three from `expo-keep-awake` — for a net +7 once the local file that left is counted. The absolute closure counts are provenance in the docstring and are not asserted, because every merge of main moves them. The absence now has a precondition: both package names are resolved from the install, so a substring matching nothing fails as a typo. The case named "reads the census file" read no file. It reads the shell's own verb table through `import()`, behind the closure guard, so a grant the shell has no row for reds instead of agreeing with itself. And the grant control re-implemented the rule's filter inline. Both the rule and the control drive one function now, over the entry C7.7 would write if it copied its neighbours' grants. Claude-Session: https://claude.ai/code/session_01JNnE9qzUZMMnqpZWCqM3nb * fix(mobile): serialise the shell's audio starts and stops Two starts racing the OS permission prompt — a page reloaded while it is up, which is the case the replacement rule exists for — both reached `listen()`, and the second overwrote the first's handlers without removing them. The engine went on calling into a capture nobody could read, and `dispose` freed one of the two. A stop that overlapped a start found nothing to end and the start opened a microphone behind it. Starts and stops now run one at a time in the order the page asked for them, and a start that comes back after the session ended opens nothing. Reads stay off the queue: they must not wait behind an opening capture, and a read with no capture is already a refusal. Claude-Session: https://claude.ai/code/session_01JNnE9qzUZMMnqpZWCqM3nb * fix(mobile): end a capture on the same two interruptions on both hosts The page treated any interruption as the capture being taken away, while the native seam has always gated on `began` and `blocked`. So an `ended` on its own — the OS handing the session back after a notification chime — cancelled a live dictation on the page and did nothing natively. The rule is now one predicate beside the vocabulary it belongs to, read by all three places that decide it: the shell, which stops filling its ring; the native seam, which raises it off `onAudioInterruption`; and the page, which raises it off a read reply. `recording` still ends the page's capture whatever the kind, because a capture the shell no longer has is gone however it went. The native half had no test of its own, which is why the drift was invisible. It has one now: the five calls it makes, the chunk it hands over, the wake tag, and which interruptions end it. Claude-Session: https://claude.ai/code/session_01JNnE9qzUZMMnqpZWCqM3nb * refactor(mobile): type the chunk sender as what its callers pass The sender's parameter was the native `MicrophoneDataEvent` though both callers hand it a `DictationCaptureChunk`. Structurally the event is the wider type, so it accepted either and read `droppedBytes` off neither — a page whose audio the shell's ring had dropped would have sent it as though nothing were missing, and nothing would have failed to compile. Typed as the chunk, with a compile fence beside the seam holding both directions: a chunk is accepted, an event is refused, and a raw buffer is not a chunk. The seam normalises the bytes, so the widening the sender did has no caller left. Claude-Session: https://claude.ai/code/session_01JNnE9qzUZMMnqpZWCqM3nb * test(mobile): re-record the session closure with dictation's page modules on it The capture seam moves this count down rather than up. Measured at2d697a4012and at this head: modules 4,324 to 4,319 and local modules 974 to 977. Three local modules join — the page's capture seam, its contract and the audio verb shapes — and eight vendored ones leave, because the seam is what stops the page importing a microphone it does not have: five of `@orca/expo-two-way-audio` and three of `expo-keep-awake`, replaced by four verbs the shell answers. Named beside the sentences already there, with the counterfactual that puts the eight back pointing at the census that runs it. Claude-Session: https://claude.ai/code/session_01JNnE9qzUZMMnqpZWCqM3nb * test(mobile): re-record the session closure over C2.9's two bridge modules C2.9 moved this pin to 4,326 by putting the page-route grants and the manifest grammar behind `bridge-envelope.ts`, which every page closure reads. Dictation's capture still moves it down from wherever it lands: measured at5d13a70ea3and at this head, modules 4,326 to 4,321 and local modules 976 to 979. Three local modules join — the page's capture seam, its contract and the audio verb shapes — and eight vendored ones leave, five of `@orca/expo-two-way-audio` and three of `expo-keep-awake`, replaced by four verbs the shell answers. Recorded beside C2.9's sentences rather than in place of them. Claude-Session: https://claude.ai/code/session_01JNnE9qzUZMMnqpZWCqM3nb * perf(config): build each route closure once in the dictation census Every `mobileWebAppRouteClosure` call is a full esbuild metafile build, and the cases here ask about six route modules across nine of them — fifteen builds. The CPU that cost tipped two timing-sensitive neighbours in this shard over: a benchmark whose child has 100 ms to write a pid file, and a census globbing a scratch tree another test was removing. Neither is reached by this file and both are fragile without it; the added load was the difference. With the census excluded the shard was green, with it three runs of three were red. Memoised per route module, so six builds. The shard still fails intermittently on this machine for its own reasons, but not because of this file. Claude-Session: https://claude.ai/code/session_01JNnE9qzUZMMnqpZWCqM3nb * fix(config): gate the dictation census's verb-table case on the mobile install `import()` defers when a module loads, not what loading costs. Vite transforms the file at that moment and resolves the nearest `tsconfig.json` for it, which is `mobile/tsconfig.json`, which extends `expo/tsconfig.base.json` — absent on the root-only shard, so the transform threw `TSConfckParseError` and reddened `test / tests node 24 1/8`. The comment claiming the dynamic import avoided that was wrong. Gated on `mobileWebAppDependenciesPresent()`, the same guard the closure cases use, which is the only thing that keeps a mobile module off that shard. Reproduced both ways in this tree by moving `mobile/node_modules` aside: before, 1 failed with that error; after, 3 passed and 6 skipped; with the install back, 9 passed. Claude-Session: https://claude.ai/code/session_01JNnE9qzUZMMnqpZWCqM3nb * fix(mobile): make the page's capture end idempotent The hook's interruption handler is `() => void cancel()`, and `cancel` reaches `capture.end()` synchronously through `closeDictationAudio`. So the last read in `end` could raise an interruption that called straight back into `end`, whose own last read was refused for the same reason the first was — the shell has no capture — and the recursion issued bridge reads until the page ran out of memory. The pin crashed the test worker with `JavaScript heap out of memory` before the fix. A second `end` returns the first one's promise, assigned before anything can await so a handler re-entering from inside the read finds it set. `begin` clears it, because the seam is memoised per client and the next dictation on the same screen has to be able to drain — pinned by a case that ends, starts again and reads the new audio. Claude-Session: https://claude.ai/code/session_01JNnE9qzUZMMnqpZWCqM3nb * fix(mobile): give back a wake tag whose activation landed after dispose `held.add(tag)` ran after the device call, so a `dispose()` falling between the activate and its reply walked an empty set and the tag was recorded afterwards. Nothing walks that set again, so the screen stayed awake for the app's lifetime — and the page is a document that can be swiped away mid-dictation, which is exactly when that window is open. A tag that lands after the session ended is deactivated on the spot and reported to the page as not held. Claude-Session: https://claude.ai/code/session_01JNnE9qzUZMMnqpZWCqM3nb * fix(mobile): swallow the native audio session's shutdown failures The contract says `end` never rejects, and the native one did: it is `async`, so a throwing JSI binding rejects rather than throws. Every caller reaches it as `void capture.end()` inside a synchronous `try`, which cannot see a rejection — so a device that would not stop recording left an unhandled rejection, and the cleanup the `try` was written to protect was never what was at risk. `release` was worse in kind: it runs bare in the unmount path, so a throwing `tearDown` took the wake tag's release and the desktop's cancel with it. Both log and continue. Pinned behaviourally against an engine that refuses to stop and a tear-down that throws; the source case that claimed the hook's `try` was the guard now says where the guard actually is. Claude-Session: https://claude.ai/code/session_01JNnE9qzUZMMnqpZWCqM3nb * test(mobile): pin the order stop() hands the tail over in The tail fix depends on `await capture.end()` running before chunks stop being accepted, and nothing held that: the source check only asserts `end()` precedes `Promise.allSettled`, which both orders satisfy, so reversing the two lines left the whole mobile suite green while the page silently dropped the end of every sentence. Driven against a capture whose `end()` delivers a chunk — what the page's seam does and the device's never does, which is why only this case can tell the orders apart. It asserts the tail reaches the desktop as `speech.dictation.chunk`, with those bytes, before `finish`; reversed, no chunk is sent at all. The mocked seam is one object for the module's life, because the hook keys its teardown effect on the capture's identity: a seam returning a fresh object per render cancels the dictation on every render. Both real seams are stable. Claude-Session: https://claude.ai/code/session_01JNnE9qzUZMMnqpZWCqM3nb * fix(mobile): keep a wake tag recorded until the device really drops it `held` was cleared whether or not `deactivate` succeeded, so a tag the device refused was forgotten. The page's owner queues exactly that failure for a retry (`pendingCleanupTags`), and the retry arrives here as another `active: false` — which a shell that had already forgotten the tag answered without calling anything, leaving the native tag on for the life of the app. The boundary: the set means "the device still has this tag", not "the page asked for it". That keeps the reason the set exists — never call `deactivateKeepAwake` for a tag this shell never took, since its failure would read to the page as a wake lock it could not drop — while letting every retry for a live tag through. Always reaching the device regardless of the set would have traded the second property for the first. Pinned with a device that refuses once and then accepts, and with a dispose whose deactivation is refused. Claude-Session: https://claude.ai/code/session_01JNnE9qzUZMMnqpZWCqM3nb * fix(mobile): tear the audio device down for a start that lost the race A start that came back after `dispose()` returned without ending the engine, and the local `end()` is a no-op with no capture, so nothing tore down the session `initialize()` had just brought up. Nobody else would: the dispose had already run and no capture was ever recorded. The device's audio session stayed up for the life of the app. It ends the engine on that path now. Pinned on both places the race can be lost — inside the permission prompt and inside the open itself — each asserting exactly one teardown and no live listeners. Claude-Session: https://claude.ai/code/session_01JNnE9qzUZMMnqpZWCqM3nb * test(mobile): drive the stop-tail pin through the typed fake client The hand-rolled client fake needed three type assertions to stand in for an `RpcClient`, which the changed-code quality gate refuses on new lines. It drives `createFakeRpcClient` instead — a real one — answering each request as the hook makes it, and reads the chunk's base64 by narrowing rather than asserting. Re-confirmed the pin still reds on the reversed order after the rewrite. Claude-Session: https://claude.ai/code/session_01JNnE9qzUZMMnqpZWCqM3nb * fix(mobile): clear the page capture's end when it settles A finished `end` stayed in `ending` until `begin` cleared it, and `begin` is not guaranteed to run between two ends: `open` starts the shell recording, and a start that goes stale after the hook sets `activeIdRef` cleans up through `capture.end()` without ever committing. The second end answered from the first one's settled promise and never issued `native.audio.stop`, leaving the shell holding a live microphone. Cleared on settle instead, and only for its own flight. The re-entrancy the latch was really for happens while the promise is still pending, so guarding the flight is enough. `release` keeps its own flag rather than a settled `ending`, which now clears itself and would unlatch it. Claude-Session: https://claude.ai/code/session_01JNnE9qzUZMMnqpZWCqM3nb * fix(mobile): serialise the shell's wake tags and record them before compensating Two ways the device could end up holding a tag nothing would ever give back. A release that arrived while its own activate was still in flight read `held` before the activate had recorded anything, found nothing, deactivated nothing and reported the tag off; then the activate landed and the device stayed on. `held` is read and written across an await, so operations are now chained per tag. Per tag rather than per server, so one hanging device call cannot hold up another dictation. And on the late-tag path, a compensating deactivate the device refused was swallowed while `{ active: false }` was returned: the device still held the lock, `held` lacked the tag, `dispose` had already walked the set, and the page believed an activate had succeeded. The tag is now recorded as soon as `activate` resolves, deleted only once the device has really dropped it, and a refusal rejects so the caller's retry path runs. Claude-Session: https://claude.ai/code/session_01JNnE9qzUZMMnqpZWCqM3nb * fix(mobile): let dispose re-read the tag set once its turn comes Queueing dispose behind each tag's own operations introduced a call the module says it does not make: a release already in flight can give the tag back before dispose runs, and deactivating an unheld tag is a native call whose failure would read to the page as a lock it could not drop. Re-reads the set when the queued action runs rather than trusting what it held when dispose was called. Claude-Session: https://claude.ai/code/session_01JNnE9qzUZMMnqpZWCqM3nb * test(mobile): pin the dictation census to the routes C4.4 registered Claude-Session: https://claude.ai/code/session_01JNnE9qzUZMMnqpZWCqM3nb