mirror of
https://github.com/stablyai/orca.git
synced 2026-09-22 16:02:32 +00:00
Jinwoo Hong
86b93e02a7
* feat(mobile): give the microphone its own screen lock (OTA phase C, ruling 36) An open microphone holds the screen; a closed one gives it back. The lock lives in the device-side capture on both hosts — the shell's `native.audio.start|stop` handler and the native seam — so the page never decides anything about the screen. One tag per capture, minted by the module that owns the mic. Both captures give it back on every close path: a stop, a page session ending with the capture open, a device that would not begin, and an engine that throws after the capture is open, which now ends the capture rather than leaving it live. Claude-Session: https://claude.ai/code/session_01JNnE9qzUZMMnqpZWCqM3nb * feat(mobile): carry the capture's tail on the stop reply (OTA phase C, ruling 36) `native.audio.stop` drains what the ring still holds into its own reply, so the page's `end()` is one verb: stop, hand the bytes on, done. The drain, await and read-once-more ordering goes with it, and so do `ending`, `reading` and `released` — three variables that existed only to order a last read against the stop and to stop a refused read re-entering `end`. The tail fields default rather than being required: the page updates over the air and the shell does not, so a page this new can meet a shell that answers `stopped` alone. That dictation loses its tail where a required field would have lost it the stop. The heap case from PR D's bot round cannot recur: `end` issues no read, and a stop reply carries no interruption, so the lane that re-entered is gone. Claude-Session: https://claude.ai/code/session_01JNnE9qzUZMMnqpZWCqM3nb * refactor(mobile): drop the dictation finish id, which ordered nothing `finishingIdRef` tracked the dictation a stop was finishing, and every state it could name was already named: `cancel`, a disable, an unmount and a newer start each bump the generation or clear the active id, so the finish guard answered the same either way. Its one distinguishing arm released pending audio bytes for a dictation whose budget `closeDictationAudio` had just reset, and could subtract those bytes from a newer dictation's reserve. `acceptingChunksRef` stays: it is what stops a late microphone event being sent after the capture handed over its tail and before the finish goes out. `pendingChunksRef` stays: `stop` awaits it so the finish cannot overtake the last chunk send. The finish guard is pinned by a case that cancels while the finish is in flight; neutered, it reds. Claude-Session: https://claude.ai/code/session_01JNnE9qzUZMMnqpZWCqM3nb * feat(mobile): delete the page's wake-lock seam (OTA phase C, ruling 36) The page never names the screen now. `native.wakelock.set` is gone with its schemas, its shell server, its grant rows and its harness entry; so are the page's keep-awake owner, the Android foreground re-acquire, and the `DictationKeepAwakeDevice` the capture contract carried. One module holds the screen — the device calls the microphone's capture makes — and both device-side captures share its one tag, because there is one microphone. Deleted: native-wakelock.ts (120), native-wakelock.test.ts (140), mobile-dictation-keep-awake.ts (248), mobile-dictation-keep-awake.test.ts (440), mobile-dictation-foreground-keep-awake.ts (78). With the tag pools gone, the desktop-start flow has one stale check instead of two, no startup budget to wait out and nothing to release. A source-scanning census pins it: no module under mobile/src or mobile/app but the one owner imports expo-keep-awake, and nothing anywhere names the retired verb. It reports the file and line, and checks the owner does import the package so the absence is the rule holding and not the match missing. KNOWN RED, reported and not recorded over: 25 golden cases in the speech.* families fail. The recorder adapter had to drop its keep-awake owner, which moves `adapterSha256` for every golden that mounts it, and the deleted owner's id minting shifts the deterministic random sequence, so the recorded `dictationId` values move too. No speech.dictation.* param, reply or operation changed. Awaiting the lead's call on a scoped re-record. Claude-Session: https://claude.ai/code/session_01JNnE9qzUZMMnqpZWCqM3nb * test(mobile): re-record the corpus after the wake-lock deletion Baseline bumped to491eb35b4dand the corpus recorded against it. Every changed line in `goldens/` falls into six classes and nothing else: baseline 1574 lines 787 goldens lockfileSha256 1574 lines 787 goldens adapterSha256 38 lines 19 goldens scenarioSha256 8 lines 4 goldens dictationId shift 728 lines 4 goldens keep-awake effects + renumbering 63 lines 2 goldens ---- 3985 lines, which is the whole diff `recorderSha256` is untouched: no recorder module outside `adapters/` moved. The id shift is attributable arithmetic, not a behaviour change. The recording scheduler seeds `Math.random` with an LCG from seed 1; replaying it gives draw 1 `8ig2henseon` and draw 2 `dakoxjr8wun`. The deleted keep-awake owner minted its id from draw 1 during the hook's mount, so the dictation id took draw 2. With the owner gone the dictation id takes draw 1, which is why four scenario steps that pinned the literal value move with it. `adapterSha256` covers `speech.setup-sheet` as well as the three dictation families, because one adapter module hosts them all. The two goldens with vanished effects also renumber the ordinals after them, which is what the removal of an entry from a sequential counter does. `lockfileSha256` is provenance that `compareGolden` copies from the actual and never fails on. It moves in all 787 files because main's own `mobile/pnpm-lock.yaml` has moved since the corpus was last recorded; this branch does not touch it. Claude-Session: https://claude.ai/code/session_01JNnE9qzUZMMnqpZWCqM3nb * fix(mobile): drop the retired wake-lock grant C7.7's route row carried The merge brought in the session route's manifest entry, which names all four dictation grants including `native.wakelock.set`. This branch deleted that verb, so the row granted a page something the shell no longer serves. Ruling 32 item 6 said C7.7 takes the dictation grant from PR D's census in this merge; this is that. Nothing caught it automatically: the shell-side grant list is derived from the verb tuple and is already three, and the closure census holds a route to the grants it needs rather than refusing ones it does not. Claude-Session: https://claude.ai/code/session_01JNnE9qzUZMMnqpZWCqM3nb * test(mobile): repin the session-route closure at the measured 4,331 Measured on this merge rather than summed: 4,331 modules, 989 local. Main is red here on its own pin. `3cfb070294` measures 4,333 / 991 against a committed 4,330, three modules this branch never touched — measured in a throwaway worktree detached at that commit, with the same generators run. This merge measures 4,331 / 989, and diffing the two local lists gives the difference exactly: `mobile-dictation-keep-awake.ts` and `mobile-dictation-foreground-keep-awake.ts` leave, and nothing joins. So the branch's own effect is the -2 ruling 36 implies, and repinning to the measurement is also what takes main's closure test green again. Claude-Session: https://claude.ai/code/session_01JNnE9qzUZMMnqpZWCqM3nb * test(mobile): repin the session-route closure at 4,331 on the merge with #22067 Measured on this merge: 4,331 modules, 989 local, against main's freshly repinned 4,333 / 991 at `3cfb070294`. Both provenances kept. #22067 names the three `src/shared` modules #21924 pulled into the page closure, which is what made main's earlier 4,330 stale; this branch's own -2 is the page's wake-tag owner and its Android foreground re-acquire, deleted by ruling 36. Diffing the two local lists gives exactly those two leaving and nothing joining, so the number is a reading rather than 4,333 minus an argument. Claude-Session: https://claude.ai/code/session_01JNnE9qzUZMMnqpZWCqM3nb * fix(mobile): close the capture when the desktop start fails The hook opens the microphone before it asks the desktop for a session, so a refused session left the mic open — and, since the screen rides the mic, the display held until the user cancelled, retried, or the screen unmounted. The failure arm now runs the same `rollbackRecordingStart` the commit failure does, because "undo the capture this start opened" is one thing and the hook owns it; guarded like that arm, so a seam that throws on the way down cannot take the desktop cancel with it. Red-first on both hosts. Natively, a new test drives the real seam under the engine and keep-awake mocks: the refusal used to leave `initialize` with no `toggleRecording(false)` and a held screen. On the page, the mic control's own test over the port pair saw `native.audio.start` with no `native.audio.stop`. Two unit cases pin the call itself, including for a start nobody will report. Ruling 36's own words: mic closed means released. This closes the mic rather than adding a release beside it. Claude-Session: https://claude.ai/code/session_01JNnE9qzUZMMnqpZWCqM3nb * test(mobile): re-record after the failed-start capture close Baseline bumped toa1bdee9af9and the corpus recorded against it. The whole diff is two classes: baseline 1574 lines 787 goldens content 74 lines 1 golden `lockfileSha256`, `adapterSha256`, `scenarioSha256` and `recorderSha256` do not move: no lockfile, adapter, scenario or recorder module changed. The one content golden is `matrix-speech.dictation-start-speech.dictation.start-1`. Its failure partitions now carry a `rollback-recording` effect at ordinal 3, which is the capture being closed, and the `speech.dictation.cancel#1` entries after it renumber from 3,4 to 3,4,4,5 — the pool holds one entry per distinct content, so a partition whose ordinal moved stops sharing an entry with the one it used to match. Removing the new effect and ignoring ordinals makes the two recordings identical, checked by dereferencing every hash rather than by reading the diff. Claude-Session: https://claude.ai/code/session_01JNnE9qzUZMMnqpZWCqM3nb * fix(mobile): only the owning start rolls its capture back There is one capture seam and it carries no start identity, so round 1's rollback let a stale start's rejection end a live newer dictation: A opens the capture and waits on the desktop, the user cancels, B starts and is recording, A's request finally rejects and ends B's microphone and hands back B's screen. The rollback now runs only while this start is still the current one, which is what `wasCurrent` on the line above already reads; a stale failure still cancels its own desktop session and touches nothing else. Past the generation the capture was either already ended by whatever superseded this start, or belongs to the one that did. Red-first on both hosts, driving that exact sequence rather than a spy: the native test over the real seam saw the screen go `+ - + -`, and the page's mic-control test over the port pair saw a fourth `native.audio.` verb after B was recording. Both now end with B still holding what it took. The mirror image is covered and now pinned at host level too: A's request resolving late does not commit A over B, because the stale check after the desktop start returns through `cancelStaleStart`, which cancels A's session without touching the capture. Claude-Session: https://claude.ai/code/session_01JNnE9qzUZMMnqpZWCqM3nb * test(mobile): re-record after the stale start stopped rolling back Baseline bumped toe17b2cf603. Two classes, the whole diff: baseline 1574 lines 787 goldens content 74 lines 1 golden `lockfileSha256`, `adapterSha256`, `scenarioSha256` and `recorderSha256` do not move. The one content golden is `matrix-speech.dictation-start-speech.dictation.start-1`, whose scenario is the superseded start, so every partition in it is a stale one. The `rollback-recording` effect round 1 put there is gone, and the `speech.dictation.cancel#1` entries fold back from 4 to 2 as the ordinals after it renumber — the pool holds one entry per distinct content, so partitions whose ordinals agree again share an entry again. Dropping that effect from the superseded partitions and ignoring ordinals makes the two recordings identical, checked by dereferencing every hash. `speech-desktop-start-recording-failed` is untouched: that start still owns its capture at failure time, so it still rolls back. Claude-Session: https://claude.ai/code/session_01JNnE9qzUZMMnqpZWCqM3nb
Orca Mobile
React Native companion app for Orca. Monitor worktrees, view terminal output, and send commands from your phone.
Local development uses two processes:
- Orca desktop/Electron from the repo root. This hosts the mobile WebSocket RPC server on port
6768. - Expo Metro from
mobile/. This serves the React Native app on port8081.
Unless a command says otherwise, run mobile app commands from the mobile/ directory.
Prerequisites
- Node.js 24+
- pnpm
- Xcode and/or Android Studio tooling for simulator or device builds
- Expo Go on your phone, or a development client build when native modules are needed
- Phone and desktop on the same LAN when testing a physical phone
Start Desktop Orca
From the repository root:
pnpm install
pnpm dev
Confirm the mobile RPC server is listening:
lsof -nP -iTCP:6768 -sTCP:LISTEN
Restart pnpm dev after changing Electron main-process code. Metro hot reload only applies to the mobile JavaScript bundle.
Start The Mobile App
cd mobile
pnpm install
pnpm start
Scan the Expo QR code with your phone's camera on iOS, or Expo Go on Android.
For a native dev-client build:
pnpm exec expo run:android
pnpm exec expo run:ios
pnpm start --dev-client
Pair With Desktop Orca
- Open Orca desktop.
- Go to Settings > Mobile.
- Scan the pairing QR code from the mobile app.
- Confirm the mobile host endpoint is
ws://<desktop-ip>:6768.
For the Android emulator, use ws://10.0.2.2:6768. For a physical phone, use the desktop LAN IP, for example ws://192.168.0.179:6768.
If the phone has a stale host entry, remove it from the app and pair again.
Development Paths
Android Phone
- Install Expo Go from Google Play
- Run
pnpm start, scan QR with Expo Go - For native modules:
pnpm exec expo run:android - Run with
pnpm start --dev-client
iOS Simulator
- Install Xcode from the App Store
- Run
pnpm start --iosto open in iOS Simulator
Physical Phone Debugging
The phone can be inspected through the connected device tooling:
orca snapshot --json
orca click --element @e3 --json
orca fill --element @e1 --value "ls" --json
orca screenshot --json
Use snapshot first to find the current element refs, then click/fill those refs. After mobile file edits, Metro usually hot reloads automatically, but navigating out of and back into the session screen can be useful because it re-runs terminal.subscribe.
Terminal Streaming Repro Without A Phone
Use this when terminal output does not render on device and you need to split server streaming bugs from WebView/UI bugs:
cd mobile
ORCA_MOBILE_WS_URL=ws://127.0.0.1:6768 pnpm exec tsx scripts/test-subscribe.ts <deviceToken> <serverPublicKeyB64>
You can pass a worktree selector as the third argument:
pnpm exec tsx scripts/test-subscribe.ts <deviceToken> <serverPublicKeyB64> "id:<worktreeId>"
pnpm exec tsx scripts/test-subscribe.ts <deviceToken> <serverPublicKeyB64> "path:/absolute/worktree/path"
pnpm exec tsx scripts/test-subscribe.ts <deviceToken> <serverPublicKeyB64> "name:my-worktree"
The expected result includes:
streamSawMarker: true
readSawMarker: true
If this repro fails, debug the desktop runtime/PTY path before the mobile WebView. If it passes but the phone is blank, debug the session screen or TerminalWebView readiness/queueing path.
Terminal Color Repro Without A Phone
Use this when terminal colors disappear after switching tabs. Open a Claude Code terminal and at least one other terminal in the target worktree, then run:
cd mobile
ORCA_MOBILE_WS_URL=ws://127.0.0.1:6768 pnpm exec tsx scripts/repro-terminal-colors.ts \
<deviceToken> <serverPublicKeyB64> "id:<worktreeId>"
The script captures terminal.subscribe snapshots in an A → B → A sequence and writes raw snapshots to mobile/terminal-color-repro/. If the two A snapshots have different sgrColor counts, the desktop snapshot changed during the switch. If they match, the ANSI color data is still present and the bug is in mobile replay/rendering.
Validation
Run these checks before committing mobile terminal changes:
cd mobile
pnpm exec tsc --noEmit
pnpm run check:tests-typecheck
pnpm lint
cd ..
pnpm typecheck:node
tsc --noEmit reads tsconfig.json, which excludes test files so Metro never bundles them.
tsconfig.test.json puts them back, and pnpm run typecheck:tests shows their errors in full.
check:tests-typecheck is the gate over it: a ratchet against tests-typecheck-baseline.txt, the
127 test files that do not typecheck yet. It fails when a file that checks today stops checking,
and when a baseline entry starts checking (prune it with
node scripts/check-tests-typecheck-ratchet.mjs --prune). The list may only shrink.
The same gate censuses the program first: every *.test.ts(x) on disk must be in it, or named in
the script's TESTS_OUTSIDE_PROGRAM with a reason. Without that, a test excluded from
tsconfig.test.json — or a Foo.test.tsx shadowed by a Foo.test.ts beside it, which a wildcard
include drops for the higher-priority extension — would leave the ratchet silently.
Protocol Version Compatibility
Mobile and desktop talk over a versioned protocol. Because mobile updates lag desktop by 24-48h via the App Store, both sides exchange version numbers on status.get so a genuinely incompatible combo can hard-block instead of silently misbehaving.
Constants live in two files (Metro can't resolve outside mobile/):
src/shared/protocol-version.ts—DESKTOP_PROTOCOL_VERSION,MIN_COMPATIBLE_MOBILE_VERSIONmobile/src/transport/protocol-version.ts—MOBILE_PROTOCOL_VERSION,MIN_COMPATIBLE_DESKTOP_VERSION
Today all four are set so evaluateCompat always returns { kind: 'ok' } — nothing blocks. The wire format is in place to flip a switch when needed.
When to bump
Bump DESKTOP_PROTOCOL_VERSION (and the mobile mirror MOBILE_PROTOCOL_VERSION when relevant) for breaking changes:
- Removed RPC method or required parameter that mobile uses
- Changed meaning (units, nullability) of an existing field mobile reads
- Changed encryption, framing, or auth handshake
Do not bump for additive changes:
- New RPC methods
- New optional fields on existing methods
- New event types in
terminal.subscribe
Set MIN_COMPATIBLE_MOBILE_VERSION (kill-switch) when desktop ships a change that requires a minimum mobile version to function safely. Same for MIN_COMPATIBLE_DESKTOP_VERSION from the mobile side.
When a verdict is blocked, mobile/src/components/ProtocolBlockScreen.tsx renders a screen pointing the user at either the App Store (mobile too old) or GitHub Releases (desktop too old).
To exercise the block screen locally: set MIN_COMPATIBLE_DESKTOP_VERSION = 999 in mobile/src/transport/protocol-version.ts, rebuild, pair to any desktop. Revert before merging.
Mock Server
Develop the mobile app without a running Orca desktop instance:
pnpm mock-server # starts mock WebSocket server on port 6768
Connect from the app using endpoint ws://localhost:6768 and token mock-device-token.
Environment variables
MOCK_NATIVE_CHAT=1— serve the native-chat scenario (one live agent tab, empty transcript, image upload) instead of the default terminal fixtures.MOCK_CHAT_AGENT=omp— withMOCK_NATIVE_CHAT=1, present an OMP tab and four decoded transcript messages, including a tool call and result, instead of the default Claude scenario. It deliberately omitstranscriptPathto exercise legacy-hook readability discovery; current OMP hooks may report a path.MOCK_SERVER_KEY_FILE— persist the server keypair across restarts so a paired device keeps its public-key pin. A missing or invalid file is re-keyed with a warning, which forces a re-pair.
Scenario control files
Read on every request, so behaviour can be flipped mid-session without a restart (a restart would re-key E2EE and force a re-pair). Write the mode into the file, or delete it for the default.
MOCK_SEND_MODE_FILE(defaultorca-mock-send-modein the system temporary directory) —accept(default) accepts the send,errorfails it withmobile_input_floor_unavailable, anything else reports the send as rejected.MOCK_TERMINAL_LIST_MODE_FILE(defaultorca-mock-terminal-list-modein the system temporary directory) —omitreturns an empty terminal list,otherreturns a list that omits the chat handle, anything else lists it.MOCK_TERMINAL_STREAM_MODE_FILE(defaultorca-mock-terminal-stream-modein the system temporary directory) —deadanswers a subscribe withsubscribedthenend(a gone PTY), which is what exercises the rearm bound and terminal prune; anything else streams normally.
Connecting to Real Orca
- Start Orca desktop with WebSocket transport enabled
- In Orca, go to Settings > Mobile and scan the QR code with this app
- The QR encodes the connection endpoint, device token, and TLS fingerprint
Project Structure
mobile/
├── app/ # Expo Router screens (file-based routing)
│ ├── _layout.tsx # Root layout with navigation stack
│ ├── index.tsx # Home screen — paired hosts list
│ └── pair-scan.tsx # QR code scanning screen
├── src/
│ ├── terminal/ # Terminal WebView and xterm bridge
│ └── transport/ # WebSocket RPC client
├── scripts/
│ ├── test-subscribe.ts # Desktop streaming repro without a phone
│ └── mock-server.ts # Standalone mock WebSocket server
└── assets/ # App icons and splash screen