mirror of
https://github.com/stablyai/orca.git
synced 2026-09-22 00:02:31 +00:00
Jinwoo Hong
d5dc7b9cf8
* fix(mobile): budget the mobile terminal snapshot on the bytes it serializes to (OTA phase C, C7.3, ruling 1) The desktop trims a mobile snapshot to 512 KiB of raw terminal text. A client reading it through the page bridge measures the serialized event against a 640 KiB frame cap, and an ANSI snapshot is mostly ESC bytes, each of which JSON spends six on. Measured here on a colour-dense 80-column screen: the raw budget hands back 465,766 bytes that serialize to 669,268 — 102.1% of the cap — so `deliver` answers `cancel(id, 'overflow')` and the terminal is dead before its first live byte, with no recovery that does not reproduce it. `terminal.subscribe` gains an optional `snapshotByteBudget`. A subscriber that sends one is trimmed against the JSON its payload will really cost: the escaped text, plus the metadata it cannot bound from its own side — a path, the OSC-link list, the pending escape tail. A subscriber that sends none, which is every socket client and every older page, keeps the raw byte rule exactly. No negotiation, and none is needed: the field is additive and optional, so an older desktop ignores it and trims as it always did. The page then still has a snapshot over its cap, the shell still ends the stream with `overflow` (C0.3 stands), and the terminal renders its stream-error state rather than a blank pane. The page derives the number from the cap less the event envelope rather than writing it down, so a cap that moves takes the budget with it. Claude-Session: https://claude.ai/code/session_01JNnE9qzUZMMnqpZWCqM3nb * feat(mobile): hold and coalesce terminal output instead of ending the stream on the window (OTA phase C, C7.3, ruling 2) The shell's backpressure window ends a stream when the page falls 4 MiB behind. That is right for a stream whose reader can survive a gap and wrong for a terminal, whose reader cannot see the hole a dropped chunk leaves — and the window does not wait for a page to go wrong. Measured by the design: the host produces 70.3 MiB/s of JSON and real xterm applies 2.2 MiB/s, so an ordinary `cat` crosses the window in 62 ms. Replayed here through the real ledger against a page draining at that rate, a 5 MB transcript ends the stream after 85 of 107 chunks plain and after 40 of 107 under `grep --color`. Keyed by method on the shell, since the page cannot pick its own window, `terminal.subscribe` now holds what it cannot send, merges consecutive output in escaped bytes under the frame cap, and delivers as the page acks. Nothing is dropped: merging concatenates, and the only exit that loses bytes is ending the stream, which the page is told about. Both transcripts now arrive whole and in order, in 104 and 81 frames, with the largest frame at 622,551 bytes against the 655,360-byte cap. It ends only on the two things that are not slowness: a page that has acked nothing for 20 s, an order of magnitude above the 1.9 s a full window takes to drain, and a backlog past 32 MiB, which at that drain is about 15 s of catching up. Both reach the page as `overflow`, because the shell is the installed app and its page comes from the desktop, so a reason the page's reader has never heard of is a frame it drops rather than an end it acts on. Which one fired, the coalesced-frame count and the peak pending bytes go to the diagnostic log, which is the device proof's only oracle for any of this. Every other stream keeps the byte window exactly, and an event over the frame cap still ends any stream, terminal or not (C0.3). The landed window cases now name a stream the window still governs, so the two rules are never read off each other. Claude-Session: https://claude.ai/code/session_01JNnE9qzUZMMnqpZWCqM3nb * fix(mobile): narrow the event arm the backlog replay reads A binary event carries no `payload`, so the tests-typecheck ratchet refused the reach into it. Claude-Session: https://claude.ai/code/session_01JNnE9qzUZMMnqpZWCqM3nb * refactor(mobile): narrow the snapshot serializer to the buffer source it reads The changed-code casting gate refused the test's stub runtime, and it was right to: a service-wide type for a function that calls one method is what made the stub need an assertion. The parameter now says what it needs, and the fixture path is no longer one a machine-path grep reads as a leaked local checkout. Claude-Session: https://claude.ai/code/session_01JNnE9qzUZMMnqpZWCqM3nb * fix(mobile): measure the snapshot budget by building the payload, not by summing fields (OTA phase C, C7.3, ruling 14) Round one summed the escaped text and four metadata fields. The payload a bridged client assembles carries nine more — `kind`, `cols`, `rows`, `requestId`, `displayMode`, `reason`, `seq` and both truncation flags — plus the `type` and `streamId` it adds, the `serialized` key and the object's own braces. So a snapshot this host accepted at exactly the budget, with `truncatedByByteBudget` false because nothing had trimmed it, published over the cap and the stream ended with `overflow` before a byte was painted. Measured here on a screen sized to land exactly on round one's budget: the published payload is 655,446 bytes against a 655,273-byte budget, 173 over, and the frame it makes is over the 640 KiB cap by the same amount. The metadata is now built by one function that `sendSnapshotFrames` and the budget both call, and the budget stringifies the payload that function produces. Nothing is summed and nothing is estimated, so a field added to the frame is paid for by the budget the moment it is sent. Where a value is not yet known — the truncation flags, and `seq` or `requestId` at a site that has not fixed them — it is measured at the widest `JSON.stringify` can write it, which is a bound rather than a guess, and forcing `seq` to a number also opens the three fields it gates so those are counted too. The budget therefore travels with the publication fields, because the payload cannot be built without them. On the page, the event envelope is now derived in one place in the protocol module and read by both the snapshot budget and the shell's own merge budget, so the two cannot drift; the page pins the number it sends and the host's cases name that pin, since the two programs cannot import from each other. The case that re-implemented the host's measure is gone: it could not have seen this, because it was the same arithmetic twice. Claude-Session: https://claude.ai/code/session_01JNnE9qzUZMMnqpZWCqM3nb * fix(mobile): arm a held terminal's silence clock only while something is pending (OTA phase C, C7.3) The invariant is "armed implies waiting on the page", and round one broke it in the one direction that kills: an ack re-armed the clock and the drain that followed emptied the queue without clearing it. A terminal that had delivered every byte and gone quiet — which is what a terminal does between commands — would die on `overflow` twenty seconds later. The clock is now synchronised after every change to the queue, so it is armed exactly while something is held. A rule that only ever arms is a rule that only ever ends more streams. Red-first: with round one's arming, an idle stream whose queue has drained still reports its clock armed, and firing it ends a healthy terminal. Claude-Session: https://claude.ai/code/session_01JNnE9qzUZMMnqpZWCqM3nb * test(mobile): pin the held-stream cases the rulings name (OTA phase C, C7.3) Six cases nothing covered. Two subscriptions on one shell keep separate backlogs, so a busy terminal cannot end a quiet one. A stream the page unsubscribed mid- backlog posts nothing after, and neither does one that has already ended, however much was still held. A payload that is not output breaks a merge run and keeps its place, because a resize is state the reader applies in order. And the budget boundary is checked on the side that enforces it: a payload at exactly the number the page asks the desktop for is delivered inside the cap, and one the cap cannot hold ends the stream under C0.3. The replay no longer acks unconditionally in its catch-up loop. That was the page behaving better than a page can — it acks on reading frames — and it is what hid the silence clock left armed over an empty queue. The held-stream cases close the window on its frame count rather than on four megabytes of string work. Claude-Session: https://claude.ai/code/session_01JNnE9qzUZMMnqpZWCqM3nb * refactor(mobile): give the event-envelope derivation its own module (OTA phase C, C7.3) `bridge-envelope.ts` is at its line cap and is the protocol's schemas; what a frame costs around its payload is a derivation over them, and two budgets read it — the snapshot the page asks the desktop for, and the output the shell merges. One module, so they cannot drift and neither file is pushed over its limit. Claude-Session: https://claude.ai/code/session_01JNnE9qzUZMMnqpZWCqM3nb * test: narrow the budget fixtures instead of asserting them The changed-code casting gate refused six `as NonNullable<...>` in the new budget cases, and it was right to: a fixture that serialized nothing is a broken case rather than a null to assert away. Claude-Session: https://claude.ai/code/session_01JNnE9qzUZMMnqpZWCqM3nb * refactor: give the snapshot payload shape its own module (OTA phase C, C7.3) `terminal-snapshot-publication.ts` crossed the root config's 300-line cap, which mobile's own lint does not apply and CI does. The frame's shape and what it costs a client reading it as one payload is a description the budget and the sender both need, so it is the part that leaves. Claude-Session: https://claude.ai/code/session_01JNnE9qzUZMMnqpZWCqM3nb * fix: empty a snapshot the budget cannot fit instead of posting it over (OTA phase C, C7.3, ruling 15) Both trimming loops published the zero-row candidate whatever it measured, and zero scrollback is not a small screen: a wide colour-dense viewport still carries its 24 live rows. A capped subscriber could get one frame over its cap, end the stream on `overflow` and paint nothing — worse than a blank terminal, because a blank one repaints on the next byte of output and a stream that never opened does not reopen. Ruling 15: a budgeted subscriber gets that frame with its text emptied and `truncatedByByteBudget` true, never over and never refused. The raw rule keeps its fallback, so an older page and every socket client are served exactly what they were before. Below the metadata the frame must carry there is nothing left to give up, and that boundary is pinned rather than claimed away. The renderer loop is the same walk reached by a different caller and had no test at all; its runtime parameter is narrowed to the two methods it reads so a case can stub it without a cast. Claude-Session: https://claude.ai/code/session_01JNnE9qzUZMMnqpZWCqM3nb * fix(mobile): report what a held terminal stream did instead of calling it an outlived view The backlog report had no branch in the reporter, so it fell through to the "a view outlived its host" warn and every field it exists to carry was discarded. The key made it worse: keyed by kind alone, one backlog per host was ever logged, and a shell holds one stream per open terminal. That report is the only oracle the coalescing rule has. Nothing crosses to the page saying how much was held or how many frames its bytes arrived inside, and both ways a held stream dies reach the page as `overflow`, because a reason its reader has never heard of is a frame it drops. In production the two rules were indistinguishable. They are now a line each, per stream. Claude-Session: https://claude.ai/code/session_01JNnE9qzUZMMnqpZWCqM3nb * test: give the renderer fixture the source its serializer returns `serializeRendererTerminalBuffer` answers `renderer`, and vitest does not typecheck, so the stub's `headless` passed every run and failed the node typecheck instead. Claude-Session: https://claude.ai/code/session_01JNnE9qzUZMMnqpZWCqM3nb * fix: budget the frame the publication actually sends (OTA phase C, C7.3) The budget and the publication were written out twice, five lines apart, and had drifted at every site: a budget for `{kind:'scrollback'}` approved a frame sent as `kind:'resized'` with a `reason` beside it, and the live module budgeted `pending-output-overflow` while sending `renderer-mount-ready`. It held only because the padded `requestId` and `seq` are absent from those frames and more than covered the difference. Each site now builds one object and hands it to both. `displayMode` cannot travel that way and was a third under-measure nobody had named: the subscribe flow re-reads it from the runtime after the snapshot is serialized and before the frame is sent, so no caller can tell the budget which mode the publication will carry. It joins `seq`, `requestId` and the truncation flags as a field taken at its widest. The mode list resolves the constant to `never` if the runtime gains a mode it does not carry, so a new one is weighed here rather than found on a phone. Red-first needed a second attempt: the first fixture had trimming slack, so three extra bytes fit and the probe could not see the defect it was written for. The case now budgets a fixed screen at exactly its `auto` measure, where the margin is the whole of the test. One figure for the overshoot everywhere, with its basis: 169 bytes over the 655,360-byte cap on a frame carrying an 8-character request id, 247 with a 24-character one. Three places said 169 and one said 173. Claude-Session: https://claude.ai/code/session_01JNnE9qzUZMMnqpZWCqM3nb * refactor(mobile): delete two backlog guards no input can reach Both survived mutation because neither is reachable, and neither became reachable when I tried to write a case for it. `next` narrowed the merge ceiling to one frame, but its only caller, `drainTerminalBacklog`, has already narrowed it: the parameter is what one payload may occupy, not what the window holds, so the second narrowing could never change the answer. The parameter now says so and the class no longer needs the frame size at all. The bound still lives in the caller and is still covered: removing it there reds a delivery case. The merge run also compared stream ids, but a backlog belongs to one subscription and every `data` payload on it carries that subscription's single stream id, so the comparison could not fail. The run still stops at anything that is not output, which is reachable and pinned. Claude-Session: https://claude.ai/code/session_01JNnE9qzUZMMnqpZWCqM3nb * docs(mobile): record the invariant the deleted stream-id guard rested on The merge run compares no stream ids because it cannot need to: a backlog belongs to one subscription and every `data` payload reaching it carries that subscription's single stream id. Written down where the run is, because the thing that would break it is a change made somewhere else — multiplexing two streams onto one record would merge their output into one payload under the first id. Claude-Session: https://claude.ai/code/session_01JNnE9qzUZMMnqpZWCqM3nb
Orca Mobile
React Native companion app for Orca. Monitor worktrees, view terminal output, and send commands from your phone.
Local development uses two processes:
- Orca desktop/Electron from the repo root. This hosts the mobile WebSocket RPC server on port
6768. - Expo Metro from
mobile/. This serves the React Native app on port8081.
Unless a command says otherwise, run mobile app commands from the mobile/ directory.
Prerequisites
- Node.js 24+
- pnpm
- Xcode and/or Android Studio tooling for simulator or device builds
- Expo Go on your phone, or a development client build when native modules are needed
- Phone and desktop on the same LAN when testing a physical phone
Start Desktop Orca
From the repository root:
pnpm install
pnpm dev
Confirm the mobile RPC server is listening:
lsof -nP -iTCP:6768 -sTCP:LISTEN
Restart pnpm dev after changing Electron main-process code. Metro hot reload only applies to the mobile JavaScript bundle.
Start The Mobile App
cd mobile
pnpm install
pnpm start
Scan the Expo QR code with your phone's camera on iOS, or Expo Go on Android.
For a native dev-client build:
pnpm exec expo run:android
pnpm exec expo run:ios
pnpm start --dev-client
Pair With Desktop Orca
- Open Orca desktop.
- Go to Settings > Mobile.
- Scan the pairing QR code from the mobile app.
- Confirm the mobile host endpoint is
ws://<desktop-ip>:6768.
For the Android emulator, use ws://10.0.2.2:6768. For a physical phone, use the desktop LAN IP, for example ws://192.168.0.179:6768.
If the phone has a stale host entry, remove it from the app and pair again.
Development Paths
Android Phone
- Install Expo Go from Google Play
- Run
pnpm start, scan QR with Expo Go - For native modules:
pnpm exec expo run:android - Run with
pnpm start --dev-client
iOS Simulator
- Install Xcode from the App Store
- Run
pnpm start --iosto open in iOS Simulator
Physical Phone Debugging
The phone can be inspected through the connected device tooling:
orca snapshot --json
orca click --element @e3 --json
orca fill --element @e1 --value "ls" --json
orca screenshot --json
Use snapshot first to find the current element refs, then click/fill those refs. After mobile file edits, Metro usually hot reloads automatically, but navigating out of and back into the session screen can be useful because it re-runs terminal.subscribe.
Terminal Streaming Repro Without A Phone
Use this when terminal output does not render on device and you need to split server streaming bugs from WebView/UI bugs:
cd mobile
ORCA_MOBILE_WS_URL=ws://127.0.0.1:6768 pnpm exec tsx scripts/test-subscribe.ts <deviceToken> <serverPublicKeyB64>
You can pass a worktree selector as the third argument:
pnpm exec tsx scripts/test-subscribe.ts <deviceToken> <serverPublicKeyB64> "id:<worktreeId>"
pnpm exec tsx scripts/test-subscribe.ts <deviceToken> <serverPublicKeyB64> "path:/absolute/worktree/path"
pnpm exec tsx scripts/test-subscribe.ts <deviceToken> <serverPublicKeyB64> "name:my-worktree"
The expected result includes:
streamSawMarker: true
readSawMarker: true
If this repro fails, debug the desktop runtime/PTY path before the mobile WebView. If it passes but the phone is blank, debug the session screen or TerminalWebView readiness/queueing path.
Terminal Color Repro Without A Phone
Use this when terminal colors disappear after switching tabs. Open a Claude Code terminal and at least one other terminal in the target worktree, then run:
cd mobile
ORCA_MOBILE_WS_URL=ws://127.0.0.1:6768 pnpm exec tsx scripts/repro-terminal-colors.ts \
<deviceToken> <serverPublicKeyB64> "id:<worktreeId>"
The script captures terminal.subscribe snapshots in an A → B → A sequence and writes raw snapshots to mobile/terminal-color-repro/. If the two A snapshots have different sgrColor counts, the desktop snapshot changed during the switch. If they match, the ANSI color data is still present and the bug is in mobile replay/rendering.
Validation
Run these checks before committing mobile terminal changes:
cd mobile
pnpm exec tsc --noEmit
pnpm run check:tests-typecheck
pnpm lint
cd ..
pnpm typecheck:node
tsc --noEmit reads tsconfig.json, which excludes test files so Metro never bundles them.
tsconfig.test.json puts them back, and pnpm run typecheck:tests shows their errors in full.
check:tests-typecheck is the gate over it: a ratchet against tests-typecheck-baseline.txt, the
127 test files that do not typecheck yet. It fails when a file that checks today stops checking,
and when a baseline entry starts checking (prune it with
node scripts/check-tests-typecheck-ratchet.mjs --prune). The list may only shrink.
The same gate censuses the program first: every *.test.ts(x) on disk must be in it, or named in
the script's TESTS_OUTSIDE_PROGRAM with a reason. Without that, a test excluded from
tsconfig.test.json — or a Foo.test.tsx shadowed by a Foo.test.ts beside it, which a wildcard
include drops for the higher-priority extension — would leave the ratchet silently.
Protocol Version Compatibility
Mobile and desktop talk over a versioned protocol. Because mobile updates lag desktop by 24-48h via the App Store, both sides exchange version numbers on status.get so a genuinely incompatible combo can hard-block instead of silently misbehaving.
Constants live in two files (Metro can't resolve outside mobile/):
src/shared/protocol-version.ts—DESKTOP_PROTOCOL_VERSION,MIN_COMPATIBLE_MOBILE_VERSIONmobile/src/transport/protocol-version.ts—MOBILE_PROTOCOL_VERSION,MIN_COMPATIBLE_DESKTOP_VERSION
Today all four are set so evaluateCompat always returns { kind: 'ok' } — nothing blocks. The wire format is in place to flip a switch when needed.
When to bump
Bump DESKTOP_PROTOCOL_VERSION (and the mobile mirror MOBILE_PROTOCOL_VERSION when relevant) for breaking changes:
- Removed RPC method or required parameter that mobile uses
- Changed meaning (units, nullability) of an existing field mobile reads
- Changed encryption, framing, or auth handshake
Do not bump for additive changes:
- New RPC methods
- New optional fields on existing methods
- New event types in
terminal.subscribe
Set MIN_COMPATIBLE_MOBILE_VERSION (kill-switch) when desktop ships a change that requires a minimum mobile version to function safely. Same for MIN_COMPATIBLE_DESKTOP_VERSION from the mobile side.
When a verdict is blocked, mobile/src/components/ProtocolBlockScreen.tsx renders a screen pointing the user at either the App Store (mobile too old) or GitHub Releases (desktop too old).
To exercise the block screen locally: set MIN_COMPATIBLE_DESKTOP_VERSION = 999 in mobile/src/transport/protocol-version.ts, rebuild, pair to any desktop. Revert before merging.
Mock Server
Develop the mobile app without a running Orca desktop instance:
pnpm mock-server # starts mock WebSocket server on port 6768
Connect from the app using endpoint ws://localhost:6768 and token mock-device-token.
Environment variables
MOCK_NATIVE_CHAT=1— serve the native-chat scenario (one live agent tab, empty transcript, image upload) instead of the default terminal fixtures.MOCK_CHAT_AGENT=omp— withMOCK_NATIVE_CHAT=1, present an OMP tab and four decoded transcript messages, including a tool call and result, instead of the default Claude scenario. It deliberately omitstranscriptPathto exercise legacy-hook readability discovery; current OMP hooks may report a path.MOCK_SERVER_KEY_FILE— persist the server keypair across restarts so a paired device keeps its public-key pin. A missing or invalid file is re-keyed with a warning, which forces a re-pair.
Scenario control files
Read on every request, so behaviour can be flipped mid-session without a restart (a restart would re-key E2EE and force a re-pair). Write the mode into the file, or delete it for the default.
MOCK_SEND_MODE_FILE(defaultorca-mock-send-modein the system temporary directory) —accept(default) accepts the send,errorfails it withmobile_input_floor_unavailable, anything else reports the send as rejected.MOCK_TERMINAL_LIST_MODE_FILE(defaultorca-mock-terminal-list-modein the system temporary directory) —omitreturns an empty terminal list,otherreturns a list that omits the chat handle, anything else lists it.MOCK_TERMINAL_STREAM_MODE_FILE(defaultorca-mock-terminal-stream-modein the system temporary directory) —deadanswers a subscribe withsubscribedthenend(a gone PTY), which is what exercises the rearm bound and terminal prune; anything else streams normally.
Connecting to Real Orca
- Start Orca desktop with WebSocket transport enabled
- In Orca, go to Settings > Mobile and scan the QR code with this app
- The QR encodes the connection endpoint, device token, and TLS fingerprint
Project Structure
mobile/
├── app/ # Expo Router screens (file-based routing)
│ ├── _layout.tsx # Root layout with navigation stack
│ ├── index.tsx # Home screen — paired hosts list
│ └── pair-scan.tsx # QR code scanning screen
├── src/
│ ├── terminal/ # Terminal WebView and xterm bridge
│ └── transport/ # WebSocket RPC client
├── scripts/
│ ├── test-subscribe.ts # Desktop streaming repro without a phone
│ └── mock-server.ts # Standalone mock WebSocket server
└── assets/ # App icons and splash screen