* feat(native-chat): visual directive grammar and host read for a chat's visuals folder
A shared grammar for the ::orca-visual{file="..." title="..."} reply line,
the per-chat visuals folder location on the owning host, and the
agentSession.readVisual runtime method that reads one visual with lexical and
canonical containment, a 512 KiB bounded read and UTF-8 refusal.
* feat(native-chat): shared frame document for chat visuals
One string builder every client wraps a visual's HTML with: the policy
(CDN assets only, no fetch, frames, workers, forms or base rewrites),
the theme variables, and a prelude that reports height, routes links to
the parent and refuses navigation. Also the validated frame-to-parent
message reader and the live theme message.
* feat(mobile): render native-chat visuals inline in the phone's chat
A finished assistant reply's ::orca-visual line now shows the visual
inline, read from the chat's owning host through agentSession.readVisual.
The visual runs in an opaque sandboxed child of a trusted host document
inside the WebView; the app accepts only a token-checked height and an
http(s) link opened under user activation. Navigation away from the
visual's own document is refused, a dead web process reloads once, and
the visual opens full screen. The hybrid shell's page renders it sealed.
* feat(native-chat): render chat visuals inline and in the right sidebar
Native-chat assistant replies render a ::orca-visual{...} line as the chat's
HTML visual in an opaque, scripts-only sandboxed frame: CSP first, the host
frame navigation guard registered before content runs, live theme without a
reload, fitted height, links opened in the viewer's browser only from a real
gesture, lazy mount, and one muted line when the visual cannot be shown.
Open in sidebar shows the same frame in the right sidebar, widened while it
is open and restored after.
* fix(mobile): chat visuals use the shared shell; only the host page may message the app
Review round 1:
- use PR 1's shared visual shell and height governor instead of a second
builder; the app decides heights and pushes them to the host page
- react-native-webview patch: the message channel accepts only string
messages from the main frame on iOS and Android, so a visual in its
sandboxed child cannot reach it (or crash Android with a non-string)
- structured replies grow in place: while a turn works, a row holds back a
directive still being typed at its tail; finished lines mount
- links need child focus + activation, one per activation window
- a refused read takes the visual down and drops its cached bytes
- in-page anchors load; text spelling a placeholder renders no visuals
* refactor(native-chat): move the visual height governor to src/shared
The phone bundles only src/shared, so the governor the desktop frame uses
moves there unchanged and the mobile frame shares it.
* fix(native-chat): point the desktop and mobile frames at the moved governor
* fix(mobile): review round 2 for chat visuals
- host page relays only messages on the visual's own channel, as its own
copy, so a visual cannot push oversized fields through it; relay rate
halved; a link is validated before it uses up the link window
- the frame denies camera, microphone, geolocation, clipboard and display
capture; iOS media capture requests are denied
- react-native-webview patch: the iOS history-shim handler also accepts
only main-frame string messages
- only the newest assistant row of a working turn holds back an unfinished
directive; earlier finished rows show their visuals
- an error reply or older host is not a verdict: it keeps a visual on
screen and its cache, and retries; only a host refusal takes it down
* fix(mobile): a drag that starts on an inline visual scrolls the chat
iOS gives a scrollable frame its own scroll view, which took the drag;
the inline frame is sized to its content, so it no longer scrolls.
Full screen still does.
* fix(mobile): review round 3 for chat visuals
- hold back only the last block of the newest assistant row, and not
while a question or approval is open, so a visual followed by a tool
call or a pending question shows at once
- an older host (method_not_found) reads as unavailable without retries
- drop a stray @pnpm/exe lockfile block; only the patch hash changes
* test(mobile): list the visual frame's web sibling; mock it in the prompt-controller harness
* fix(mobile): an older desktop's mobile allowlist refusal reads as unavailable at once
* fix(native-chat): visual CI fixes, shared height governor, live-turn streaming hold
Registers agentSession.readVisual from the methods index so the structured
method file stays under its line budget, replaces reflective reads with checked
narrowing, moves the pure height governor to src/shared for mobile, and holds a
half-written directive tail while the turn works (structured text rows carry no
running state).
* fix(native-chat): harden the visual read and link opening
Re-checks after the open that the chat's visuals folder is still the real
directory at Orca's path, reports unexpected filesystem faults by code without
host paths, and lets one click in a visual open at most one page.
* fix(native-chat): keep visual lines out of plain-text reply surfaces; review fixes
One shared helper drops visual lines (outside fenced code) from reply text where
it becomes plain text: the structured status summary that feeds the sidebar row,
dashboard, notifications, phone rows and handoffs, and AI Vault reply previews.
Review fixes: height also counts a pinned body's overflow, only the live
frontier row holds a half-written visual line, the runaway-height stop needs the
same step repeated, and any host refusal evicts the cached revision.
* fix(native-chat): resolve the visuals folder without the removed journal-paths helper
Main removed the per-chat journal paths and the journal database's state
directory; the visuals folder keeps the same sha256 layout on its own and the
read method uses the profile state directory the chat host is opened in.
* fix(mobile): keep visual lines out of the worktree row and copied message text
A reply's ::orca-visual line renders only in the transcript. The
worktree list's agent row and the message actions sheet's copy text now
drop it with the shared helper; a reply that is only a visual falls back
to the prompt, as an empty one does.
* fix(native-chat): review round 2 fixes; copy a reply without its visual lines
Reply previews in Agent Session History drop visual lines per text part before
lines are folded; the frame adds a body's overflow only when the body really
overflows; fence tracking follows CommonMark closers and openers; the copy
button copies a reply without visual lines; a coded read fault keeps its cause.
* fix(native-chat): update the frame's theme ref after render; read the visuals folder pair at once
* test(native-chat): declare agentSession.readVisual on the cross-version agent-session surface
* fix(native-chat): copying a reply keeps its code blocks and indentation
Removing visual lines now closes only the gap each removal leaves, instead of
collapsing blank lines across the whole reply and trimming its indentation; the
visuals folder is checked parent first again so a broken path answers the same
way every time.
Orca Mobile
React Native companion app for Orca. Monitor worktrees, view terminal output, and send commands from your phone.
Local development uses two processes:
- Orca desktop/Electron from the repo root. This hosts the mobile WebSocket RPC server on port
6768. - Expo Metro from
mobile/. This serves the React Native app on port8081.
Unless a command says otherwise, run mobile app commands from the mobile/ directory.
Prerequisites
- Node.js 24+
- pnpm
- Xcode and/or Android Studio tooling for simulator or device builds
- Expo Go on your phone, or a development client build when native modules are needed
- Phone and desktop on the same LAN when testing a physical phone
Start Desktop Orca
From the repository root:
pnpm install
pnpm dev
Confirm the mobile RPC server is listening:
lsof -nP -iTCP:6768 -sTCP:LISTEN
Restart pnpm dev after changing Electron main-process code. Metro hot reload only applies to the mobile JavaScript bundle.
Start The Mobile App
cd mobile
pnpm install
pnpm start
Scan the Expo QR code with your phone's camera on iOS, or Expo Go on Android.
For a native dev-client build:
pnpm exec expo run:android
pnpm exec expo run:ios
pnpm start --dev-client
Pair With Desktop Orca
- Open Orca desktop.
- Go to Settings > Mobile.
- Scan the pairing QR code from the mobile app.
- Confirm the mobile host endpoint is
ws://<desktop-ip>:6768.
For the Android emulator, use ws://10.0.2.2:6768. For a physical phone, use the desktop LAN IP, for example ws://192.168.0.179:6768.
If the phone has a stale host entry, remove it from the app and pair again.
Development Paths
Android Phone
- Install Expo Go from Google Play
- Run
pnpm start, scan QR with Expo Go - For native modules:
pnpm exec expo run:android - Run with
pnpm start --dev-client
iOS Simulator
- Install Xcode from the App Store
- Run
pnpm start --iosto open in iOS Simulator
Physical Phone Debugging
The phone can be inspected through the connected device tooling:
orca snapshot --json
orca click --element @e3 --json
orca fill --element @e1 --value "ls" --json
orca screenshot --json
Use snapshot first to find the current element refs, then click/fill those refs. After mobile file edits, Metro usually hot reloads automatically, but navigating out of and back into the session screen can be useful because it re-runs terminal.subscribe.
Terminal Streaming Repro Without A Phone
Use this when terminal output does not render on device and you need to split server streaming bugs from WebView/UI bugs:
cd mobile
ORCA_MOBILE_WS_URL=ws://127.0.0.1:6768 pnpm exec tsx scripts/test-subscribe.ts <deviceToken> <serverPublicKeyB64>
You can pass a worktree selector as the third argument:
pnpm exec tsx scripts/test-subscribe.ts <deviceToken> <serverPublicKeyB64> "id:<worktreeId>"
pnpm exec tsx scripts/test-subscribe.ts <deviceToken> <serverPublicKeyB64> "path:/absolute/worktree/path"
pnpm exec tsx scripts/test-subscribe.ts <deviceToken> <serverPublicKeyB64> "name:my-worktree"
The expected result includes:
streamSawMarker: true
readSawMarker: true
If this repro fails, debug the desktop runtime/PTY path before the mobile WebView. If it passes but the phone is blank, debug the session screen or TerminalWebView readiness/queueing path.
Terminal Color Repro Without A Phone
Use this when terminal colors disappear after switching tabs. Open a Claude Code terminal and at least one other terminal in the target worktree, then run:
cd mobile
ORCA_MOBILE_WS_URL=ws://127.0.0.1:6768 pnpm exec tsx scripts/repro-terminal-colors.ts \
<deviceToken> <serverPublicKeyB64> "id:<worktreeId>"
The script captures terminal.subscribe snapshots in an A → B → A sequence and writes raw snapshots to mobile/terminal-color-repro/. If the two A snapshots have different sgrColor counts, the desktop snapshot changed during the switch. If they match, the ANSI color data is still present and the bug is in mobile replay/rendering.
Validation
Run these checks before committing mobile terminal changes:
cd mobile
pnpm exec tsc --noEmit
pnpm run check:tests-typecheck
pnpm lint
cd ..
pnpm typecheck:node
tsc --noEmit reads tsconfig.json, which excludes test files so Metro never bundles them.
tsconfig.test.json puts them back, and pnpm run typecheck:tests shows their errors in full.
check:tests-typecheck is the gate over it: a ratchet against tests-typecheck-baseline.txt, the
127 test files that do not typecheck yet. It fails when a file that checks today stops checking,
and when a baseline entry starts checking (prune it with
node scripts/check-tests-typecheck-ratchet.mjs --prune). The list may only shrink.
The same gate censuses the program first: every *.test.ts(x) on disk must be in it, or named in
the script's TESTS_OUTSIDE_PROGRAM with a reason. Without that, a test excluded from
tsconfig.test.json — or a Foo.test.tsx shadowed by a Foo.test.ts beside it, which a wildcard
include drops for the higher-priority extension — would leave the ratchet silently.
Protocol Version Compatibility
Mobile and desktop talk over a versioned protocol. Because mobile updates lag desktop by 24-48h via the App Store, both sides exchange version numbers on status.get so a genuinely incompatible combo can hard-block instead of silently misbehaving.
Constants live in two files (Metro can't resolve outside mobile/):
src/shared/protocol-version.ts—DESKTOP_PROTOCOL_VERSION,MIN_COMPATIBLE_MOBILE_VERSIONmobile/src/transport/protocol-version.ts—MOBILE_PROTOCOL_VERSION,MIN_COMPATIBLE_DESKTOP_VERSION
Today all four are set so evaluateCompat always returns { kind: 'ok' } — nothing blocks. The wire format is in place to flip a switch when needed.
When to bump
Bump DESKTOP_PROTOCOL_VERSION (and the mobile mirror MOBILE_PROTOCOL_VERSION when relevant) for breaking changes:
- Removed RPC method or required parameter that mobile uses
- Changed meaning (units, nullability) of an existing field mobile reads
- Changed encryption, framing, or auth handshake
Do not bump for additive changes:
- New RPC methods
- New optional fields on existing methods
- New event types in
terminal.subscribe
Set MIN_COMPATIBLE_MOBILE_VERSION (kill-switch) when desktop ships a change that requires a minimum mobile version to function safely. Same for MIN_COMPATIBLE_DESKTOP_VERSION from the mobile side.
When a verdict is blocked, mobile/src/components/ProtocolBlockScreen.tsx renders a screen pointing the user at the update that clears it. When mobile is too old it opens the newest release if the installed app's update check knows one, otherwise the App Store (iOS) or GitHub Releases (Android). When desktop is too old it opens GitHub Releases.
To exercise the block screen locally: set MIN_COMPATIBLE_DESKTOP_VERSION = 999 in mobile/src/transport/protocol-version.ts, rebuild, pair to any desktop. Revert before merging.
Mock Server
Develop the mobile app without a running Orca desktop instance:
pnpm mock-server # starts mock WebSocket server on port 6768
Connect from the app using endpoint ws://localhost:6768 and token mock-device-token.
Environment variables
MOCK_NATIVE_CHAT=1— serve the native-chat scenario (one live agent tab, empty transcript, image upload) instead of the default terminal fixtures.MOCK_CHAT_AGENT=omp— withMOCK_NATIVE_CHAT=1, present an OMP tab and four decoded transcript messages, including a tool call and result, instead of the default Claude scenario. It deliberately omitstranscriptPathto exercise legacy-hook readability discovery; current OMP hooks may report a path.MOCK_SERVER_KEY_FILE— persist the server keypair across restarts so a paired device keeps its public-key pin. A missing or invalid file is re-keyed with a warning, which forces a re-pair.
Scenario control files
Read on every request, so behaviour can be flipped mid-session without a restart (a restart would re-key E2EE and force a re-pair). Write the mode into the file, or delete it for the default.
MOCK_SEND_MODE_FILE(defaultorca-mock-send-modein the system temporary directory) —accept(default) accepts the send,errorfails it withmobile_input_floor_unavailable, anything else reports the send as rejected.MOCK_TERMINAL_LIST_MODE_FILE(defaultorca-mock-terminal-list-modein the system temporary directory) —omitreturns an empty terminal list,otherreturns a list that omits the chat handle, anything else lists it.MOCK_TERMINAL_STREAM_MODE_FILE(defaultorca-mock-terminal-stream-modein the system temporary directory) —deadanswers a subscribe withsubscribedthenend(a gone PTY), which is what exercises the rearm bound and terminal prune; anything else streams normally.
Connecting to Real Orca
- Start Orca desktop with WebSocket transport enabled
- In Orca, go to Settings > Mobile and scan the QR code with this app
- The QR encodes the connection endpoint, device token, and TLS fingerprint
Project Structure
mobile/
├── app/ # Expo Router screens (file-based routing)
│ ├── _layout.tsx # Root layout with navigation stack
│ ├── index.tsx # Home screen — paired hosts list
│ └── pair-scan.tsx # QR code scanning screen
├── src/
│ ├── terminal/ # Terminal WebView and xterm bridge
│ └── transport/ # WebSocket RPC client
├── scripts/
│ ├── test-subscribe.ts # Desktop streaming repro without a phone
│ └── mock-server.ts # Standalone mock WebSocket server
└── assets/ # App icons and splash screen