Jinwoo Hong
7ffcb7c7b4
<!-- orca-pr-loc -->
<!-- Programmatic LoC summary. Do not edit by hand; rewritten on every commit. -->
| | Files | Added | Deleted | Net |
| :--- | ---: | ---: | ---: | ---: |
| Test | 1 | $\color{#1a7f37}{\Huge{\mathbf{+}}}$279 | 0 | $\color{#1a7f37}{\Huge{\mathbf{+}}}$279 |
| Prod | 1 | $\color{#1a7f37}{\Huge{\mathbf{+}}}$58 | $\color{#cf222e}{\Huge{\mathbf{−}}}$13 | $\color{#1a7f37}{\Huge{\mathbf{+}}}$45 |
<!-- /orca-pr-loc -->
## ELI5
When we restart a relay server during a routine rollout, computers that were offline during the rollout still have that server written down as "their" server. If one of them comes back during the ~45 seconds while the replacement server is starting up, the director currently tells it "your server is unavailable, try again" over and over, instead of just sending it to another server. This change lets the director send it elsewhere, but only when it has proof that the old server process is really gone.
## What Changed
**Before (production, 2026-10-24 23:43Z, cell production-gce-c15):** during a same-capacity roll the cell is roll-isolated, drained, then its single VM is replaced. A desktop host that was offline during the drain reconnected in the gap between the old process stopping and the new one being ready. The director refused it for ~26 s (`assignment rejected route=assign lane=placement hinted=true reason=relay_home_cell_unavailable cause=not_ready`, x5, plus `route=resolve ... cause=unheard`, x2). It got a grant somewhere else only once the new process was live.
**After:** while the replacement process is starting up (`not_ready`), the same host is moved to another cell on its first try. A host on a roll-isolated cell can be moved off even though the cell is not live, as long as the process heartbeating for the cell *started after the roll isolated it* (`relay_cell_runtime.started_at > relay_cell_admission.roll_isolated_at`), its heartbeat is fresh, and (for connection-capped cells) its latest heartbeat reports zero connections. Both lanes that move an isolated host apply it: the narrowed isolated-replacement path (`isolatedIncumbentMayMove`) and the all-rows path it falls back to. Together these serve assign and resolve, because resolve hands a non-live cell to `assign`. "Roll-isolated" is the existing predicate: `migration-only` with a roll stamp younger than 2 h.
**Still refused (residual):** the `unheard` window, when the old process's heartbeat is past the 45 s TTL and the new process has not been heard yet. On c15 that was about 5-10 s. Nothing proves the drained sockets are gone there, so the refusal stays, and the client's retry recovers once the new process heartbeats. Also unchanged: an ordinary readiness dip, a `migration-only` cell without a roll stamp, a stale (over 2 h) stamp, and a new process that later goes unheard. Fence handling is untouched.
## Why
The refusal exists because moving a host off a cell we cannot fully trust could leave the host's old socket served by the old process while its clients go to the new cell. Until now, the only proof the code accepted was an operator-written fence. This adds a second proof that costs no operator step:
- **A new incarnation means the drained process is gone.** Each process mints a fresh incarnation UUID and `startedAt` at boot (`cell-heartbeat-client.ts`), and the director rejects a heartbeat with an older `startedAt` once a newer one is recorded. Each cell VM runs exactly one relay container (`docker run --name orca-relay --restart always`, with stale containers force-removed at boot). The cell MIG uses `replacement_method = RECREATE` with `max_surge = 0` (pinned by a Terraform assert), so the old VM is deleted before the new one is created. A same-VM container restart also produces a new incarnation and is just as safe: Docker restarts only after the old process exits, and that exit closes all of its sockets.
- **Whatever the new process holds, it can enforce.** It is heartbeating (fresh), and on capped cells it reports no connections at all. A host that attaches between two heartbeats carries the old epoch, so the epoch bump this move writes makes the cell refuse it or close it with 4409. That check is the same live `verifyCellAssignment` / control-renewal read the existing live-isolated path relies on.
- **Clock skew** between the cell's `startedAt` and the director's stamp is covered by the zero-connections check on capped cells only: a drained process that only *looks* post-stamp still holds its sockets, so it is refused. Uncapped cells report no counts and get no such guard; they already move off without a fence on the dead-cell path today, so this adds no new exposure there.
Alternatives considered: (a) adding an incarnation column to `relay_assignments`: a schema change for no extra proof. (b) Honouring any not-live roll-isolated cell: that drops the proof. (c) Having the roll write a fence for the `unheard` window: the only fence driver refuses enabled cells, a fence is a MIG resize to 0 that would fight the roll's own apply, and the window is a few seconds the client retry already absorbs.
## Linked Issue
N/A (internal production incident, 2026-10-24 c15 roll).
## Visual Proof
N/A: director placement logic only, no UI.
## Testing
- [x] Automated tests added: `cloud/apps/relay/src/assignment-roll-replaced-cell.test.ts` (18 cases; the redial cases run for both hinted assign and resolve):
- not_ready with an incarnation started after isolation: re-placed, epoch bumped.
- Refused as before: not_ready with the pre-isolation incarnation, with reported connections, after the new process goes unheard, with a stale stamp, on an ordinary unstamped readiness dip, and on an unstamped migration-only cell. Also refused: the unheard window between the old and new process.
- One case per lane. The narrowed path's cross-region move fails if only the narrowed-path change is reverted. The all-rows path's move, forced by units no lease backs, fails if only the all-rows change is reverted.
- [x] Full relay vitest suite, `--maxWorkers=2`: 130 files / 1344 tests passed, 39 files (201 tests) skipped. The Postgres-backed suites skip locally (no `ORCA_RELAY_TEST_POSTGRES_URL`); CI runs them.
- [x] `pnpm typecheck` in `cloud/apps/relay`.
## Review
## Agent skill upstream boundary
- [x] Not applicable, or this change copies or mechanically translates no upstream skill-installer source, tests, fixtures, registry entries, path tables, comments, or documentation.
## Notes
Backwards compatibility: no schema, wire or RPC change; old directors keep refusing as today. Performance: the new query runs only when a host's home cell is roll-isolated and not live; the general sticky hot path is unchanged.
## Checklist
- [x] This PR is small and focused
- [x] I explained what changed and why (ELI5, the user-facing before/after, the mechanism, and why over the alternatives)
- [x] Before/after screenshots or videos attached for UI changes, or `N/A` with reason
- [x] Self-reviewed for correctness, security, and performance
- [x] Cross-platform, SSH/remote, and path/shortcut impact considered (or N/A)
- [x] `pnpm lint`, `pnpm typecheck`, `pnpm test`, and `pnpm build` pass (or CI will cover; local preferred)
Orca
中文 · 日本語 · 한국어 · Español · Français · Português
The AI Orchestrator for 100x builders.
Run Codex, ClaudeCode, OpenCode or Pi side-by-side — each in its own worktree, tracked in one place.
Download Orca
Features
Mobile CompanionMonitor and steer your agents from your phone — get notified when an agent finishes and send follow-ups from anywhere. |
 |
Parallel WorktreesFan one prompt across five agents, each in its own isolated git worktree — compare the results and merge the winner. |
 |
Terminal SplitsGhostty-class terminals with WebGL rendering, infinite splits, and scrollback that survives restarts. |
 |
Design ModeClick any UI element in a real Chromium window to send its HTML, CSS, and a cropped screenshot straight into your agent's prompt. |
 |
GitHub & Linear, NativeBrowse PRs, issues, and project boards in-app — open a worktree from any task and review without a context switch. |
 |
SSH WorktreesRun agents on a beefy remote box with full file editing, git, and terminals — auto-reconnect and port forwarding included. |
 |
Annotate AI DiffsDrop comments on any diff line and ship them back to the agent — review, edit, and commit without leaving Orca. |
 |
Drag Files to AgentsVS Code's editor with autosave everywhere — drag files or images straight into an agent prompt. |
 |
Orca CLIAgents drive Orca too — script every workflow with |
 |
Also in the box:
- Quick open — Search across worktrees, files, agents, commands, and repo context without leaving your flow.
- Account switcher & usage tracking — See Claude and Codex usage and rate-limit resets, and hot-swap accounts without re-logging in.
- Rich repo previews — Preview Markdown, images, PDFs, and repo docs in the workspace.
- Computer Use — Let agents operate desktop apps and visible UI when a workflow needs real interaction.
- Notifications and unread state — Know when an agent finishes or needs attention, then mark threads unread to come back later.
- And many, many more — we ship daily, so this list is perpetually behind. The changelog is the real feature list.
Supported Agents
Works with any CLI agent — if it runs in a terminal, it runs in Orca.
Claude Code
Codex
Grok
Cursor
GitHub Copilot
Muse
DeepSeek Harness
ZCode
OpenCode
MiMo Code
Amp
OpenClaude
Antigravity
Pi
oh-my-pi
Hermes Agent
Devin
Goose
Auggie
Autohand Code
Charm
Cline
CodeBuddy
Codebuff
Freebuff
Command Code
Continue
Droid
Kilocode
Kimi
Kiro
Mistral Vibe
Qwen Code
Rovo Dev
+ any CLI agent
Install
Desktop — macOS, Windows, Linux
- Download from onOrca.dev
- Or grab a build directly: macOS Apple Silicon · macOS Intel · Windows (.exe) · Linux AppImage · All builds
Or via a package manager:
# macOS (Homebrew)
brew install --cask stablyai/orca/orca
# Arch Linux (AUR) — or stably-orca-git to build from source
yay -S stably-orca-bin
Mobile Companion — iOS, Android
Pair with your desktop app to monitor and steer your agents from your phone.
- iOS: Download on the App Store
- Android: Download APK · Install guide
Community & Support
-
Discord: Join the community on Discord.
-
Twitter / X: Follow @orca_build for updates and announcements.
-
WeChat: Group 12 is full. Scan to join the Orca community WeChat group 13.
-
Feedback & Ideas: We ship fast. Missing something? Request a new feature.
-
Privacy: See the privacy & telemetry docs for what anonymous usage data Orca collects and how to opt out.
-
Show Support: Star this repo to follow along with our daily ships.
Developing
Want to contribute or run locally? See our CONTRIBUTING.md guide.
The relay that pairs the mobile app with a desktop host is also in this repository under
cloud/, with a separate pnpm workspace and setup guide.
Signed Builds
Windows code signing sponored/provided by SignPath.io, certificate by SignPath Foundation.
License
Orca is free and open source under the MIT License.