Jinwoo Hong c8a5580659 fix(relay): re-place hosts off a cell isolated for a roll (#21911)
* fix(relay): re-place hosts off a cell isolated for a roll

A roll isolates a cell by moving it out of the 'general' admission class; the
cell then refuses every attach with 4503. The director never noticed, because
the only liveness test it applies to a host's current cell reads
`relay_cell_runtime.ready` and the heartbeat, and an isolated cell keeps
heartbeating ready=1 for the whole drain. So every host on that cell was handed
its own dead cell, closed, and handed it back — 500-1,900 hosts looping for
13-16 minutes per cell roll, at ~6 dials each per minute, with no neighbour
absorbing anything.

The sticky lane now treats a live incumbent whose admission is 'migration-only'
— the state a roll's isolate step writes — the same way it treats a dead one:
it returns null, which means "fall through to placement". The placement lane
had the identical hole eleven lines further down, so it takes the same
predicate; without that second swap the sticky change is inert, because
placement would hand the pin straight back (a draining cell has more headroom
than anyone). An isolated incumbent skips the dead-cell fence branch: that
branch exists to prove an unreachable cell stopped serving a host, and this one
is reachable and enforces the epoch itself.

'existing-only' is deliberately untouched — those cells serve the hosts they
already hold, and only `assignmentStrandedOnUnservedCell` may release that pin.
A host with an open `relay_assignment_migrations` row keeps its pin too, so
this stays disjoint from the migration machinery.

Claude-Session: https://claude.ai/code/session_01JNnE9qzUZMMnqpZWCqM3nb

* fix(relay): gate re-placement on a roll-isolation marker, not on admission

Review of the first commit found the predicate wrong. `migration-only` is an
admission class, not a drain signal: an Asia `--mode rollback`, an evacuation or
forward-recovery target awaiting a separate promote dispatch, a failed same-cap
wave's re-isolate, an abandoned migration retired on its target and a rehome
settlement all park loaded cells there durably, with no migration lease and no
open migration row. All five were indistinguishable from a roll's isolate, so
the first commit would have converted `operate-relay-asia-admission --mode
rollback` from a reversible admission flip into a mass move of ~4,000 hosts —
and, because `leastLoadedCell` treated region as a preference, into us-central1.

The signal is now an explicit stamp. `relay_cell_admission` gains a nullable
`roll_isolated_at`, added through the shared schema runner's catalog pre-check
so a migrated database takes no relation lock on boot and an un-migrated one
gets a catalog-only rewrite. The same-cap isolate step is its only writer, via a
new optional `rollIsolatedCells` on the selector apply; the same UPDATE that
writes the state clears the stamp whenever a cell leaves 'migration-only', so a
restore cannot leave one behind and a failed wave's re-isolate keeps the one it
has. Every other admission writer omits the field, so its cells stay unmarked
and their hosts stay pinned. Old directors ignore the field; old callers never
send it.

Region is now a constraint rather than a preference on this path only: a
re-placement must find a general, live cell with connection headroom in the
host's own region, or the pin is kept and one
`orca_relay_sticky_replacement_deferred` event is logged. Cross-region spill is
no longer reachable here.

The fence bypass is narrowed to a live incumbent. It was always a no-op for the
intended case, and for a stamped cell that stops heartbeating while still
holding sockets it reopened split-brain; that cell now takes the dead-cell path
unchanged.

Also: the hot-path admission reader no longer throws on an unrecognised state —
it sits on every sticky dial and the rule it feeds is "move the host", so an
unreadable row has to mean "don't". And the sticky lane reads the admission row
once for both the stranded rule and the stamp instead of twice.

Claude-Session: https://claude.ai/code/session_01JNnE9qzUZMMnqpZWCqM3nb

* fix(relay): emit the re-placement events after the transaction commits

CodeRabbit on assignment-store.ts:1075. Both events were written where they are
decided, which is inside assignOnce's transaction. A reservation or lease write
failing after that point rolls the placement back, but a line already on stdout
cannot be rolled back with it — so the canary this PR asks an operator to read
would count re-placements that never happened, and a Postgres transaction retry
could leave a stale line behind as well.

The transaction now returns its events alongside the RelayAssignment and the
caller flushes them once it has resolved. Returning them rather than setting a
variable in the enclosing scope is what makes the retry case safe too: only the
attempt that committed can carry its events out. assign()'s signature is
unchanged; the extra shape lives entirely inside assignOnce.

orca_relay_sticky_replacement_deferred was moved the same way. It cost one more
push into the array that already existed, and it is decided inside the same
transaction, so leaving it behind would have been the odd case rather than the
cheap one.

The new test injects a failure on the first write after the decision, asserts no
event is emitted, and asserts the assignment is still on its original cell —
without that second assertion the absence would only prove the emit was early,
not that it would have been wrong. A control dial with nothing injected emits
exactly one event, so the case cannot pass on a broken harness. With the emit
put back inside the transaction, it fails.

Claude-Session: https://claude.ai/code/session_01JNnE9qzUZMMnqpZWCqM3nb

* fix(relay): expire the roll stamp, correct the wire note, assert the stamp landed

Delta review findings B, D and E. A (the deferral path's cost) is deliberately
not implemented; it is now written up under Follow-ups in the PR body as
required before any Asia roll, because it cannot fire in a US canary.

B, which also closes C: the stamp was written, carried and never compared to
anything. A roll isolates and restores one cell inside ~15 minutes, so a stamp
older than two hours is not a roll in progress. It is a failed wave whose
failsafe re-isolated a possibly healthy cell and is waiting on an operator — the
postmortem in this tree records gaps of hours — or an orphan left by a director
rollback whose restore wrote 'general' without the clause that clears the stamp,
which the selector's 'keep' branch would then preserve until some later park
reactivated it. Both want the same answer and it is the pre-existing one: keep
the pin. One comparison against a value already on the row.

The bound takes the caller's `now` rather than reading the clock again, so one
assign reasons about one instant; the stamp's age is now a thing that decides
whether a host moves, and two clock reads could disagree across it.

D: the comment beside the new request field claimed an updated caller reaching
an older director "is simply ignored". The schema is .strict(), so it is a 400.
That fails closed — the isolate aborts before MUTATION_STARTED is set and
nothing is written — but it is a deploy ordering constraint, and it was
undocumented. The comment now says so and the PR body's rollout notes carry it.

E: nothing read the `rollIsolated` the script already prints, so an older script
against a newer director would silently produce today's behaviour and the canary
would read as "the fix did nothing" with no way to tell that from a wrong
premise. Both isolate steps now assert it, beside the generation they already
parse.

Claude-Session: https://claude.ai/code/session_01JNnE9qzUZMMnqpZWCqM3nb
2026-09-21 04:18:57 -04:00
2026-05-04 20:42:03 -07:00
2026-03-16 22:27:51 -07:00
2026-03-28 10:19:14 -07:00

Orca Orca

GitHub stars Total downloads across all releases License: MIT Join the Orca Discord Follow Orca on X Supported platforms: macOS, Windows, and Linux

中文 · 日本語 · 한국어 · Español · Français · Português

The AI Orchestrator for 100x builders.
Run Codex, ClaudeCode, OpenCode or Pi side-by-side — each in its own worktree, tracked in one place.

Download Orca

Orca desktop app running agents in parallel worktrees, with the Orca mobile companion app in the corner

Features

Mobile Companion

Monitor and steer your agents from your phone — get notified when an agent finishes and send follow-ups from anywhere.

iOS App Store · TestFlight · Android APK 0.0.50 · Docs →

Orca desktop with the mobile companion app

Parallel Worktrees

Fan one prompt across five agents, each in its own isolated git worktree — compare the results and merge the winner.

Docs →

Parallel worktree orchestration

Terminal Splits

Ghostty-class terminals with WebGL rendering, infinite splits, and scrollback that survives restarts.

Docs →

Terminal splits

Design Mode

Click any UI element in a real Chromium window to send its HTML, CSS, and a cropped screenshot straight into your agent's prompt.

Docs →

Embedded browser and Design Mode

GitHub & Linear, Native

Browse PRs, issues, and project boards in-app — open a worktree from any task and review without a context switch.

Docs →

GitHub and Linear task workflows in Orca

SSH Worktrees

Run agents on a beefy remote box with full file editing, git, and terminals — auto-reconnect and port forwarding included.

Docs →

Remote worktrees over SSH

Annotate AI Diffs

Drop comments on any diff line and ship them back to the agent — review, edit, and commit without leaving Orca.

Docs →

Annotate AI-generated diffs

Drag Files to Agents

VS Code's editor with autosave everywhere — drag files or images straight into an agent prompt.

Docs →

Drag files and images into an agent prompt

Orca CLI

Agents drive Orca too — script every workflow with orca worktree create, snapshot, click, and fill.

Docs →

Script Orca from the CLI

Also in the box:

  • Quick open — Search across worktrees, files, agents, commands, and repo context without leaving your flow.
  • Account switcher & usage tracking — See Claude and Codex usage and rate-limit resets, and hot-swap accounts without re-logging in.
  • Rich repo previews — Preview Markdown, images, PDFs, and repo docs in the workspace.
  • Computer Use — Let agents operate desktop apps and visible UI when a workflow needs real interaction.
  • Notifications and unread state — Know when an agent finishes or needs attention, then mark threads unread to come back later.
  • And many, many more — we ship daily, so this list is perpetually behind. The changelog is the real feature list.

Supported Agents

Works with any CLI agent — if it runs in a terminal, it runs in Orca.

Claude Code logo Claude Code   Codex logo Codex   Grok logo Grok   Cursor logo Cursor   GitHub Copilot logo GitHub Copilot   OpenCode logo OpenCode   MiMo Code logo MiMo Code   Amp logo Amp   OpenClaude logo OpenClaude   Antigravity logo Antigravity   Pi logo Pi   oh-my-pi logo oh-my-pi   Hermes Agent logo Hermes Agent   Devin logo Devin   Goose logo Goose   Auggie logo Auggie   Autohand Code logo Autohand Code   Charm logo Charm   Cline logo Cline   Codebuff logo Codebuff   Command Code logo Command Code   Continue logo Continue   Droid logo Droid   Kilocode logo Kilocode   Kimi logo Kimi   Kiro logo Kiro   Mistral Vibe logo Mistral Vibe   Qwen Code logo Qwen Code   Rovo Dev logo Rovo Dev   + any CLI agent


Install

Desktop — macOS, Windows, Linux

Or via a package manager:

# macOS (Homebrew)
brew install --cask stablyai/orca/orca

# Arch Linux (AUR) — or stably-orca-git to build from source
yay -S stably-orca-bin

Mobile Companion — iOS, Android

Pair with your desktop app to monitor and steer your agents from your phone.


Community & Support

  • Discord: Join the community on Discord.

  • Twitter / X: Follow @orca_build for updates and announcements.

  • WeChat: Scan to join the Orca community WeChat group 9.

    WeChat group 9 QR code for the Orca community
  • Feedback & Ideas: We ship fast. Missing something? Request a new feature.

  • Privacy: See the privacy & telemetry docs for what anonymous usage data Orca collects and how to opt out.

  • Show Support: Star this repo to follow along with our daily ships.


Developing

Want to contribute or run locally? See our CONTRIBUTING.md guide.

The relay that pairs the mobile app with a desktop host is also in this repository under cloud/, with a separate pnpm workspace and setup guide.

Orca contributors

GitHub star history chart for stablyai/orca

Signed Builds

Windows code signing sponored/provided by SignPath.io, certificate by SignPath Foundation.

License

Orca is free and open source under the MIT License.

S
Description
Orca is the ADE for working with a fleet of parallel agents. Run any coding agent with your own subscription. Available on desktop, mobile and remote runtime.
Readme MIT
1.4 GiB
Languages
TypeScript 95.2%
JavaScript 4.1%
Swift 0.2%
CSS 0.1%
HCL 0.1%