Golden source builds publish their orcad + in-VM CLI as a commit-pinned bundle (R2, uploaded by the build VM over a signed PUT; the Worker computes the stored sha256). On every boot and wake, before orcad starts and inside the existing orcad phase, a sandbox whose /opt/orca/runtime/orcad/SOURCE_COMMIT differs from its region's active golden fetches that bundle from the control plane's own host (already on the egress allowlist) with a 10-minute signed URL, verifies the sha256 the control plane passes in, checks the commit and node runtime, and swaps orcad + orca-cli atomically. Any failure keeps the old orcad and boot continues; every outcome is logged (runtime_refresh: outcome, from, to, ms). RUNTIME_REFRESH=off is the kill switch. Dev R2 bucket orca-sandbox-runtime-dev.
Once an Orca account stops applying (revoked after a confirmed 401, expired, removed, turned off, shared, owner changed), requests still carrying the sandbox's account placeholder (shells and agents already running, a Vercel forward that hasn't caught up) now get Anthropic's own 401 authentication_error, saying the Orca sign-in for Claude needs renewing, instead of 403 host_not_proxied or a bare deny. Nothing is swapped on those routes.
Auth pushes PropelAuth offboarding and Orca-side member removals here (stablyai/orca-cloud#739);
it should not need the full operator token for that. The operator token is still accepted.
Terminal prompts read ubuntu@<random VM id> and changed after every wake. Each start (create and wake)
now sets the hostname to the workspace's title (else its repo's name) as a hostname label, plus an
/etc/hosts entry, inline before orcad starts: no image change. Verified on a dev VM: hostname ja-hello,
127.0.1.1 ja-hello, no sudo warnings.
- POST /v1/sandboxes takes an Idempotency-Key (api_idempotency, 24 h, scoped per account and member):
the same key and body return the first create's sandbox instead of starting another; same key with a
different body is a 422. The desktop sends context.idempotencyKey when its caller provides one.
- The desktop sends the workspace name as the create's title, so lists show it, not an untitled row.
- GET /v1/sandboxes/boot-plan?repo= says whether a create would clone or update (environment build);
the driver asks in parallel with the create, so the build sheet names 'Clone repository' or 'Update
repository' once, with a placeholder until it knows, and never relabels mid-run.
Home's team list leaves routine-run VMs out unless the new Routine runs switch is on. The switch is
offered only when the team has routines, counts the running runs it hides, and is remembered per
account. Hidden runs leave the workspace count but keep their sandbox hours in the total. Listed
runs carry a routine mark and 'Routine · <name>'. Control plane: GET /v1/team/workspaces
?routines=include, plus fromRoutine/routineName per row and a routineRuns summary.
Design-squad review of #336: a teammate's heatmap is shown only if they opted in (same
Experimental switch as Pokes, renamed 'Pokes and activity'); otherwise their profile says they
haven't shared it. Your own heatmap always shows. Team rows get 'View profile' first in the menu.
Own-profile hint: 'Hover a day to see your sandbox hours.'
Conductor 0.91 parity. Clicking a teammate opens their profile, which shows their role, a
26-week daily usage heatmap and their recent team workspaces. Teammates see only 0-4 levels,
and amounts are shown only to you. Pokes are opt-in in Settings > Experimental and need both
people to have them on. They are rate-limited to 1 per minute per teammate and 20 per hour, and
delivered in-app by a 30 s poll that runs only while Pokes are on and Orca is in front.
Control plane: migration 0050 (member_usage_hours, member_poke_prefs, member_pokes),
src/team-presence.ts routes. The heatmap reads usage_outbox sandbox time plus dated
token_usd_micros deltas. A member's poke opt-in is dropped when they are removed.
The first create of a repo that no environment of the account lists makes a default environment for it
(just the repo, cloned blobless, no install script) and builds it in the background with the existing
environment build pipeline (EnvironmentBuildRunner). Later creates boot from its active build and fetch
only what changed, in the background, shown as 'Update repository' with git's real progress.
- environments.kind 'org' | 'auto' + last_used_at (migration 0049, applied to dev); an admin's edit
adopts an auto environment; an admin environment listing the repo always wins
- auto environments unused for 14 days are deleted with their snapshots (cron); per account only
- golden refresh, staleness, pruning and the warm pool apply as for any environment
- the caller's machine size stands for auto environments; an explicit other region skips them
- AUTO_ENVIRONMENTS=off disables it
- desktop: feature repo-update, 'Update repository' / 'Fetching changes…' / sidebar 'Updating the
repository…'
Measured on dev (stablyai/orca, staging OWNER): before 6 s to ready + 58 s clone; the background build
took 79 s; after, ready in 6.6-8.7 s with the repo current 0.7 s later (7.4-9.4 s total).
Runs the confirm-before-revoke auth check on demand: a pass within 10 min answers at once (verified_at, migration 0048), otherwise one check rate-limited to one a minute per person, revoking (audited) only on a definite authentication_error. Storing a new token forgets the old verification.
A single 401 on an Orca-account request no longer revokes the person's Claude account everywhere. The hook only fires after the real credential was swapped in and sent. The control plane then makes one free auth check with the real token (Anthropic count_tokens, oauth beta) and revokes only on a definite authentication_error; anything else (passes, 429, 5xx, other 401 types, network) keeps the account. Checks are rate-limited to one per person and agent per minute (agent_credentials.checked_at), and every keep/revoke is audited (agent_credential_audit, migration 0047). Codex keeps its lease-refresh path.
The team and per-member concurrency counts (billing's checkStart input and the guardrail backstop)
counted 'suspending' rows, which drain for up to 5 min after someone chose Sleep and read as
asleep everywhere, and 'creating' rows whose Worker was cut off (until the cron released them).
Both could refuse a start that the user's view says fits. The global capacity cap still counts them.
A repo-backed create of stablyai/orca (2.7 GB) did a full git clone inside the 120 s orcad start
command, failed at ~127 s with a bare create_failed, then the pooled path retried on a fresh VM
(~255 s total). When the desktop gave up, the cut-off Worker left the retry row in 'creating' for
15 min.
- workspace prelude clones --filter=blob:none (stablyai/orca: 68 s instead of >120 s timeout)
- feature clone-progress: the clone runs detached after ready; GET /v1/sandboxes/:id/clone reports
{state, percent, stage} and records a failed clone's reason in last_error; 15 min clone timeout
- failed creates record why (orcad_start_failed, provider_<status>, ...) instead of create_failed
- a pooled boot that failed after 30 s is not retried on a fresh VM
- 'creating' rows whose Worker was cut off are released after 5 min (create_abandoned)
GET /v1/agent-credentials adds providerState {claude, codex: on|off|disabled} next to providerEnabled.
Clients read it through orcaCloudProviderState (older servers: disabled = kill switch; a state that
contradicts providerEnabled is never trusted). Flag off is a new 'unavailable' kind: Settings says
it isn't available in Orca Cloud yet (no 'right now / until it's back'), and the composer hides the
Agent row like an agent without an account path. Kill-switch copy is unchanged.
The boot resolves the session env before the VM is up, and a secret change only refreshes running sandboxes, so a value saved from the build-sheet Secrets row during the boot never reached the workspace (found in the W6 live e2e). Once running, the boot re-resolves the env and refreshes the session when it changed.
Server: only a row known to be private counts as unshared. Desktop panel: a failed sharing read leaves out the caller's Personal values and the Personal save scope, as for a teammate.
Billing's setup chain now passes the GitHub identity that Orca Cloud auth
verified from a GitHub sign-in (githubUserId, githubLogin). The install URL's
signed state carries it, and the setup URL checks it against the installation
with the App's own credentials: the user's own account, or an active org admin
(members:read installation token). A match binds with method 'idp' and skips
GitHub's separate Authorize page. Anything else falls back to the OAuth step
(A4b) as before.
A shared list of agent and model-provider credential names (AWS, Bedrock, OpenRouter, Groq, xAI,
Mistral, DeepSeek, Together, Fireworks, Perplexity, Cerebras, Moonshot/Kimi, DashScope, Z.ai,
MiniMax, NVIDIA, Cohere, Azure OpenAI, Hugging Face, Amp, Factory, ...) marks every one that has no
egress swap. AWS keeps its own sentence; others read 'Orca can't swap {{name}} at the network edge,
...'. The control-plane test now walks the server's agent keys, this list and the desktop's Agent
API key presets: each is swapped, or flagged.
Verified on a dev sandbox (Claude Code 2.1.319, codex-cli 0.166.0): runCommand runs as ubuntu with
HOME=/vercel (same user as orcad); `claude auth logout` and `codex logout` exist and exit 0; the
script reported {loggedOut:2,left:0}, removed both credential files and kept other ~/.claude.json keys.
- M1: a permanent refresh failure revokes only the login it refreshed (CAS on updated_at).
- M2: dead logins (refresh_token_expired/reused/invalidated) are never retried; a new login clears it.
- M3: the Codex account route only carries the token to /backend-api/codex/*, the plugin catalog and
wham/usage (GET), seen live; plain segments only.
- M4: one atomic INSERT enforces one live broker per person (409 broker_in_progress) and 6 starts/hour;
broker sandboxes use 2 vCPUs.
- L5: auth.json is read only after Codex exited 0.
- L6: the 401 hook respects the refresher's backoff and dead logins.
- L7: a cancel while the broker is starting wins: the sandbox goes and POST answers 409 cancelled.
- L8: orca-sandbox-env writes the marker before the swap, recognises any Orca placeholder, and never
overwrites a login already set aside.
Codex signs in once via the person's Orca account:
- POST /v1/agent-credentials/codex/broker starts a short-lived broker sandbox (golden image, egress
auth.openai.com + chatgpt.com only, non-persistent, tagged orcaCodexBroker/orcaInfra so the workspace
orphan sweep and billing never see it) running `codex login --device-auth` (Codex 0.166), and returns
{brokerId, verificationUrl, userCode}. GET .../broker/:id -> {status: pending|active|failed}; DELETE
cancels. On completion the Worker reads auth.json through the files API, stores the tokens
envelope-encrypted and Personal-only in agent_credentials, and deletes the broker sandbox.
- Lease refresher (cron, every 5 min): refreshes the access token with the refresh token before expiry,
single-flight per person (D1 lease) with a compare-and-swap write, so a refresh token is spent once and
a delete or new login is never overwritten (no refresh_token_reused). Failures mark expiring/expired
(by the real access-token expiry) or revoked (refresh_token_reused/invalidated). A chatgpt.com 401
triggers a refresh instead of revoking.
- codexAccount AccountProvider: placeholder access token bound to the sandbox id (D14), swapped for the
real one on chatgpt.com with ChatGPT-Account-Id; never in a shared workspace (D13); kill switch
AGENT_ACCOUNTS_CODEX_DISABLED=1.
- VM: orca-sandbox-env writes a placeholder ~/.codex/auth.json (no refresh token) from
ORCA_CODEX_AUTH_PLACEHOLDER, sets a managed HTTPS ChatGPT provider (Codex's Responses WebSocket can't
pass the egress proxy; verified live), and sets a person's own login aside / restores it.
- Migration 0045_codex_broker (codex_broker_sessions, codex_leases; no token columns).
- Flag S2 on in wrangler.dev.jsonc; absent in prod.
Neil
Neil