orca/cloud/docs/vercel-driver-design.md

Orca Cloud sandboxes on Vercel — white-label driver design (v0)

Status: design + mock-tested code in nwparker/orca-private (integration branch nwparker/vercel-driver). Nothing here has touched a real Vercel or Cloudflare resource. Inputs: bake-off (reports/provider-bakeoff.md), conductor-parity-matrix.html (Vercel-first, 24h cap), vm-just-works.html, reports/credentials.md, stablyai/orca#20777 (recipe that talks to Vercel with the user's own account), billing design (cloud/docs/accounts-billing-design.md).

Goal and non-goals

• One click "Orca Cloud" workspace. Orca's Vercel Pro team owns every sandbox. Users never see a Vercel account, token, dashboard or brand. They pay Orca (billing Worker), and Orca pays Vercel.
• Not in v0: memory pause (Vercel has none), BYO Vercel account (that is #20777's recipe path), live fork, Drives.

Shape

desktop (Electron main)                         Orca sandbox control plane (CF Worker)        Vercel (Orca's team)
 HostedSandboxDriver ──HTTPS, Orca JWT────────▶  /v1/sandboxes/*  ──Vercel token (secret)──▶  /v2/sandboxes/*
   implements EnvironmentProviderDriver           D1: sandboxes, sessions, usage cursor         persistent sandbox
   └ recipe adapter → existing runtime record     billing: /v1/internal/entitlements/check,     per workspace,
     / pairing / resume pipeline                    /v1/internal/usage (CONTROL_PLANE_TOKEN)    golden snapshot
 desktop ◀───────── wss pairing (orcad, E2EE) ─────────────────────────────────────────────▶  orcad :6768

• Control plane = Cloudflare Worker + D1 (cloud/apps/sandbox-control). Same stack as billing and relay-edge, so one toolchain, JWKS auth pattern and deploy story. It is the only holder of VERCEL_TOKEN/team/project. Workers can't run @vercel/sandbox (undici, fs, xdg paths), so the Worker calls Vercel's REST API (/v2, /v3 endpoints read from SDK 3.9.1's api-client) through a SandboxProviderApi interface. Tests use an in-memory fake; the REST client is tested against a fake fetch. A Durable Object per sandbox is the upgrade path if D1 conditional updates prove too weak for concurrent lifecycle calls (v0 uses a generation compare-and-set).
• Desktop holds only a short-lived Orca JWT (aud=orca-sandbox, minted by Orca Cloud auth like the billing token; until that endpoint exists the token provider is injected). It never sees a Vercel credential.
• Integration with existing VM code: the hosted driver returns the same orca-server recipe result (pairingCode, projectRoot, userData) a shell recipe prints, so write-ahead records (#39241), timeouts (#39238), pairing reuse on resume (#39225), host health and sleep/wake work unchanged. The adapter lives in src/main/environment-providers/environment-provider-recipe-adapter.ts; recipe catalog/UI wiring is follow-up.

Desktop interface (src/main/environment-providers/)

EnvironmentProviderDriver: create(spec), resume(handle), suspend(handle), destroy(handle), status(handle), optional renewLease(handle, ms) and rollover(handle); every call takes {signal, onProgress(phase)}. Handles are {driverId, environmentId, driverState} (opaque JSON persisted in the runtime record's userData). Capability flags drive UX and schedulers:

flag	Vercel hosted
memoryPause	false (stop = reboot, PTYs lost; measured)
maxSessionMs	24h (Pro). Scheduler forces rollover at 23h50m
fsPersistOnStop	true (auto snapshot on stop, keepLastSnapshots 2)
egressInjection	true
idleStopMs	20 min default (Orca-owned, not Vercel's timeout)

Errors are EnvironmentProviderError{code} with a closed code set (entitlement_denied:<reason>, not_found, conflict, unavailable, timeout, unauthorized, invalid_response). A driver contract kit (environment-provider-driver-contract.ts) runs the lifecycle against any driver; the hosted driver runs it against a fake control plane.

Control-plane API (Worker)

All desktop routes require the Orca JWT; the account is org:<id> or user:<id> (same rule as billing).

route	does
POST /v1/sandboxes {workspaceId, machineSize, region?, repo?}	entitlement check → write-ahead row → create from golden snapshot → start orcad → {sandbox, connection, session}
GET /v1/sandboxes[/:id]	status from D1 + provider reconcile
POST /v1/sandboxes/:id/suspend	drain orcad → stop session (auto snapshot) → settle usage
POST /v1/sandboxes/:id/resume	entitlement check → resume → start orcad → connection (same pairing; identity on disk)
POST /v1/sandboxes/:id/lease {extendMs}	extend-timeout, clamped to the session deadline
POST /v1/sandboxes/:id/rollover	drain → stop → resume → start orcad (24h cap path)
DELETE /v1/sandboxes/:id	delete with deleteOrphanSnapshots, then verify every recorded snapshot is gone
POST /v1/egress/proxy	Vercel forwardURL target (see egress)
cron (5 min)	settle usage, stop sandboxes past idle/deadline or whose account lost entitlement, sweep stuck creating rows

Vercel resource naming: orca-<sandboxId> where sandboxId is our random id (never user text). Tags: orcaSandbox, orcaEnv (prod/dev). Ownership is checked against D1 + the name; nothing is ever deleted by prefix. Create params: source: {type:'snapshot', snapshotId: GOLDEN_SNAPSHOT_ID}, persistent: true, ports: [6768], resources.vcpus from machine size (standard 4, large 8; xlarge 16 needs Enterprise), timeout = Orca lease (30 min, renewed), keepLastSnapshots: {count: 2}, snapshotExpiration: 0 while live (finite TTL after archive), region default iad1, networkPolicy from the policy compiler.

Entitlements: POST billing /v1/internal/entitlements/check {accountId, machineSize, runningSandboxes} before create/resume. If billing is unreachable or unconfigured the Worker refuses (fail closed); a stub client is used only when ENTITLEMENTS_MODE=stub in local dev. Usage: per-session segments [lastSettledAt, now] (≤24h) sent to /v1/internal/usage with usageId = <sandboxId>:<sessionId>:<segmentStart> (idempotent). If the answer says new starts are no longer allowed for included_hours_exhausted/overage_cap_reached, running sandboxes are drained and stopped; payment_past_due keeps running (grace is an open billing decision).

Pairing material: the Worker runs orca-sandbox-start in the VM and returns orcad's ready JSON (pairingCode, projectRoot). Pairing codes are never logged or stored in D1. v1 hardening: the desktop sends a device public key and the VM pre-authorizes it, so the control plane never carries a usable pairing secret. Endpoint branding: v0 dials the *.vercel.run route internally (not shown in UI); v1 fronts it with *.sandbox.onorca.dev if white-label needs to extend to network inspection.

Golden image (cloud/apps/sandbox-control/image/)

Operator-only script (build-golden-image.ts, dry-run by default). Steps: create build sandbox (node24, 4 vCPU, non-persistent, 40 min) → upload pinned orcad linux-x64 bundle (sha256-checked) → provision-golden.sh: apt basics, gh, pinned agent CLIs (Claude Code, Codex, OpenCode, Gemini), git credential helper + orca-sandbox-start, orca-sandbox-drain, orca-egress-probe → smoke (orcad reaches ready, CLIs --version) → wipe orcad identity and any auth homes (clone-safe, risk #1) → snapshot with expiration: 0 → delete build sandbox → print manifest (snapshotId, versions, sha256) for GOLDEN_SNAPSHOT_ID. Snapshot → orcad ready measured at 3.3–4.3s.

Network policy (fail closed)

Compiled per sandbox by compileNetworkPolicy():

• Allowlist record form, no "*" entry (unlisted domains don't resolve). Default list: package registries (npm, pypi, crates, go proxy, rubygems), github.com + codeload + objects, agent APIs (Anthropic, OpenAI/ChatGPT, Google), and Orca endpoints. Workspaces can opt into "open" (adds "*": []), never silently.
• Credential hosts get [...inject rules, {response: 403 orca-egress-denied}]: a request without our placeholder is refused, so the real token can't be paired with an attacker-chosen request shape.
• GitHub: VM sets http.https://github.com/.extraheader = Authorization: Basic b64(x-access-token:<placeholder>) so every git request carries the placeholder (git otherwise sends an unauthenticated probe first, which the trailing deny would 403). The control plane mints a 1h GitHub App installation token and rewrites the policy before expiry.
• Node fetch hang: with a transform rule, the Vercel proxy answers HTTP/1.1 with no content-length or transfer-encoding; undici waits forever. curl/git (libcurl) and Go (gh) are fine. Handling: (1) transform is used only for hosts whose clients are curl/Go (github.com git traffic); (2) hosts Node clients use (api.github.com) get a forwardURL rule to POST /v1/egress/proxy on the control plane instead, which verifies Vercel's sandbox OIDC token (oidc.vercel.com, aud = forward URL), maps sandbox_name → our row, injects the header and returns a normally framed response — this also keeps users' GitHub tokens out of Vercel's policy store; (3) the image ships orca-egress-probe that does a bounded Node fetch per credential host at boot and reports egress_framing_hang instead of letting agents hang. File the framing bug with Vercel. Needs a live test.
• Agent sign-in is native inside the VM (unmodified CLI, Anthropic-compliant); model hosts are allowlisted, not injected. Auth homes live on the sandbox disk and persist across stop/resume.

Lifecycle, idle stop and the 24h cap

• One persistent sandbox per workspace. Orca owns the timer: Vercel timeout is a 30 min lease that the desktop renews while orcad reports busy; if the desktop disappears the lease lapses and Vercel stops the session (files kept).
• Idle stop: 20 min without clients, agents or PTY input → suspend.
• Opportunistic rollover after ~12h when all agents idle and PTYs quiet ≥2 min. Forced rollover at 23h50m: warn at T−60 and T−15, stop admitting turns at T−10, drain (orcad SIGTERM → durable restart offers, scrollback/cwd hints) up to 5 min, stop, resume, start orcad. One state machine covers idle stop, cap, crash and "Restart".
• After reconnect the desktop auto-continues the durable offers (agentSession.restartResumable → agentSession.restartContinue) for sessions the rollover interrupted, so agents carry on without user action.

Failure / leak handling

• Write-ahead D1 row before any Vercel call; the deterministic name makes create retries idempotent (GET by name first). Cron reconciles rows stuck in creating/destroying.
• Destroy deletes the sandbox with deleteOrphanSnapshots=true, then lists snapshots by name and deletes any left (bake-off: ~325 MB auto snapshots survive delete otherwise).
• Vercel errors map to catalogued codes; bodies are never echoed to the desktop.

Security

• Secrets via wrangler secret put only: VERCEL_TOKEN (team-scoped, Sandbox only), CONTROL_PLANE_TOKEN, GITHUB_APP_PRIVATE_KEY. Worker refuses to start lifecycle routes if VERCEL_TEAM_ID/VERCEL_PROJECT_ID are missing. No routes / workers_dev:false in the checked-in wrangler file.
• Per-account authorization on every route (row account_id must equal the caller's); ids are random.
• Strict zod bodies, 16 KiB limit, no redirects on provider calls, provider error bodies dropped.

GitHub App + billing dev (2026-11-09): app "Orca Cloud (dev)" (stablyai, id 5677658, https://github.com/apps/orca-cloud-dev; contents/pull_requests write, metadata read, no webhook) created by the manifest flow; key only as a Worker secret. From a real sandbox via the dev Worker: private repo cloned at create (6.6s total), real git push of a new branch in 1.0s with the transform-injected repo token, gh and Node fetch to api.github.com through the forwardURL proxy with a minted token (200, ~1s), no token on the VM disk. Billing dev is reached over a service binding (same-account workers.dev fetches fail with 1042): an account without a subscription gets 403 no_subscription; the cron's suspensions poll runs clean. dev:* operator accounts bypass billing outside prod.

Desktop dogfood (2026-11-09, dev app → dev Worker → Vercel Pro): "Orca Cloud" is a built-in provider in Run on (reserved id cloud); create → orcad ready in the app 8–12s with the private repo cloned via the GitHub App; shell work + real git push from the cloud workspace; 15-min cap warning toast; forced rollover at the (shortened) cap with reconnect and files kept; idle sleep through the ordinary Sleep flow after a (shortened) 2-min idle; wake on click; destroy through runtime cleanup leaves no sandbox or snapshot. Found live and fixed: agent hosts must be wildcard-allowlisted (platform.claude.com), the runtime store dropped provider (now in the feature store), remote keystrokes are not 'driving' input (verdict reads all input), Vercel answers 400 when deleting an already-deleted snapshot. Not run: agent auto-resume with a real chat (needs a Claude/Codex sign-in).

Built (mock-tested) and next wiring

• (b) src/main/environment-providers/*, src/main/hosted-sandbox/* — driver, client, contract kit, recipe adapter.
• (c) cloud/apps/sandbox-control — Worker, D1 schema, Vercel REST client, entitlements/usage, egress proxy, cron.
• (d) cloud/apps/sandbox-control/image — golden image builder + VM entry points.
• (e) hosted-session-lease-policy.ts / -scheduler.ts / hosted-agent-auto-resume.ts — lease renew, T−60/T−15 warnings, idle suspend, opportunistic + forced rollover, then restartResumable → restartContinue for offers recorded since the drain (older offers stay with the user). The cron only force-stops 5 min after the deadline.
• Next: a built-in "Orca Cloud" recipe source whose create/resume/suspend/destroy call the driver through the recipe adapter; start one scheduler per ready hosted runtime (stop on suspend/destroy); an orcad RPC exposing the idle-probe verdict + last PTY input for readActivity; header warning + mobile push for warn.

Live results (2026-11-09, Vercel Pro team lovecast, project orca-sandbox-control, iad1)

Driven by the real Worker handler in-process (real Vercel REST, node:sqlite D1, stub billing) and the real desktop driver/scheduler. Base OS is Amazon Linux 2023 (dnf), not Ubuntu.

• Golden image: orcad built from public stablyai/orca@7c14832 in a 8 vCPU build sandbox in 98s (67 MB tgz); provision 14.3s (gh 2.103.3, Claude Code 2.1.319, Codex 0.166.0; orcad loopback smoke passed); wipe ok; snapshot 8.2s. Bug found and fixed: provision downloaded the bundle over its own source path.
• Snapshot → orcad ready (POST /v1/sandboxes, incl. Vercel create + orca-sandbox-start): 5.4 / 4.1 / 4.9 s; WS open.
• Suspend (drain + stop) 7.6s; resume → ready 6.0s; same endpoint and same orcad host key (no re-pair).
• Shortened cap (6 min): scheduler warned T−15, then forced rollover at the deadline in 17.3s (drain, stop, boot, orcad ready), reconnect WS open, new deadline set. Destroy 0.4–0.7s; zero sandboxes/snapshots left.
• Egress: allowlist resolves npm (200), example.com does not resolve (curl rc 6). Policy update 0.25s. Placeholder → injected fake GitHub credential reaches GitHub (curl/gh token scheme: 401 Bad credentials); attacker token and placeholder-less git → 403 orca-egress-denied. Node fetch under a transform rule still hangs on Pro; orca-egress-probe reports egress_framing_hang (npm, untransformed: ok). No real GitHub token was used, so a successful authenticated push is still unverified.
• Not covered live: agent auto-resume with a real chat (needs a real Claude/Codex sign-in).

Dev deployment (2026-11-09): Worker https://orca-sandbox-control-dev.lovecast.workers.dev, D1 orca-sandbox-control-dev, config wrangler.dev.jsonc, billing stubbed (no billing dev Worker yet). Through it: create 5.1s, rollover 12.2s (same host key), destroy 1.0s, no snapshots left. Egress proxy end to end from a real sandbox: Bearer and gh token placeholders forwarded and re-injected (GitHub answers the injected fake: 401), Node fetch completes in 450 ms with chunked framing (the transform-rule hang is gone), attacker token / no auth → 403.

Secret store and injection (org / personal / repo)

Product decision: "API-key injection" in the Conductor parity matrix (form 2 = this store; form 3 = egress swap).

• Store (src/secrets.ts, migration 0003): D1 secrets keyed (scope, owner_id, name). Envelope encryption (src/secrets-crypto.ts): per-secret random AES-256-GCM DEK; DEK wrapped by the KEK (SECRETS_KEK, base64 32 bytes, Worker secret only; id SECRETS_KEK_ID, default k1). Both layers take ["orca-secret-v1", layer, kekId, scope, ownerId, name] as AAD, so a row copied onto another owner/name fails to open (skipped at injection, logged by name). Rotation: set the new KEK + id, move the old one into SECRETS_KEK_RETIRED ({"k1":"…"}); rows re-seal on their next write.
• API (desktop JWT): GET /v1/secrets?scope=org|personal|repo[&repo=owner/name], PUT /v1/secrets/:scope/:NAME {value}, DELETE …. Write-only values: responses carry name, scope, kind, updatedAt, updatedBy and a last-4 hint for values ≥16 chars. Owners: org = the org account (org:<id>), personal = user:<sub>, repo = <account>#github.com/<owner>/<repo>. Org secrets and an org account's repo secrets need orgRole owner/admin on the token (any member lists names); personal secrets are the user's alone; dev tokens stay under dev:.
• Validation: ^[A-Z_][A-Z0-9_]*$ ≤128; reserved names (PATH, HOME, NODE_OPTIONS, proxy/CA vars, GIT_SSH*, GH_TOKEN/GITHUB_TOKEN, …) and prefixes (LD_, DYLD_, ORCA_, VERCEL_, BASH_FUNC_, GIT_CONFIG). Values ≤8 KiB, no NUL; 100 secrets per owner; 128 KiB injected per start. Never stored: CLAUDE_CODE_OAUTH_TOKEN, any value starting sk-ant-oat, JWT-shaped ANTHROPIC_API_KEY/OPENAI_API_KEY (ChatGPT/Claude.ai sessions).
• Precedence: env secrets repo > personal > org; agent keys (ANTHROPIC/OPENAI/GEMINI/GOOGLE/CURSOR keys, AWS Bedrock and Vertex vars) org-shared > repo > personal. Per name, so a partial org Bedrock set mixes with personal.
• Injection: every boot (create, resume, rollover) resolves secrets and passes them to orca-sandbox-start as ORCA_SECRET_ENV (NAME=<base64> lines). The script decodes them into orcad's env -i launch env (agents inherit it) through a pipe, unsets the variable, and writes nothing to disk, so the stop snapshot never holds them and orca-image-wipe has nothing to wipe. Edits apply on the next start/resume, no image rebuild. Values transit Vercel's command API (as the VM's env would anyway); logs carry counts and names only.
• Phase 2 egress swap (src/secret-egress.ts, SECRETS_EGRESS_PLACEHOLDERS=on, off by default): VM env gets orca-ph-anthropic-<sbx> / orca-ph-openai-<sbx>; boot records the hosts on the row (egress_secret_hosts) and the policy gets forwardURL claims (api.anthropic.com: x-api-key: <ph> and Authorization: Bearer <ph>; api.openai.com: Authorization: Bearer <ph>) followed by the 403 deny. The egress proxy (OIDC-verified, as for api.github.com) swaps in the real key, cached per session for 60 s. Transform rules are not used: they hang Node clients (Claude Code, Codex), the forwardURL path is the one proven live for api.github.com. Not on by default because (1) SSE model streams through Vercel forwardURL → Worker are not yet tested live, and (2) with a key routed, api.anthropic.com denies everything else, including a subscription OAuth Bearer or a plain ANTHROPIC_AUTH_TOKEN in the same VM. Enable on dev after a live streaming test with the rebuilt image.
• Needs: golden image rebuild (new orca-sandbox-start), and Orca Cloud auth minting orgRole on sandbox tokens (stablyai/orca-cloud#695).

GitHub installation binding, guardrails, retention (2026-11-09)

• Repo tokens need a binding: github_installations(installation_id, account_id). tokenForRepo(url, accountId) looks up the repo's installation, refuses unless the caller's account holds a live binding (re-checked on cached tokens too), mints for that one repo (contents/pull_requests write, metadata read) and rejects a wider response. Binding: desktop POST /v1/github/connect → signed (HMAC), single-use, 10-min state carrying the Orca account → GitHub install / OAuth → /v1/github/setup + /v1/github/callback exchange the code for a user-to-server token and bind only installations on the GitHub user's own account or on an org where they are an active admin (/user/memberships/orgs, needs the app's Members:read). Installation ids in URLs are never trusted. Operator binding for support/dev: POST /v1/internal/github/bindings {installationId|repoUrl, accountId}.
• Guardrails before billing: kill switch (D1 flag via POST /v1/internal/kill-switch, or SANDBOX_CREATES_DISABLED) refuses create/resume/rollover with 503 maintenance and drains running sandboxes (suspend, snapshot kept; cron keeps draining while on); global SANDBOX_MAX_RUNNING (50, 503 capacity); per-account 3 × seats backstop (403 concurrency_limit); CREATE_RATE_PER_MINUTE (10, 429).
• Hourly housekeeping: provider sandboxes tagged with this env and no live row are stopped on first sight and deleted after ORPHAN_GRACE_HOURS (24); automatic snapshots whose sandbox no longer exists anywhere in the project are deleted after the same grace (manual/golden never; partial listings delete nothing). Canceled/unpaid accounts (suspensions list + daily standing probe; outages never start a clock) get a warning hook at T−7 days and are destroyed DATA_RETENTION_DAYS (30) after; suspended sandboxes idle IDLE_RETENTION_DAYS (90) likewise. New sandboxes set snapshotExpiration to 104 days (backstop). Destroyed rows drop repo_url.
• Live on dev: unbound account → private clone refused (no token minted); after operator binding → clone + push + gh/Node via proxy OK; kill switch drill drained 1, refused create/resume, resume OK after lifting.

What needs live access (not done)

Vercel Pro team id + project id + a Sandbox-scoped token (as Worker secrets), wrangler auth for a dev D1, a billing Worker dev deployment + CONTROL_PLANE_TOKEN, and an Orca Cloud auth change to mint aud=orca-sandbox tokens. Live checks then: golden image build, snapshot→ready timing on Pro, forwardURL + OIDC proxy, Basic-auth inject for git, 24h rollover rehearsal with a shortened cap.