Jinwoo HongandE2E Test
20fc7af557
* feat(relay): add the per-cell phonePath switch, effective only in reserve mode Step 6 of the relay redesign keys every phone-path change on this switch. The cell reports the effective value in runtime status, and the flag tool refuses phonePath=memory unless the cell is reserve, and a reserve flip that would turn phones over unnamed. * feat(relay): resume paired phones from one device read on memory cells With phonePath=memory in reserve mode, a phone resume is admitted from one device read and the cell's own seat, and takes no activity leases. A refused device closes 4401, a failed read 4503, and only a missing device row falls through to the invite path. A phone whose host has no seat gets one assignment read: 4409 when the row names another cell or the read fails, otherwise 4404. * feat(relay): keep a memory resume's basis in the cell and confirm it from one read A phone resumed on the memory path keeps its connection basis in the cell, not in relay_connection_bases. Its confirm rereads the device row once, renews it only when less than 29 days remain, and always answers a current-version confirm renewed:true with the expiry the database holds. Confirms route by where their basis lives, so a flip between accept and confirm completes either way. Retries answer from a 10-min in-memory result map. * test(relay): roll a memory phone cell back with confirms in flight Flipping back by admitMode=db, phonePath=db or a dead-man trip leaves in-flight confirms to finish on their memory basis, writes no phone lease and leaks none, and evacuation still refuses the cell while Postgres records it as reserve. * feat(relay): batch a memory cell's resume-confirmed audit rows Memory confirms buffer their audit rows and the cell writes them as one multi-row insert every 5 s or 500 rows, idempotent by a per-event id. The buffer holds at most 5,000 rows and drops a row older than 10 minutes rather than retrying it, counting and logging both kinds of drop. * feat(relay): count and log phone-path failures, and every attach deadline The runtime metrics line gains phone refusals by close code, attach deadlines, accept database errors, resume confirm failures by code, completed accepts by phone path and the resume audit drop total. Every attach-deadline 4404 now logs what the cell knew of the desktop's control (pong age, ping outstanding, control age and round trip), so a half-open control can be told from a slow data dial. * feat(relay): close an attach-deadline 4404 with the attach-timeout reason The phone close reason from #30607 now rides the close frame of the 4404 a cell sends when the desktop's data leg misses the attach deadline, so OTA phones can redial the same cell once instead of waiting out the host-offline delay. The failed relay-hello frame is unchanged, and old phones ignore the reason. * test(relay): drill a memory cell through a database outage, and pin phone renewal A blocked database leaves splices forwarding, closes new resumes 4503 and fails confirms cleanly, lets an in-flight confirm finish on its memory basis after a flip back, and drops audit rows past 10 minutes instead of writing them late. On the phone, a relay-only session lives 31 days under the once-a-day renewal rule, and would not if a not-due confirm answered renewed:false. * fix(relay-ops): store phonePath=db as no key, so old images keep taking writes An explicit phonePath=db left in the switch file would make the flag tool refuse every later write after an image rollback, including a watcher's admitMode=db flip back. * fix(relay): log a failed no-seat read, recheck the switch after the device read A failed assignment read for a phone with no seat now logs a rate-limited line before its 4409. A switch that turned off during the device read sends the phone down today's path. resolveResume reuses the one-read lookup, so the expiry and grace rules have one copy. * fix(relay): log a failed memory renewal; test flips, replays and basis cleanup A renewal write that fails still answers the stored expiry, but now logs a rate-limited line so a persistently failing update is visible before phones age toward re-pairing. * test(relay): assert a rolled-back memory splice issues no statement when it closes * fix(relay): flush resume audit early only on crossing the batch size, drop the shutdown flush A failing database no longer takes a 500-row insert per confirm; retries wait for the 5 s timer. The shutdown flush raced the pool closing, so it is gone and the at most 5 s loss is documented. A stopped buffer no longer re-arms its timer. * test(relay): assert the aged audit rows were dropped, not written --------- Co-authored-by: E2E Test <e2e@test.local>
Orca Relay
The relay that connects the Orca mobile app to a desktop host. Phones and desktops never talk to each other directly: each opens an outbound WebSocket to a relay cell, the relay pairs the two sessions, and it splices frames between them. A director assigns hosts to cells and coordinates migrations; cells carry the user connections.
This directory is an independent pnpm workspace inside the Orca monorepo. Run
its commands from cloud/, not the repository root. The source is covered by
the repository's root MIT license.
Packages
packages/relay-contract: the wire contract shared by the relay, the desktop app, and the mobile app (frame shapes, close codes, admission budgets, splice state machine).apps/relay: the relay server. The same image runs as a director or a cell depending onORCA_RELAY_ROLE.apps/relay-fence-broker: a private, IAM-only service that owns the durable mutation lease, the Terraform checkout, and the narrow Compute mutation used when a registered target is superseded. The workflow that calls it holds read and invoke rights only, never those mutation permissions.apps/relay-ops: the relay operations console and the incident monitor behindpnpm ops:relay,pnpm incident:relay, andpnpm incident:relay-preflight.apps/pushandpackages/push-contract: the mobile push gateway that holds the APNs key and sends to phones through APNs and FCM, and its wire contract. It is deployed and operated from here but is not part of the relay data path; see docs/push-gateway.md.
Mobile push gateway
apps/push is a separate Cloud Run service from the relay. Phones never hold an
Orca credential for it: the desktop host authenticates with the same X25519
key it uses for the relay, answering an encrypted challenge to mint a 24 hour
session, then registers each paired phone's native push token and asks the
gateway to push. The gateway queues each event as its own notification,
enforces per-host quotas and request limits, and retires a
registration as soon as Apple or Google reports the token unregistered.
Provider push is the only ordinary mobile OS-banner path. The notification
socket is retained only for live dismissal and reconnect tray reconciliation;
it never creates or recovers banners. Desktop notification categories remain
authoritative.
Each delivery is persisted as one notification event. Before deploying an
incompatible queue format, stop all older push gateway revisions and clear only
unpublished push delivery fixtures; no queue preservation or migration is required.
FCM notification messages are inherently collapsible while offline and have a
small concurrent collapse-key budget, so every pending alert is not guaranteed.
Storage follows the relay pattern: PostgreSQL in production, SQLite for tests
and local development. Configure it with ORCA_PUSH_PUBLIC_URL, ORCA_PUSH_FCM_PROJECT_ID,
ORCA_PUSH_DATABASE_URL, the three APNs variables (ORCA_PUSH_APNS_KEY,
ORCA_PUSH_APNS_KEY_ID, ORCA_PUSH_APPLE_TEAM_ID, all three or none), and
optionally ORCA_PUSH_APNS_TOPIC. The FCM credential comes from
the runtime service account, so no key material is configured for Android. See
push gateway operations for deployment and recovery.
Logging is aggregate counters only. Tokens, notification titles, notification bodies, and full host fingerprints never reach a log line.
Infrastructure and operations
infra/terraform: the relay Terraform root. It owns the cells, the director, the shared Cloud SQL instance, DNS, observability, and every GitHub Workload Identity provider the relay workflows authenticate through.backend/holds the per-environment backend configuration andenvironments/the tfvars. Drive it throughpnpm infra:init,pnpm infra:plan, andpnpm infra:apply.dev/scripts: the deploy, capacity, admission, rehome, monitoring, and load scripts the workflows call, plus the contract tests that pin each workflow and Terraform surface. Run them withpnpm test.dev/contractsanddev/fixtures: the checked-in data those contract tests read, including the Terraform root partition.docs/: the relay runbooks, capacity-testing guide, incident-monitor reference, the workflow variable reference indocs/relay-workflows.md, and the push gateway runbook indocs/push-gateway.md.
Workflows
The 25 .github/workflows/cloud-*.yml workflows are the deploy and operate
surface: publish and deploy the director, roll GCE cell capacity, operate Asia
admission and regional rehoming, prove staging capacity, monitor production,
power staging up and down, and deploy the mobile push gateway.
.github/actions/cloud-sql-rollout-lease is the compare-and-swap lease that
serializes rollouts against the shared Cloud SQL instance. Push reuses that
action with its own lease object and deployment concurrency group.
Every one of them is inert. Each top-level job is gated on
vars.ORCA_CLOUD_OPERATIONS_ENABLED == 'true', a repository variable that is
unset here, so the two scheduled triggers and every manual dispatch skip
without running a step. Only the repository owner, holding the GCP identities
these workflows authenticate as, can turn them on.
Cloud Verify is not gated. It builds, typechecks, lints, tests, secret-scans,
and validates the relay Terraform on every change under cloud/, and it runs
on fork pull requests, so it configures no backend and holds no credential.
What is not here
The terraform-foundation and terraform-apps roots and the API and auth
services live in the private stablyai/orca-cloud repository. Scripts and
tests that spanned both trees were narrowed to the relay side rather than
carrying a dangling reference.
Local development
cd cloud
pnpm install
pnpm build
pnpm test
pnpm test runs the SQLite-backed suites. Tests that need PostgreSQL run only
when ORCA_RELAY_TEST_POSTGRES_URL points at a disposable PostgreSQL 16 or 17
database, for example:
docker run --rm -d --name orca-relay-pg -e POSTGRES_HOST_AUTH_METHOD=trust \
-e POSTGRES_DB=orca_relay_test -p 55440:5432 postgres:16-alpine
ORCA_RELAY_TEST_POSTGRES_URL=postgres://postgres@127.0.0.1:55440/orca_relay_test \
pnpm --filter @orca-cloud/relay test
docker rm -f orca-relay-pg
Configuration is read from environment variables validated in
apps/relay/src/config.ts. ORCA_RELAY_ASSIGNMENT_SIGNING_KEY (at least 32
bytes) is the only required value; everything else has a local default.