Commit Graph
1052 Commits
Author SHA1 Message Date
Neil 362b8b32bb Merge remote-tracking branch 'private/cloud/integration' into cloud/dev-seed-gate
# Conflicts:
#	cloud/apps/sandbox-control/src/index.ts
2026-11-16 01:40:20 -08:00
Neil 1d95740dc5 fix(cloud): every dev-only route and behavior is on only where SANDBOX_ENV is dev, local or test (devOnly allowlist; was 'anything but prod') 2026-11-16 01:34:35 -08:00
Neil 3fee610d06 fix(cloud): the dev GitHub seed route exists only where SANDBOX_ENV is dev, local or test (allowlist; was 'anything but prod') 2026-11-16 01:32:20 -08:00
Neil 80fdb67d64 Merge remote-tracking branch 'private/cloud/integration' into cloud/team-sharing-hold 2026-11-16 01:30:35 -08:00
Neil 6057044fd3 feat(cloud): TEAM-SHARING capability on the single-workspace response too 2026-11-16 01:11:48 -08:00
Neil 3e12a58b15 fix(cloud): TEAM-SHARING a revoke queued while the workspace slept leaves the next session private (queue-time stamp kept; no session = not shared for it) 2026-11-16 01:07:26 -08:00
Neil a7df7fbbc8 fix(cloud): Make private keeps shared credential mode until the next boot too (a private row with a grant revoked this session); tests for it and concurrent sweeps 2026-11-16 01:06:01 -08:00
Neil 62881cb01e test(cloud): GIT-ACTOR read-count test's synthetic ids can no longer collide with the real one (flaky)
(cherry picked from commit e735f41be6)
2026-11-16 01:05:51 -08:00
Neil b99fa165c1 fix(cloud): TEAM-SHARING hold keeps shared credential mode for the session a teammate's grant was revoked in (M1); sweep only team workspaces (L1) 2026-11-16 01:04:38 -08:00
Neil 3aa45164f0 fix(cloud): GH-SOLO a teammate whose grant is still being revoked keeps the workspace from being solo 2026-11-16 01:01:02 -08:00
Neil ce48b4769f fix(cloud): GH-SOLO solo is decided from the people set (owner without a grant counts), never from no expiry; re-check after the VM write and drop the mark if a grant began 2026-11-16 00:59:46 -08:00
Neil 05fe1508f2 feat(cloud): TEAM-SHARING hold sweep revokes non-owner grants (team_sharing_held); credential mode follows effective visibility once no teammate grants remain 2026-11-16 00:58:26 -08:00
Neil e735f41be6 test(cloud): GIT-ACTOR read-count test's synthetic ids can no longer collide with the real one (flaky) 2026-11-16 00:57:33 -08:00
Neil c122522e53 fix(cloud): GH-SOLO a one-person workspace's gh and git act as that member with no attested sender (a TUI typed into directly), matching the control plane 2026-11-16 00:54:03 -08:00
Neil 79781e805c feat(cloud): TEAM-SHARING hold: TEAM_SHARING (default off) refuses sharing with the team and makes stored team workspaces act private 2026-11-16 00:50:25 -08:00
Neil a66b711d81 perf(cloud): GIT-ACTOR the workspace list reads grants, sends and GitHub connections once per request, not per row (one pure rule, pickActing) 2026-11-15 22:54:50 -08:00
Neil 8769dddccc fix(cloud): a device id claimed by more than one member maps to nobody in the VM's devices map (no last-wins attestation as another member) 2026-11-15 22:52:34 -08:00
Neil bf0f77f791 feat(cloud): GIT-ACTOR as githubActing + you on the workspace view (list and detail), with the desktop wire schema 2026-11-15 22:51:11 -08:00
Neil d3bd6a8475 feat(cloud): GIT-ACTOR who pushes and PRs go out as right now, on GET status (owner) and GET team workspace (members), behind feature git-actor 2026-11-15 22:48:20 -08:00
Neil 9a770d4686 feat(cloud): GIT-COAUTHOR a commit that falls back to the workspace identity while a known member drove it carries Co-authored-by for them 2026-11-15 22:48:20 -08:00
Neil 36303ac480 fix(cloud): GH-AS-MEMBER gh wrapper actually runs: the real gh moves off PATH (tools/gh/bin) and tools/bin/gh links to the wrapper; older images migrated at entry-point install 2026-11-15 22:37:07 -08:00
Neil e994d835ba fix(cloud): GH-AS-MEMBER approve/merge path checks case-insensitive with optional trailing slash; an unread review body counts as an approval 2026-11-15 21:24:54 -08:00
Neil 3d96f7d720 fix(cloud): GH-AS-MEMBER approvals never as a member (any workspace); merges as a member only in a one-person workspace (incl. REST branch merges); approve event case-insensitive; shared cap default 60 min 2026-11-15 21:23:00 -08:00
Neil 730334b4f5 fix(cloud): GH-AS-MEMBER PRs, comments and pushes from a cloud workspace show as the member, not the bot
stablyai/orca#42960 (Neil, dev) was authored by orca-cloud-dev[bot]. Neil had connected GitHub; member
identities only reached git: gh ran on the workspace placeholder (the App's installation token), and
the Worker's member route on api.github.com allowed only REST under the repo while gh pr create is
GraphQL.

- VM: a gh wrapper runs gh with the attested sender's own placeholder (orcad attests, as for git);
  plain gh otherwise. Behind GH_AS_MEMBER=on (dev on).
- Worker, api.github.com with a member placeholder: the member's own token only for REST under the
  workspace repo and for allowlisted authorship mutations whose every target resolves to it
  (repositoryId equals its node id; other ids looked up with the installation token, cached per repo,
  failures not cached). graphql-js parse; variable defaults, owner-level projectIds, duplicate
  top-level keys and anything unexpected are not member requests. Everything else goes out on the
  installation token as before; an out-of-repo pinned mutation is refused with a GraphQL error.
- Shared workspaces never approve or merge as a member (GraphQL and REST): those go out on the
  installation token, so branch protection still applies.
- Identity rule: the only person who can use a workspace always acts; in a shared one the latest
  sender acts until another member sends, for at most MEMBER_GIT_SHARED_MAX_MINUTES (default 480).
- A GitHub connect or disconnect mid-session reaches the running VM's identities now.
- Tokens never enter the VM.
2026-11-15 21:18:00 -08:00
Neil 3136c355a9 feat(cloud): dev-only session-token-free probe route
Puts the token-free compile on a probe account's LIVE session, so the probe can see whether a
header-matched respond rule answers at all (separating 'respond not enforced' from 'resume restored
something else').
2026-11-15 16:04:50 -08:00
Neil 6314a21518 fix(cloud): provider-policy audit breaks rules down per domain; echo-probe reports the HTTP status
egress-docs' A4 run: after resume-raw the echo to the custom base-URL host answered 200 where the
token-free record should answer the placeholder 403. The Worker's token-free compile is right (the
host keeps its respond rules: checked in the harness), so the probe needs to see what Vercel holds
per host and what status came back:
- the audit adds domains: {host: {injection, forward, response}} (hostnames only);
- echo-probe returns the HTTP status again. The echoed body stays in a shell variable, never a file.
2026-11-15 16:04:05 -08:00
Neil bb8fc53fb8 Merge pull request #807 from nwparker/cloud/relay-warm
chore(cloud): keep one ChatGPT relay instance warm
2026-11-15 15:59:38 -08:00
Neil 3131fbdc04 docs(cloud): relay README: the warm instance, and keeping it in production 2026-11-15 15:38:55 -08:00
Neil 7b3ff2df30 feat(sandbox-control): EGRESS-REFRESH central credential refresh and push outbox 2026-11-15 15:38:30 -08:00
Neil f16725a7d1 chore(cloud): keep one ChatGPT relay instance warm
A Cloud Run cold start cost Codex's first chatgpt.com call ~10.8 s on dev (seen twice on
2026-11-15, each right after an idle period). min_instance_count 0 -> 1 on staging (applied).
2026-11-15 15:38:11 -08:00
Neil 5a6532ca93 Merge remote-tracking branch 'private/cloud/integration' into cloud/egress-a4-scrub 2026-11-15 15:33:20 -08:00
Neil 602f923f1e Merge pull request #805 from nwparker/cloud/policy-audit-fixes
fix(cloud): provider-policy audit counts injection/forward/response rules; dev resume-raw and echo-probe
2026-11-15 15:33:08 -08:00
Neil 5210eca04f fix(cloud): echo-probe writes nothing in the VM; resume-raw/echo-probe only on probe accounts
egress-docs' review of #805: the echoed request goes straight into grep -c (a swapped value never
touches the VM's disk), and both routes refuse any row outside the throwaway probe accounts
(account_id dev:egress-…).
2026-11-15 15:03:50 -08:00
Neil 7da2415f79 fix(cloud): A4 review R2: a record scrub never strips a workspace that resumed meanwhile; R4 note
scrubStoppedRecord re-reads the row right before the PATCH and only scrubs one that is still
suspended or archived with no session; if the row resumed while the PATCH was in flight, the live
policy is re-applied at once (reason scrub_race). Failed rows are left out of the sweep on purpose:
failAndRelease destroys their provider sandbox.
2026-11-15 15:02:05 -08:00
Neil 326340b8cd fix(cloud): provider-policy audit counts injection/forward/response rules; dev resume-raw and echo-probe
egress-docs' probe 2 re-run through #803 found transformRuleCount always 0: Vercel's redacted shapes
name header swaps injectionRules (session) next to forwardRules and responseRules. The audit now
reports each kind separately (count + header names), so a scrubbed policy is checkable: 0 injection
rules, response rules only.

Two more dev-only operator routes so the probe needs no personal Vercel login:
- POST …/resume-raw: resume the provider session from its stored record with no policy write of
  ours (the row is left as it was; dev throwaways only).
- POST …/echo-probe?marker=orcaprobebogus…: one fixed request from inside the VM to httpbin.org
  or postman-echo.com carrying a placeholder header (placeholders only, validated); the answer
  is whether the echo contained the marker. No body comes back or is logged.
2026-11-15 15:00:25 -08:00
Neil 97c12b73f7 fix(cloud): A4 the post-resume policy write retries with backoff, and a lasting failure is visible
A wake resumes from the scrubbed record, so until its policy write lands the firewall answers the
placeholders itself (the keys-withheld or signed-out errors agents print). The write now retries
(250 ms, 1 s, 3 s); if it never lands, the wake still completes (fail closed, never a hang), logs
policy_apply_failed {reason: boot} by code, and marks policy_stale_at for reconcile's next tick.
2026-11-15 14:58:22 -08:00
Neil a5deecca4b cloud: renumber the archive hook migration to 0064 (0063 landed concurrently) 2026-11-15 14:57:55 -08:00
Neil 4b311e95f3 fix(cloud): unattended archives bound the hook to 30 s; an archived workspace is never offered a wake for it 2026-11-15 14:57:54 -08:00
Neil 24de08a2a5 fix(cloud): a failed archive hook run after a successful one makes it pending again 2026-11-15 14:57:52 -08:00
Neil cc5211efd9 cloud: renumber the archive hook migration to 0063 (0062 landed concurrently) 2026-11-15 14:57:51 -08:00
Neil a2551a3190 fix(cloud): archive hook picks checkouts in git's worktree order (the primary is listed under its real path) 2026-11-15 14:57:51 -08:00
Neil 0cd490186d feat(cloud): run the repo's orca.yaml archive hook in the VM before archiving or deleting a cloud workspace
Archive and delete stop the VM but nothing it started elsewhere (a preview
database branch keeps billing). The control plane now runs scripts.archive
on the still-running VM: bash with set -e, as the user, with the session's
env and secrets, 2 min for all checkouts, output per checkout and in
~/.orca-sandbox/archive-hook/archive.log.

- POST/GET /v1/sandboxes/:id/archive-hook; migration 0062 remembers whether
  the repo has a hook (learned at each sleep) so an asleep archive/delete can
  offer a wake for it, and never wakes one without a hook.
- Desktop archive runs it before the Sleep; a failure shows the output with
  Archive anyway / Keep it. Asleep: the Undo toast says it wakes first and
  offers Skip hook.
- Asleep/unreachable delete runs it through the control plane; a failure
  refuses with the usual Delete anyway waiver.
- Unattended archives (public API, team list) run it on a running VM and
  archive even if it fails (logged).
- Docs: orca.yaml scripts.archive for Orca Cloud workspaces.
2026-11-15 14:57:51 -08:00
Neil 4df5205c66 feat(cloud): A4 stopped sandboxes keep no credential at rest in the provider
Probe 2 (egress-docs): a session's last network policy survives a stop or a timeout on the
persistent record and is re-applied on resume, so firewall-mode values would sit at rest in Vercel
while a workspace sleeps.

- After every voluntary stop (sleep, rollover, restart, archive) and when reconcile finds the
  provider stopped a session (timeout), the record is PATCHed with the token-free compile (same
  routes, every firewall value answered by the firewall, no App token). A failed PATCH sets
  policy_scrub_pending.
- Reconcile scrubs stopped rows that are pending, or whose last policy carried a firewall class and
  was written after their last scrub (timeouts, crashes, and a one-time sweep on deploy).
- A wake PATCHes the record with the full policy right before resuming (no gap); the post-resume
  apply stays as the backstop.
- Boots stamp policy_config_hash (and a generation one below now), so create and wake writes are
  covered like applies. Proxy-only policies hold no values: nothing to scrub, no record writes.
- Migration 0067 (0064 and 0066 are held by #783 and #792): policy_scrubbed_at,
  policy_scrub_pending.
2026-11-15 14:56:20 -08:00
Neil 7ea40ac042 feat(cloud): dev-only redacted provider-policy audit, and A4's record scrub primitive
For egress-docs' probe 2, re-run on the Worker's own Vercel token (dev only; elsewhere these routes
don't exist; operator token):
- GET /v1/internal/sandboxes/:id/provider-policy-audit?marker=orcaprobebogus…: the raw record and
  session read from Vercel, answered as redacted facts only (state, session status, whether a
  network policy is present, transform rule count, header names, markerFound by substring test).
  No value or body is returned or logged.
- POST /v1/internal/sandboxes/:id/provider-policy-scrub-test: A4's primitive, scrubRecordPolicy,
  which PATCHes the persistent record with the token-free compile (every firewall value answered by
  the firewall, no App token; proxy routes hold no values anyway).

compileSandboxPolicy gains { tokenFree }; the provider gains updateRecordNetworkPolicy and a
dev-audit raw read.
2026-11-15 14:27:41 -08:00
Neil c90ba953af fix(cloud): WORKER-XREQ-PROMISE sweep: JWKS, the policy chain, the keyring, and a guard
Follow-up to #801: the rest of the Workers' shared in-flight promises.

- Auth (sandbox-control desktop, staging and dev keys; billing) and the egress proxy's OIDC keys used
  jose's createRemoteJWKSet, which shares one in-flight JWKS fetch across every caller: a request
  cancelled mid-fetch hung every token check behind it, and jose refetches after ten minutes. That
  fits the 'Start sandbox' creates stuck before any log 10–15 min after a deploy and cleared by a
  redeploy. settledRemoteJWKSet keeps the key set as a value; a request that needs a fresh one
  fetches it for itself (refetch every 10 min, an unknown kid at most every 30 s).
- applySandboxPolicy no longer chains writes per sandbox in the isolate (a chained promise from a
  cancelled request would hang every later write for that sandbox); the D1 generation orders them.
- The KMS keyring cache keeps the loaded keyring and the last failure as values; no in-flight load
  is stored at all.
- Guard (both Workers): a unit test fails on createRemoteJWKSet, or a module-level Map, variable
  or object holding a Promise, unless the line says why it is safe ('xreq-ok:', e.g. a WebCrypto
  import).
2026-11-15 14:24:44 -08:00
Neil fc5c502bb5 fix(cloud): WORKER-XREQ-PROMISE module caches hold settled values, never another request's in-flight promise
Egress proxy secret routes (custom-base-URL keys, OPENAI_BASE_URL) hung until the client gave up
(dev, proxy mode, 2026-11-15). The Worker was invoked (tail: `canceled`, wall 65 s, CPU 3 ms, no log
line) and was waiting, never reaching its upstream fetch. GitHub routes on the same sandbox answered.

An isolate serves many requests, and a promise one request starts is bound to that request's I/O. If
that request is cancelled while the promise waits on D1, KMS or fetch, the promise never settles, and
every later request awaiting the same cached promise hangs. Several module-level caches shared
in-flight promises: the KMS keyring (forever, per isolate), decrypted provider keys and account
credentials (per minute), the relay's ID token mint, the App's pending-request owners, create timing
and share-link lookups.

They now cache settled values only (settled-cache.ts and the same pattern inline). A request that
misses loads for itself; a cold isolate may run a few duplicate loads.
2026-11-15 14:20:14 -08:00
Neil cd778ffc80 Merge pull request #800 from nwparker/cloud/egress-probe-modes
ci(cloud): egress probe in proxy and firewall modes (streams, uploads, WS, git, latency)
2026-11-15 13:57:15 -08:00
Neil 46a1c8d79f ci(cloud): egress probe in proxy and firewall modes, with streams, uploads, WebSocket, git and latency
Workstream C of the egress migration. `scripts/egress-probe.ts --mode proxy|firewall|both` flips the
same two dev workspaces with the operator egress-mode route and runs every check in each mode. Each
request is a Node fetch (undici, like the agents) with a 30 s limit, so the transform body hang shows
up as a named failure. New coverage:
- a chunked stream, a 1 MB upload and a WebSocket upgrade through the swap;
- a placeholder sent to an unrelated host must arrive unswapped;
- the WebSocket host fails closed;
- the LFS batch API, git fetch, push and delete;
- the App-repo clone through the desktop's create.
Latency is reported per class (median/p90). The terminal wait is now 15 min, and a project whose
clone failed is registered as a folder, so the checks still run.

On dev at 2cc12610 (A2+A3), firewall mode passes 22/22. Proxy-mode findings went to gh-access.
2026-11-15 13:54:30 -08:00
Neil 99ae1d0be7 fix(cloud): EGRESS review F1–F3 before any firewall canary
Security review of A1/A2 (egress-docs):
- F1: the Copilot PAT never reaches the firewall. Eligibility is decided per secret
  (firewallEligibleSecret): no credential the proxy narrows on any host (response filtering,
  paths, per-host paths). Before, its Copilot API host entries had no paths and compiled to
  transforms, putting a user's GitHub PAT in the policy store.
- F2: claims are ordered but landed writes weren't. After a write, if a newer generation was
  claimed meanwhile, the write is redone from current state, so a stale compile (say from before a
  share) is never the last to land.
- F3: a config-hash change fails closed only when it removes access or custody (firewall to proxy,
  a firewall class removed, an account provider turned off, open to limited). Anything else retries
  via policy_stale_at, so a deploy during a provider blip can't sleep every workspace. The stamp now
  carries the deciding inputs; a pre-F3 bare-hash stamp is retried.
2026-11-15 13:32:38 -08:00
Neil 1d93777378 Merge pull request #798 from nwparker/cloud/egress-firewall-a3
feat(cloud): EGRESS firewall mode puts every Codex call on the firewall (no relay)
2026-11-15 13:28:57 -08:00