docs(skills): record cloud rollout hardening

This commit is contained in:
Jinwoo-H
2026-08-11 13:30:28 -04:00
parent 6ba6107dba
commit 46e412d40e
2 changed files with 37 additions and 5 deletions
@@ -6,14 +6,14 @@ Last updated: 2026-08-11.
Implementation baselines captured by this checklist update:
- Orca implementation: `81bc526d81` on `skills-share` (pushed; no PR).
- Orca Cloud: `ddbd4e2` on `skills-share-cloud` (pushed; no PR).
- Orca implementation: `6ba6107dba` on `skills-share` (pushed; no PR).
- Orca Cloud: `cdf22de` on `skills-share-cloud` (pushed; no PR).
Validated so far:
- Local Node and web typechecks, changed-code quality gates, 94 skill-domain files with 770 tests
passed and 3 skipped, 132 Orca Cloud API tests, the full Cloud monorepo test/typecheck/lint/build
gates, and Terraform validation.
passed and 3 skipped, 133 Orca Cloud API tests with one opt-in integration skip, the full Cloud
monorepo test/typecheck/lint/build gates, and isolated Terraform formatting and validation.
- Final Orca validation passed repository-wide lint, all Node/CLI/web typechecks, the production
desktop/native build, the Node 18 Relay bundle contract, 78 focused skill files with 557 tests
passed and 10 platform skips, and the mixed-version wire suite. The full repository run passed
@@ -52,10 +52,18 @@ Validated so far:
response, and validated structured SSH failure data without arbitrary error-data disclosure.
- Isolated staging bucket, IAM, secret container, log metrics, dashboard, and alerts in
`onorca-cloud-staging`.
- PostgreSQL migration startup is serialized by a transaction-scoped advisory lock. Transactional
DDL rollback and eight concurrent startup callers passed against ephemeral PostgreSQL 16 and 17,
with exactly one recorded schema version.
- The immutable API deploy path now stamps every artifact and skill literal variable plus exactly
one approved skill-database secret, and rejects secret, Cloud SQL, service-account, scaling,
CPU, memory, volume, mount, probe, or unexpected-environment drift before traffic promotion.
Both staging and production workflows explicitly carry all four skill controls as `false`.
Rollout gate: staging Cloud SQL is shared with Auth/Relay and intentionally stopped. Do not create
the skills database/user/secret version or deploy Cloud Run routes until the supported staging SQL
power workflow deliberately wakes it and the reviewed plan still shows no unrelated changes.
`gcloud` currently requires `gcloud auth login jinwoo@stably.ai` before that fresh plan can run.
Source plan: [Agent skill sharing and installation plan](./agent-skill-sharing-installation-plan.md).
@@ -479,6 +487,12 @@ does not mean the surrounding phase is complete.
database, and error data.
- [x] Define criteria for splitting finalization into a worker service if it harms existing API
traffic.
- [x] Make immutable API candidates atomically replace the complete Terraform-owned literal
environment and the exact `ORCA_SKILLS_DATABASE_URL` Secret Manager reference.
- [x] Reject serving or candidate drift in the runtime service account, Cloud SQL attachment,
scaling, CPU, memory, volumes, mounts, ports, and probes before moving traffic.
- [x] Keep all four skill controls explicitly disabled in staging and production deploy workflows
until their rollout gates are intentionally changed.
### Terraform review and apply
@@ -496,6 +510,10 @@ does not mean the surrounding phase is complete.
### PostgreSQL migrations
- [x] Serialize concurrent Cloud Run startup migrations with a transaction-scoped PostgreSQL
advisory lock before migration-table inspection.
- [x] Prove failed DDL records no schema version and eight concurrent callers converge on one
version against real PostgreSQL 16 and 17.
- [x] Add `skill_packages` with owner organization, slug/name, creator, timestamps, and deletion
state.
- [x] Add immutable `skill_package_versions` with package and archive identities, GCS key and
+15 -1
View File
@@ -39,7 +39,7 @@ that larger package-manager surface.
## Current execution status
Orca implementation through `81bc526d81` and Orca Cloud implementation through `ddbd4e2` are
Orca implementation through `6ba6107dba` and Orca Cloud implementation through `cdf22de` are
pushed on feature branches without an Orca pull request. Local, Windows, WSL, paired-runtime, and
Docker-backed SSH validation are substantially complete; the implementation checklist records the
exact evidence and remaining physical-host and failure-recovery gates.
@@ -57,6 +57,13 @@ stopped. Review a fresh Terraform plan after refreshing `gcloud` authentication,
only through the supported Relay power workflow if the reviewed skill changes require the
database. Keep skill routes disabled until migrations and staging smoke tests pass.
The API release workflow now deploys an immutable no-traffic candidate with the complete artifact
and skill environment plus exactly one approved skill-database secret. It verifies the inherited
Terraform runtime identity, Cloud SQL attachment, scaling, resources, volumes, mounts, ports, and
probes before promotion, and rolls back to a re-smoked prior revision on failure. All four skill
controls remain explicitly disabled. Database migration startup uses a transaction-scoped advisory
lock; transactional rollback and eight concurrent callers pass against PostgreSQL 16 and 17.
## Research baseline
The upstream assessment used `vercel-labs/skills` commit
@@ -459,6 +466,13 @@ ORCA_SKILLS_DATABASE_URL=<Secret Manager reference>
Use the existing auth base URL and CORS-origin configuration. Do not add a second authentication
implementation for skills.
Every API image rollout must stamp the complete Terraform-owned literal environment and exact
`ORCA_SKILLS_DATABASE_URL` secret reference in one no-traffic candidate. Before smoke or promotion,
compare the candidate with both the reviewed release values and the serving Terraform runtime
shape. Reject any unexpected secret or environment entry, changed service account, Cloud SQL
attachment, scaling bound, resource limit, volume, mount, port, or probe. This keeps ordinary image
deploys from clearing the skill database secret or silently detaching Cloud SQL.
The API streams upload validation with fixed buffers under its existing 512 MiB memory limit. Load
test the finalize lane before choosing its fixed concurrency. Keep the current Cloud Run instance
scaling bounds initially, then adjust only from observed CPU, latency, database connection, and