From fc33acc64d8801b3380ae551ea2579f1bf6d4d1d Mon Sep 17 00:00:00 2001 From: Matthew Meszaros Date: Wed, 10 Jun 2026 17:17:19 +0200 Subject: [PATCH] feat: document the per-organization DEK model Update the encryption sections in the agent notes, internal API auth doc, OAuth setup doc, and architecture notes, and state explicitly that per-user DEKs must not be reintroduced. --- AGENTS.md | 6 +++--- docs/INTERNAL_API_AUTH.md | 8 ++++---- docs/integrations-oauth-setup.md | 2 +- resources/architecture.md | 6 +++--- resources/cicd.md | 2 +- 5 files changed, 12 insertions(+), 12 deletions(-) diff --git a/AGENTS.md b/AGENTS.md index 9056f38e..af7664db 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -183,7 +183,7 @@ Warmbly uses envelope encryption for application secrets and sensitive payloads. High-level flow: - AWS KMS is the root of trust -- each user gets a data encryption key (DEK) +- each organization gets a data encryption key (DEK) - the plaintext DEK is used for application-layer encryption and decryption - the encrypted DEK is stored, not the plaintext DEK - decrypted DEKs are cached for reuse @@ -191,7 +191,7 @@ High-level flow: Current implementation: - KMS generates a 32-byte DEK for AES-256 -- the encrypted DEK blob is base64-encoded and stored via the pluggable `encryptedkeys.Store` (the `postgres` backend writes the `user_encrypted_keys` table; workers use the `http` backend, which proxies to the backend's `/api/v1/internal/dek` endpoint) +- the encrypted DEK blob is base64-encoded and stored via the pluggable `encryptedkeys.Store` (the `postgres` backend writes the `organization_encrypted_keys` table; workers use the `http` backend, which proxies to the backend's `/api/v1/internal/dek` endpoint) - the plaintext DEK is cached in Redis with a TTL - encrypted fields are sealed with AES-GCM and then base64-encoded @@ -209,7 +209,7 @@ Main code paths: Operational guidance: - do not introduce plaintext storage of secrets or message content where the current design expects encrypted values -- DEKs live in the `user_encrypted_keys` Postgres table behind the `encryptedkeys.Store` interface (provider selected by `ENCRYPTED_KEYS_PROVIDER`: `postgres` for backend/consumer, `http` for workers). DynamoDB is no longer used anywhere; do not reintroduce it. Losing a DEK is unrecoverable, so any change to DEK storage needs a migration plan +- DEKs are per-organization and live in the `organization_encrypted_keys` Postgres table behind the `encryptedkeys.Store` interface (provider selected by `ENCRYPTED_KEYS_PROVIDER`: `postgres` for backend/consumer, `http` for workers). DynamoDB is no longer used anywhere; do not reintroduce it. Losing a DEK is unrecoverable, so any change to DEK storage needs a migration plan. Do not reintroduce per-user DEKs: mailboxes, integration tokens, and message content are organization assets, and keying them by user breaks when that user is offboarded - if workers need access to encrypted payloads, prefer passing encrypted material plus access to KMS-backed decryption primitives, or an internal backend API, rather than introducing direct SQL dependencies - be explicit about which fields are encrypted at rest in app code versus stored in infrastructure services like S3 diff --git a/docs/INTERNAL_API_AUTH.md b/docs/INTERNAL_API_AUTH.md index 0c66390b..df74472a 100644 --- a/docs/INTERNAL_API_AUTH.md +++ b/docs/INTERNAL_API_AUTH.md @@ -10,9 +10,9 @@ connection of their own. | Method | Path | Caller | Purpose | |---------|-----------------------------------|-------------------|--------------------------------------| -| GET | `/api/v1/internal/dek/:userID` | worker | Fetch a user's encrypted DEK | -| PUT | `/api/v1/internal/dek/:userID` | worker (rarely) | Store a new encrypted DEK | -| DELETE | `/api/v1/internal/dek/:userID` | admin / cleanup | Delete an encrypted DEK | +| GET | `/api/v1/internal/dek/:orgID` | worker | Fetch an organization encrypted DEK | +| PUT | `/api/v1/internal/dek/:orgID` | worker (rarely) | Store a new encrypted DEK | +| DELETE | `/api/v1/internal/dek/:orgID` | admin / cleanup | Delete an encrypted DEK | | GET | `/api/v1/internal/worker/config` | worker | Fetch runtime config for the worker | | POST | `/api/v1/internal/worker/heartbeat` | worker | Liveness report | @@ -94,7 +94,7 @@ alongside the worker config endpoint: Benefits: ability to revoke a single worker without rotating the whole fleet, audit-log a specific worker's actions, and tier-gate DEK access (e.g., a -free-tier worker can only request DEKs for free-tier users). +free-tier worker can only request DEKs for free-tier organizations). Implementation lives behind Task #9 in the project tracker. diff --git a/docs/integrations-oauth-setup.md b/docs/integrations-oauth-setup.md index fccda6cf..11ef6148 100644 --- a/docs/integrations-oauth-setup.md +++ b/docs/integrations-oauth-setup.md @@ -23,7 +23,7 @@ to light it up; just add the env vars and restart the backend. 5. SPA calls `POST /integrations/oauth/finish` → backend validates+consumes the `state`, exchanges the code for tokens, resolves the connected account, and stores the access/refresh tokens **sealed with the connecting user's - envelope-encryption DEK** (KMS → per-user DEK → AES-GCM, the same path used + envelope-encryption DEK** (KMS → per-organization DEK → AES-GCM, the same path used for mailbox OAuth tokens). Plaintext tokens never touch a database column. Access tokens are refreshed automatically (60s before expiry) using the stored diff --git a/resources/architecture.md b/resources/architecture.md index aa21dcd2..ca2e7bf9 100644 --- a/resources/architecture.md +++ b/resources/architecture.md @@ -30,7 +30,7 @@ Object storage: encrypted email bodies (EMSG format — see [EMSG.md](EMSG.md)) | Store | Purpose | |-------|---------| -| Postgres | Users, organizations, campaigns, mailboxes, workers, credentials, warmup state, per-user encrypted DEKs, message-ID maps, Gmail history IDs | +| Postgres | Users, organizations, campaigns, mailboxes, workers, credentials, warmup state, per-organization encrypted DEKs, message-ID maps, Gmail history IDs | | Redis | Caching (including decrypted DEKs), rate limiting, ephemeral state | | S3 | Email body blobs (EMSG) | | KMS | Root of trust for envelope encryption | @@ -39,7 +39,7 @@ Object storage: encrypted email bodies (EMSG format — see [EMSG.md](EMSG.md)) Warmbly uses envelope encryption end-to-end for sensitive data. -KMS holds the master key. Each user gets a 32-byte data encryption key (DEK), generated by KMS and stored encrypted in the `user_encrypted_keys` Postgres table keyed by user ID (workers reach it over the backend's internal API rather than touching Postgres). The DEK is decrypted only at the moment of use; cached in Redis with a TTL to amortize cost. +KMS holds the master key. Each organization gets a 32-byte data encryption key (DEK), generated by KMS and stored encrypted in the `organization_encrypted_keys` Postgres table keyed by organization ID (workers reach it over the backend's internal API rather than touching Postgres). The DEK is decrypted only at the moment of use; cached in Redis with a TTL to amortize cost. Application-layer secrets are sealed with AES-256-GCM under the DEK and base64-encoded. This applies to: @@ -48,7 +48,7 @@ Application-layer secrets are sealed with AES-256-GCM under the DEK and base64-e - **worker SSH private keys** (since admins drive workers over SSH) - **AWS credential rows and worker-profile secrets** (Kafka SASL passwords, Schema Registry secrets, Redis URLs) used to configure remote workers -Worker-related secrets are encrypted under a platform DEK (user ID = `uuid.Nil`). Same envelope as user secrets — same trust boundary, different identity. See `internal/app/cipher/` and `internal/app/worker_orchestrator/orchestrator.go`. +Worker-related secrets are encrypted under a platform DEK (key ID = `uuid.Nil`). Same envelope as organization secrets — same trust boundary, different identity. See `internal/app/cipher/` and `internal/app/worker_orchestrator/orchestrator.go`. ## Worker model diff --git a/resources/cicd.md b/resources/cicd.md index 68672873..77eba6fa 100644 --- a/resources/cicd.md +++ b/resources/cicd.md @@ -176,7 +176,7 @@ Webhook not firing: ## Security notes -- Worker SSH private keys are encrypted at rest via KMS-wrapped DEK (same envelope as user secrets). +- Worker SSH private keys are encrypted at rest via KMS-wrapped DEK (same envelope as organization secrets). - The release webhook authenticates via HMAC-SHA256, not bearer token, so the secret never appears in logs. - All worker runtime credentials (Kafka SASL, Schema Registry secret, Redis URL, AWS secret access key) are stored encrypted; the dashboard only ever sees "set / not set" booleans for sensitive fields. - Worker AWS keys should be least-privilege: KMS Decrypt, S3 read/write to the configured bucket, plus Kafka SASL. Nothing else. (Workers no longer use DynamoDB — encrypted DEKs and the message-ID map are reached over the backend's internal API.)