feat: document the per-organization DEK model

Update the encryption sections in the agent notes, internal API auth
doc, OAuth setup doc, and architecture notes, and state explicitly that
per-user DEKs must not be reintroduced.
This commit is contained in:
Matthew Meszaros
2026-06-10 17:17:19 +02:00
parent 03134317ca
commit fc33acc64d
5 changed files with 12 additions and 12 deletions
+3 -3
View File
@@ -183,7 +183,7 @@ Warmbly uses envelope encryption for application secrets and sensitive payloads.
High-level flow:
- AWS KMS is the root of trust
- each user gets a data encryption key (DEK)
- each organization gets a data encryption key (DEK)
- the plaintext DEK is used for application-layer encryption and decryption
- the encrypted DEK is stored, not the plaintext DEK
- decrypted DEKs are cached for reuse
@@ -191,7 +191,7 @@ High-level flow:
Current implementation:
- KMS generates a 32-byte DEK for AES-256
- the encrypted DEK blob is base64-encoded and stored via the pluggable `encryptedkeys.Store` (the `postgres` backend writes the `user_encrypted_keys` table; workers use the `http` backend, which proxies to the backend's `/api/v1/internal/dek` endpoint)
- the encrypted DEK blob is base64-encoded and stored via the pluggable `encryptedkeys.Store` (the `postgres` backend writes the `organization_encrypted_keys` table; workers use the `http` backend, which proxies to the backend's `/api/v1/internal/dek` endpoint)
- the plaintext DEK is cached in Redis with a TTL
- encrypted fields are sealed with AES-GCM and then base64-encoded
@@ -209,7 +209,7 @@ Main code paths:
Operational guidance:
- do not introduce plaintext storage of secrets or message content where the current design expects encrypted values
- DEKs live in the `user_encrypted_keys` Postgres table behind the `encryptedkeys.Store` interface (provider selected by `ENCRYPTED_KEYS_PROVIDER`: `postgres` for backend/consumer, `http` for workers). DynamoDB is no longer used anywhere; do not reintroduce it. Losing a DEK is unrecoverable, so any change to DEK storage needs a migration plan
- DEKs are per-organization and live in the `organization_encrypted_keys` Postgres table behind the `encryptedkeys.Store` interface (provider selected by `ENCRYPTED_KEYS_PROVIDER`: `postgres` for backend/consumer, `http` for workers). DynamoDB is no longer used anywhere; do not reintroduce it. Losing a DEK is unrecoverable, so any change to DEK storage needs a migration plan. Do not reintroduce per-user DEKs: mailboxes, integration tokens, and message content are organization assets, and keying them by user breaks when that user is offboarded
- if workers need access to encrypted payloads, prefer passing encrypted material plus access to KMS-backed decryption primitives, or an internal backend API, rather than introducing direct SQL dependencies
- be explicit about which fields are encrypted at rest in app code versus stored in infrastructure services like S3
+4 -4
View File
@@ -10,9 +10,9 @@ connection of their own.
| Method | Path | Caller | Purpose |
|---------|-----------------------------------|-------------------|--------------------------------------|
| GET | `/api/v1/internal/dek/:userID` | worker | Fetch a user's encrypted DEK |
| PUT | `/api/v1/internal/dek/:userID` | worker (rarely) | Store a new encrypted DEK |
| DELETE | `/api/v1/internal/dek/:userID` | admin / cleanup | Delete an encrypted DEK |
| GET | `/api/v1/internal/dek/:orgID` | worker | Fetch an organization encrypted DEK |
| PUT | `/api/v1/internal/dek/:orgID` | worker (rarely) | Store a new encrypted DEK |
| DELETE | `/api/v1/internal/dek/:orgID` | admin / cleanup | Delete an encrypted DEK |
| GET | `/api/v1/internal/worker/config` | worker | Fetch runtime config for the worker |
| POST | `/api/v1/internal/worker/heartbeat` | worker | Liveness report |
@@ -94,7 +94,7 @@ alongside the worker config endpoint:
Benefits: ability to revoke a single worker without rotating the whole fleet,
audit-log a specific worker's actions, and tier-gate DEK access (e.g., a
free-tier worker can only request DEKs for free-tier users).
free-tier worker can only request DEKs for free-tier organizations).
Implementation lives behind Task #9 in the project tracker.
+1 -1
View File
@@ -23,7 +23,7 @@ to light it up; just add the env vars and restart the backend.
5. SPA calls `POST /integrations/oauth/finish` → backend validates+consumes the
`state`, exchanges the code for tokens, resolves the connected account, and
stores the access/refresh tokens **sealed with the connecting user's
envelope-encryption DEK** (KMS → per-user DEK → AES-GCM, the same path used
envelope-encryption DEK** (KMS → per-organization DEK → AES-GCM, the same path used
for mailbox OAuth tokens). Plaintext tokens never touch a database column.
Access tokens are refreshed automatically (60s before expiry) using the stored
+3 -3
View File
@@ -30,7 +30,7 @@ Object storage: encrypted email bodies (EMSG format — see [EMSG.md](EMSG.md))
| Store | Purpose |
|-------|---------|
| Postgres | Users, organizations, campaigns, mailboxes, workers, credentials, warmup state, per-user encrypted DEKs, message-ID maps, Gmail history IDs |
| Postgres | Users, organizations, campaigns, mailboxes, workers, credentials, warmup state, per-organization encrypted DEKs, message-ID maps, Gmail history IDs |
| Redis | Caching (including decrypted DEKs), rate limiting, ephemeral state |
| S3 | Email body blobs (EMSG) |
| KMS | Root of trust for envelope encryption |
@@ -39,7 +39,7 @@ Object storage: encrypted email bodies (EMSG format — see [EMSG.md](EMSG.md))
Warmbly uses envelope encryption end-to-end for sensitive data.
KMS holds the master key. Each user gets a 32-byte data encryption key (DEK), generated by KMS and stored encrypted in the `user_encrypted_keys` Postgres table keyed by user ID (workers reach it over the backend's internal API rather than touching Postgres). The DEK is decrypted only at the moment of use; cached in Redis with a TTL to amortize cost.
KMS holds the master key. Each organization gets a 32-byte data encryption key (DEK), generated by KMS and stored encrypted in the `organization_encrypted_keys` Postgres table keyed by organization ID (workers reach it over the backend's internal API rather than touching Postgres). The DEK is decrypted only at the moment of use; cached in Redis with a TTL to amortize cost.
Application-layer secrets are sealed with AES-256-GCM under the DEK and base64-encoded. This applies to:
@@ -48,7 +48,7 @@ Application-layer secrets are sealed with AES-256-GCM under the DEK and base64-e
- **worker SSH private keys** (since admins drive workers over SSH)
- **AWS credential rows and worker-profile secrets** (Kafka SASL passwords, Schema Registry secrets, Redis URLs) used to configure remote workers
Worker-related secrets are encrypted under a platform DEK (user ID = `uuid.Nil`). Same envelope as user secrets — same trust boundary, different identity. See `internal/app/cipher/` and `internal/app/worker_orchestrator/orchestrator.go`.
Worker-related secrets are encrypted under a platform DEK (key ID = `uuid.Nil`). Same envelope as organization secrets — same trust boundary, different identity. See `internal/app/cipher/` and `internal/app/worker_orchestrator/orchestrator.go`.
## Worker model
+1 -1
View File
@@ -176,7 +176,7 @@ Webhook not firing:
## Security notes
- Worker SSH private keys are encrypted at rest via KMS-wrapped DEK (same envelope as user secrets).
- Worker SSH private keys are encrypted at rest via KMS-wrapped DEK (same envelope as organization secrets).
- The release webhook authenticates via HMAC-SHA256, not bearer token, so the secret never appears in logs.
- All worker runtime credentials (Kafka SASL, Schema Registry secret, Redis URL, AWS secret access key) are stored encrypted; the dashboard only ever sees "set / not set" booleans for sensitive fields.
- Worker AWS keys should be least-privilege: KMS Decrypt, S3 read/write to the configured bucket, plus Kafka SASL. Nothing else. (Workers no longer use DynamoDB — encrypted DEKs and the message-ID map are reached over the backend's internal API.)