mirror of
https://github.com/warmbly/warmbly.git
synced 2026-08-26 00:00:41 +00:00
feat: document the per-organization DEK model
Update the encryption sections in the agent notes, internal API auth doc, OAuth setup doc, and architecture notes, and state explicitly that per-user DEKs must not be reintroduced.
This commit is contained in:
@@ -183,7 +183,7 @@ Warmbly uses envelope encryption for application secrets and sensitive payloads.
|
||||
High-level flow:
|
||||
|
||||
- AWS KMS is the root of trust
|
||||
- each user gets a data encryption key (DEK)
|
||||
- each organization gets a data encryption key (DEK)
|
||||
- the plaintext DEK is used for application-layer encryption and decryption
|
||||
- the encrypted DEK is stored, not the plaintext DEK
|
||||
- decrypted DEKs are cached for reuse
|
||||
@@ -191,7 +191,7 @@ High-level flow:
|
||||
Current implementation:
|
||||
|
||||
- KMS generates a 32-byte DEK for AES-256
|
||||
- the encrypted DEK blob is base64-encoded and stored via the pluggable `encryptedkeys.Store` (the `postgres` backend writes the `user_encrypted_keys` table; workers use the `http` backend, which proxies to the backend's `/api/v1/internal/dek` endpoint)
|
||||
- the encrypted DEK blob is base64-encoded and stored via the pluggable `encryptedkeys.Store` (the `postgres` backend writes the `organization_encrypted_keys` table; workers use the `http` backend, which proxies to the backend's `/api/v1/internal/dek` endpoint)
|
||||
- the plaintext DEK is cached in Redis with a TTL
|
||||
- encrypted fields are sealed with AES-GCM and then base64-encoded
|
||||
|
||||
@@ -209,7 +209,7 @@ Main code paths:
|
||||
Operational guidance:
|
||||
|
||||
- do not introduce plaintext storage of secrets or message content where the current design expects encrypted values
|
||||
- DEKs live in the `user_encrypted_keys` Postgres table behind the `encryptedkeys.Store` interface (provider selected by `ENCRYPTED_KEYS_PROVIDER`: `postgres` for backend/consumer, `http` for workers). DynamoDB is no longer used anywhere; do not reintroduce it. Losing a DEK is unrecoverable, so any change to DEK storage needs a migration plan
|
||||
- DEKs are per-organization and live in the `organization_encrypted_keys` Postgres table behind the `encryptedkeys.Store` interface (provider selected by `ENCRYPTED_KEYS_PROVIDER`: `postgres` for backend/consumer, `http` for workers). DynamoDB is no longer used anywhere; do not reintroduce it. Losing a DEK is unrecoverable, so any change to DEK storage needs a migration plan. Do not reintroduce per-user DEKs: mailboxes, integration tokens, and message content are organization assets, and keying them by user breaks when that user is offboarded
|
||||
- if workers need access to encrypted payloads, prefer passing encrypted material plus access to KMS-backed decryption primitives, or an internal backend API, rather than introducing direct SQL dependencies
|
||||
- be explicit about which fields are encrypted at rest in app code versus stored in infrastructure services like S3
|
||||
|
||||
|
||||
@@ -10,9 +10,9 @@ connection of their own.
|
||||
|
||||
| Method | Path | Caller | Purpose |
|
||||
|---------|-----------------------------------|-------------------|--------------------------------------|
|
||||
| GET | `/api/v1/internal/dek/:userID` | worker | Fetch a user's encrypted DEK |
|
||||
| PUT | `/api/v1/internal/dek/:userID` | worker (rarely) | Store a new encrypted DEK |
|
||||
| DELETE | `/api/v1/internal/dek/:userID` | admin / cleanup | Delete an encrypted DEK |
|
||||
| GET | `/api/v1/internal/dek/:orgID` | worker | Fetch an organization encrypted DEK |
|
||||
| PUT | `/api/v1/internal/dek/:orgID` | worker (rarely) | Store a new encrypted DEK |
|
||||
| DELETE | `/api/v1/internal/dek/:orgID` | admin / cleanup | Delete an encrypted DEK |
|
||||
| GET | `/api/v1/internal/worker/config` | worker | Fetch runtime config for the worker |
|
||||
| POST | `/api/v1/internal/worker/heartbeat` | worker | Liveness report |
|
||||
|
||||
@@ -94,7 +94,7 @@ alongside the worker config endpoint:
|
||||
|
||||
Benefits: ability to revoke a single worker without rotating the whole fleet,
|
||||
audit-log a specific worker's actions, and tier-gate DEK access (e.g., a
|
||||
free-tier worker can only request DEKs for free-tier users).
|
||||
free-tier worker can only request DEKs for free-tier organizations).
|
||||
|
||||
Implementation lives behind Task #9 in the project tracker.
|
||||
|
||||
|
||||
@@ -23,7 +23,7 @@ to light it up; just add the env vars and restart the backend.
|
||||
5. SPA calls `POST /integrations/oauth/finish` → backend validates+consumes the
|
||||
`state`, exchanges the code for tokens, resolves the connected account, and
|
||||
stores the access/refresh tokens **sealed with the connecting user's
|
||||
envelope-encryption DEK** (KMS → per-user DEK → AES-GCM, the same path used
|
||||
envelope-encryption DEK** (KMS → per-organization DEK → AES-GCM, the same path used
|
||||
for mailbox OAuth tokens). Plaintext tokens never touch a database column.
|
||||
|
||||
Access tokens are refreshed automatically (60s before expiry) using the stored
|
||||
|
||||
@@ -30,7 +30,7 @@ Object storage: encrypted email bodies (EMSG format — see [EMSG.md](EMSG.md))
|
||||
|
||||
| Store | Purpose |
|
||||
|-------|---------|
|
||||
| Postgres | Users, organizations, campaigns, mailboxes, workers, credentials, warmup state, per-user encrypted DEKs, message-ID maps, Gmail history IDs |
|
||||
| Postgres | Users, organizations, campaigns, mailboxes, workers, credentials, warmup state, per-organization encrypted DEKs, message-ID maps, Gmail history IDs |
|
||||
| Redis | Caching (including decrypted DEKs), rate limiting, ephemeral state |
|
||||
| S3 | Email body blobs (EMSG) |
|
||||
| KMS | Root of trust for envelope encryption |
|
||||
@@ -39,7 +39,7 @@ Object storage: encrypted email bodies (EMSG format — see [EMSG.md](EMSG.md))
|
||||
|
||||
Warmbly uses envelope encryption end-to-end for sensitive data.
|
||||
|
||||
KMS holds the master key. Each user gets a 32-byte data encryption key (DEK), generated by KMS and stored encrypted in the `user_encrypted_keys` Postgres table keyed by user ID (workers reach it over the backend's internal API rather than touching Postgres). The DEK is decrypted only at the moment of use; cached in Redis with a TTL to amortize cost.
|
||||
KMS holds the master key. Each organization gets a 32-byte data encryption key (DEK), generated by KMS and stored encrypted in the `organization_encrypted_keys` Postgres table keyed by organization ID (workers reach it over the backend's internal API rather than touching Postgres). The DEK is decrypted only at the moment of use; cached in Redis with a TTL to amortize cost.
|
||||
|
||||
Application-layer secrets are sealed with AES-256-GCM under the DEK and base64-encoded. This applies to:
|
||||
|
||||
@@ -48,7 +48,7 @@ Application-layer secrets are sealed with AES-256-GCM under the DEK and base64-e
|
||||
- **worker SSH private keys** (since admins drive workers over SSH)
|
||||
- **AWS credential rows and worker-profile secrets** (Kafka SASL passwords, Schema Registry secrets, Redis URLs) used to configure remote workers
|
||||
|
||||
Worker-related secrets are encrypted under a platform DEK (user ID = `uuid.Nil`). Same envelope as user secrets — same trust boundary, different identity. See `internal/app/cipher/` and `internal/app/worker_orchestrator/orchestrator.go`.
|
||||
Worker-related secrets are encrypted under a platform DEK (key ID = `uuid.Nil`). Same envelope as organization secrets — same trust boundary, different identity. See `internal/app/cipher/` and `internal/app/worker_orchestrator/orchestrator.go`.
|
||||
|
||||
## Worker model
|
||||
|
||||
|
||||
+1
-1
@@ -176,7 +176,7 @@ Webhook not firing:
|
||||
|
||||
## Security notes
|
||||
|
||||
- Worker SSH private keys are encrypted at rest via KMS-wrapped DEK (same envelope as user secrets).
|
||||
- Worker SSH private keys are encrypted at rest via KMS-wrapped DEK (same envelope as organization secrets).
|
||||
- The release webhook authenticates via HMAC-SHA256, not bearer token, so the secret never appears in logs.
|
||||
- All worker runtime credentials (Kafka SASL, Schema Registry secret, Redis URL, AWS secret access key) are stored encrypted; the dashboard only ever sees "set / not set" booleans for sensitive fields.
|
||||
- Worker AWS keys should be least-privilege: KMS Decrypt, S3 read/write to the configured bucket, plus Kafka SASL. Nothing else. (Workers no longer use DynamoDB — encrypted DEKs and the message-ID map are reached over the backend's internal API.)
|
||||
|
||||
Reference in New Issue
Block a user