feat: make a split deployment work end to end by fixing the three defects that made an off-host node impossible to configure (nodeEnvKeys shipped S3_BUCKET and KMS_KEY_ID, which nothing reads, so an AWS-backed node silently used the default bucket and key alias; a joined consumer never received PRIMARY_DB and died at boot; and node.env was rewritten on every join with no file an operator could add to), then removing the need for cloud credentials on a node at all with brokered KMS and blob providers that renderNodeEnv hands out automatically, plus deploy/split-cloud, scripts/aws-bootstrap.sh, two fleet instance checks and the docs

This commit is contained in:
Matthew Meszaros
2026-09-10 13:58:59 +02:00
parent 70ef48a3de
commit 47ba13083e
39 changed files with 2394 additions and 36 deletions
@@ -0,0 +1,56 @@
# A worker on a machine you own, against a control plane elsewhere.
#
# Copy to worker.env and fill in. `warmbly join` writes the equivalent of this
# file for you; it is here so the hand-run path is not guesswork.
#
# What is NOT here is the point: no AWS access key, no bucket credential, no
# KMS key id, and no database DSN. A worker reaches relational data through the
# internal API, and the brokered providers below ask the control plane to
# perform the two operations that would otherwise need a cloud credential.
APP_ENV=prod
# One per machine, generated once: uuidgen. Keep it stable, because moving a
# mailbox to a different worker changes the client address its provider sees
# and buys a sign-in challenge for nothing.
WORKER_ID=
# Optional. Placement prefers a worker near where a mailbox's provider expects
# sign-ins. Blank is fine.
WARMBLY_NODE_REGION=
# --- the control plane ------------------------------------------------------
# One credential, for one instance, revocable from it.
ENCRYPTED_KEYS_PROVIDER=http
ENCRYPTED_KEYS_BACKEND_URL=https://api.example.com
ENCRYPTED_KEYS_WORKER_TOKEN=
INTERNAL_API_TOKEN=
# --- the bus and cache ------------------------------------------------------
# TLS on both: the bus carries recipient addresses and the cache carries each
# organization's decrypted data key.
EVENTBUS_PROVIDER=nats
NATS_URL=tls://<token>@bus.example.com:4222
CODEC_PROVIDER=json
REDIS=rediss://:<password>@bus.example.com:6380
# --- crypto and storage -----------------------------------------------------
# brokered: hold no key material and no bucket credential, and ask the control
# plane for the one privileged operation each. Blob bytes still travel directly
# between this machine and the object store.
KMS_PROVIDER=brokered
BLOB_PROVIDER=brokered
# Seals mailbox SMTP and IMAP passwords and OAuth tokens at rest. The same
# value as the control plane, or nothing this machine reads will decrypt.
CREDENTIALS_ENCRYPTION_KEY=
# --- mailbox OAuth clients --------------------------------------------------
# Needed to refresh Gmail and Microsoft 365 tokens. Plain SMTP and IMAP
# mailboxes need none of it.
BOX_GOOGLE_CLIENT_ID=
BOX_GOOGLE_CLIENT_SECRET=
BOX_OUTLOOK_CLIENT_ID=
BOX_OUTLOOK_CLIENT_SECRET=
# Real mailboxes present real certificates.
MAIL_TLS_INSECURE=false