starred/warmbly
1
0
Fork 0
mirror of https://github.com/warmbly/warmbly.git synced 2026-08-18 16:01:18 +00:00
Code Issues Packages Projects Releases Wiki Activity
Files
fix/108-orphaned-worker-assignment
warmbly/deploy
T
History
Matthew Meszaros ee61c19faf feat: create /data/blobs in the backend, consumer and worker images owned by the non-root user they run as, because Docker seeds a fresh named volume from the image and the path did not exist there, so it created the mount point root:root while every service runs as uid 1000 and the very first attempt to store an outbound body failed with 'mkdir /data/blobs/emails: permission denied' and the task went to the dead-letter queue, which is invisible until a real send is attempted since the stack is otherwise healthy, and documenting the one-time chown that existing installs still need because Docker only applies image ownership when it first creates the volume (#124)
2026-08-16 07:46:22 +02:00
..
config
feat: make self-hosted auth work without a mail relay by rewriting the platform SMTP transport with real AUTH and TLS (it did neither, so SMTP_USERNAME/SMTP_PASSWORD were dead and every documented relay was unreachable), adding MAIL_TRANSPORT=smtp|log|ses with a log transport that prints codes so a fresh install can sign in with no relay, demoting the emailed login code to AUTH_LOGIN_CODE=always|new_device|off (off on self-host, per NIST SP 800-63B and OWASP ASVS), claiming the first owner through a single-use setup link or WARMBLY_BOOTSTRAP_* instead of register-then-psql, deriving every emailed URL from APP_URL rather than a hardcoded app.warmbly.com that leaked live reset tokens to the vendor, fixing the confirm hooks that read path params against paramless routes and broke login, register and reset confirmation in the dashboard everywhere, adding generic OIDC with PKCE, one-time state, verified nonce and (issuer,subject) identity binding, enforcing 2FA on the social paths that skipped it, adding a per-IP limiter and trusted-proxy handling to the unthrottled auth group, refusing boot on the published default secrets, and dropping mailpit from the default stack (#99)
2026-08-14 14:57:09 +02:00
docker
feat: create /data/blobs in the backend, consumer and worker images owned by the non-root user they run as, because Docker seeds a fresh named volume from the image and the path did not exist there, so it created the mount point root:root while every service runs as uid 1000 and the very first attempt to store an outbound body failed with 'mkdir /data/blobs/emails: permission denied' and the task went to the dead-letter queue, which is invisible until a real send is attempted since the stack is otherwise healthy, and documenting the one-time chown that existing installs still need because Docker only applies image ownership when it first creates the volume (#124)
2026-08-16 07:46:22 +02:00
README.md
docs: refresh the documentation site, fix inaccurate claims and contact addresses, add SEO primitives (#90)
2026-08-05 10:37:27 +02:00

README.md

Warmbly Deployment

Two distinct planes, deployed differently.

Plane Services How
Control backend, consumer, tracking, realtime, web Container hosting in one region (Railway in production). Stable region-pinning so KMS/S3 calls stay local.
Execution worker One process per VPS, anywhere with a public IPv4. Managed from the admin dashboard over SSH.

Directory layout

deploy/
├── docker/
│   ├── backend.Dockerfile          # also builds the seed + migrate binaries
│   ├── consumer.Dockerfile
│   ├── worker.Dockerfile
│   ├── realtime.Dockerfile
│   ├── go-dev.Dockerfile           # hot-reload dev images (make app)
│   ├── rust-dev.Dockerfile
│   ├── elixir-dev.Dockerfile
│   └── air.toml
└── config/
    └── env.example

The tracking Dockerfile lives at tracking/Dockerfile, and the frontends build from web/Dockerfile and admin/Dockerfile (nginx static builds with runtime config injection). The self-host compose is docker-compose.yml at the repo root.

Building images

docker build -f deploy/docker/backend.Dockerfile  -t warmbly/backend  .
docker build -f deploy/docker/consumer.Dockerfile -t warmbly/consumer .
docker build -f deploy/docker/worker.Dockerfile   -t warmbly/worker   .
docker build -f deploy/docker/realtime.Dockerfile -t warmbly/realtime .
docker build -f tracking/Dockerfile               -t warmbly/tracking tracking/
docker build -f web/Dockerfile                    -t warmbly/web      web/
docker build -f admin/Dockerfile                  -t warmbly/admin    admin/

The default builds have no Kafka/Avro support; add --build-arg GO_TAGS=kafka (Go images) or --build-arg CARGO_FEATURES=kafka (tracking) to opt in.

GitHub Actions publishes these to GHCR automatically. See the self-hosting guide.

Local development

make dev    # one-command native dev stack (infra + migrations + seed + app)
make infra  # postgres, redis, nats, mailpit (leave running, shared across worktrees)
make app    # backend, consumer, worker, tracking, realtime, web, admin (hot reload, in Docker)
make seed   # rich fixtures
make reset  # nuke volumes

Full reference: local development.

Deploying the control plane

The Dockerfiles in deploy/docker/ are the deployment unit. Production runs on Railway. Other valid targets: Fly.io, ECS Fargate, single-VPS systemd. Migrations run automatically on backend boot.

Configuration is env-driven — see deploy/config/env.example for the full env reference, or the self-hosting guide for a step-by-step.

Realtime transport

Backend, consumer, and the Elixir realtime service all pick their event transport from one flag, PUBSUB_ENABLED, so they cannot disagree:

  • PUBSUB_ENABLED=false (default): events bridge over Redis (REDIS_URL). No GCP needed. This is the local-dev and simple self-host path.
  • PUBSUB_ENABLED=true: events flow through Google Pub/Sub. Also set GCP_PROJECT_ID and GOOGLE_APPLICATION_CREDENTIALS_JSON on every service. The backend and consumer auto-provision the realtime topics and their <topic>-sub pull subscriptions on boot (idempotent), so there is no manual gcloud step. The service account needs roles/pubsub.editor.

Set the flag the same on all three services. A publisher on Pub/Sub with a subscriber on Redis silently drops every realtime event.

Worker deployment

Workers run on per-VPS machines so cold-mail traffic spreads across many IPs. Worker identity is a deterministic UUIDv5 derived from the VPS's public IPv4 — same IP, same worker.

Add a worker from the admin dashboard:

  1. Provision a VPS, note its public IP + root user
  2. Admin → Workers → Add Worker
  3. Copy the generated enrollment command
  4. Run it on the VPS as root

The installer is served by the backend at /worker-install.sh. It exchanges the one-time token for worker config, writes /etc/warmbly/worker.env, configures systemd, enables a daily randomized self-update timer, and starts the worker container. The worker then heartbeats back to the backend and marks itself installed.

The older SSH-managed path is still supported: paste the generated SSH public key into the VPS's ~/.ssh/authorized_keys, then click Test and Install. From then on, lifecycle operations (restart, update, system updates, reboot, rotate keys, logs, uninstall) can happen from the dashboard.

Manual install on the VPS is also supported:

curl -fsSL https://api.example.com/worker-install.sh | sudo bash -s -- \
  --enroll wmenroll_... \
  --api-base https://api.example.com

# or fully manual, passing a prepared env file:
sudo bash scripts/install-worker.sh \
  --image ghcr.io/<owner>/warmbly/worker:prod \
  --env-file worker.env

scripts/install-worker.sh --help lists every flag (--ips for multi-IP machines, --update, --uninstall, --purge, --status, --no-auto-update, plus the legacy Kafka/AWS prompts).

Why per-VPS instead of Kubernetes DaemonSet

Cold-mail reputation lives at the IP level. K8s nodes typically NAT pods through a small set of egress IPs, so a per-node DaemonSet does not deliver IP diversity. Workers don't depend on Postgres, so cluster-level service discovery isn't needed. Spreading across VPS providers and regions is the only thing that actually moves the deliverability needle.

Worker env reference

Workers in production should be assigned to a worker profile in the dashboard. The profile bundles all of these:

Env var Source Notes
APP_ENV profile
EVENTBUS_PROVIDER / NATS_URL profile nats on the default stack
CODEC_PROVIDER profile json on the default stack
REDIS profile full URL with embedded password; encrypted at rest
ENCRYPTED_KEYS_BACKEND_URL profile the backend's public/internal URL
ENCRYPTED_KEYS_WORKER_TOKEN profile must equal the backend's INTERNAL_API_TOKEN
BOX_GOOGLE_* / BOX_OUTLOOK_* profile mailbox OAuth clients; needed for token refresh
KAFKA_* / SCHEMA_REGISTRY_* profile Kafka path only; secrets encrypted at rest
AWS_REGION / AWS_ACCESS_KEY_ID / AWS_SECRET_ACCESS_KEY profile (via AWS credentials row) only when using AWS KMS/S3; secret encrypted at rest
WORKER_TIER (worker row) shared or dedicated

The worker does not open a Postgres connection. Do not add one.

Auto-update

Each worker profile picks a release channel (pinned / stable / dev) and an auto_update toggle. When a GitHub release fires the webhook, the backend resolves the channel and (if auto_update=true) rolls every assigned worker. See the self-hosting guide.

Health checks

curl http://localhost:8080/health    # backend
curl http://localhost:3000/health   # tracking
curl http://localhost:4000/health   # realtime

Documentation

Reference in New Issue View Git Blame Copy Permalink