Commit Graph

6 Commits

Author SHA1 Message Date
Matthew Meszaros 2430095197 feat: move infrastructure state off dynamodb
Remove DynamoDB-backed storage paths, add Postgres/HTTP repositories for mailbox state maps, wire the internal message-map API, and add provisioning runner/migration plumbing.
2026-06-02 15:54:12 +02:00
Matthew Meszaros b168f4d466 feat: simplify worker installer route 2026-05-30 09:37:26 +00:00
Matthew Meszaros d8a546eca5 feat: add worker enrollment install 2026-05-30 05:10:27 +00:00
Matthew Meszaros 5463e3986e worker: multi-IP support (one systemd unit per Primary IP)
Hetzner CX32 + 16 Primary IPs becomes 16 sending identities with one
install command, without expanding ops complexity.

cmd/worker/main.go: WORKER_ID now resolves via 4-tier precedence:
  1. WORKER_ID env (explicit UUID)
  2. WORKER_BIND_IP env (derive UUIDv5 from the bound IP)
  3. hostname-as-UUID (legacy single-IP VPS)
  4. generated UUID (local dev fallback)

Boot also constructs the chosen Codec + EventBus + EncryptedKeyStore
via the FromEnv factories from earlier commits, so a worker process is
fully configured by its envelope env file plus the runtime config it
pulls from the backend on first boot.

scripts/install-worker.sh gains --ips <ipv4,ipv4,...> which:
  - writes a warmbly-worker@.service systemd template
  - drops a per-instance env file at /etc/warmbly/instances/<dashed-ip>.env
    with WORKER_BIND_IP and WORKER_ID
  - shares one /etc/warmbly/worker.env for the common config
  - --status, --update, --uninstall now multi-IP aware
  - single-IP mode preserved when --ips is absent

5 worker tests pin the UUIDv5 derivation against the installer's
uuidgen --sha1 output so the two never drift.

docs/MULTI_IP_WORKERS.md is the operator runbook with the Hetzner
recipe, OS-level IP attachment, rDNS automation, day-2 ops, and the
25%-of-fleet blast-radius rule.
2026-05-27 14:43:20 +00:00
Matthew Meszaros d25eed3eb6 feat(dev): root docker-compose with profiles, LocalStack, richer seed
Hoist the dev/sim stack to a single docker-compose.yml at the repo root.
Adds profiles (default / sim / seed / tools) so you can opt into heavier
setups, and bundles dependencies that were previously missing:

- LocalStack (KMS + DynamoDB + S3) with a localstack-init one-shot that
  idempotently creates alias/master-key-dev, the UserEncryptedKeys and
  EmailMessageData tables, and the main S3 bucket. Backend and workers
  wait on it via service_completed_successfully.
- stripe-mock for billing flows
- kafka-ui under the tools profile

Three workers with deterministic UUIDv5 hostnames (shared / premium /
dedicated) so assignment, rebalancing, and per-pool routing all have
real targets to exercise.

Richer seed (cmd/seed/main.go) loads 3 orgs across tiers, 6 mailboxes
joined to free/premium warmup pools, a Beta campaign with a 2-step
sequence, and 10 contacts (2 unsubscribed) so suppression behaviour is
visible in the UI. Idempotent — safe to re-run.

Makefile targets:
  make dev    — infra + app + one worker
  make sim    — adds premium + dedicated workers
  make seed   — rich fixtures
  make tools  — kafka-ui at :18090
  make reset  — nuke volumes
2026-05-18 13:08:34 +00:00
Matthew Meszaros 89e5533dd0 feat(worker): add one-command VPS installer with IP-derived identity
scripts/install-worker.sh is a single bash script any Debian/Ubuntu/RHEL/
Fedora/Arch/Alpine VPS can curl|sh to add a worker to the fleet.

Identity is bound to the VPS's public IPv4 via UUIDv5 (URL namespace):

  same IP  → same worker  (reputation persists across reinstalls)
  new IP   → new worker   (fresh identity, no inherited reputation)

The installer detects the public IP via api.ipify.org / ifconfig.me /
checkip.amazonaws.com, derives the deterministic UUID, installs Docker if
missing, writes /etc/warmbly/worker.env (0600) and /etc/warmbly/worker.id,
installs a systemd unit that runs the worker container with --hostname
<uuid>, and starts the service.

Supports --install/--update/--uninstall/--purge/--status, --env-file for
non-interactive config, --ip override, --image override, and a full set
of per-credential flags.

Worker reads its UUID from os.Hostname() at startup, so the systemd
hostname value becomes the worker identity — no separate registration
step needed.
2026-05-18 13:08:16 +00:00