Commit Graph
39 Commits
Author SHA1 Message Date
Matthew Meszaros d68bbcd2ab feat: add a one-command self-host installer at warmbly.com/install.sh with an interactive data-control wizard, give docker-compose.yml image keys and per-store volume variables, add an image-mode updater, move engagement/form/audit retention into instance settings, and add warmblyctl backup/restore 2026-09-04 05:49:54 -07:00
Matthew Meszaros 701dc404a2 Merge remote-tracking branch 'origin/main' into fix/click-without-open-tracking 2026-09-03 05:22:05 -07:00
Matthew Meszaros bb35de0455 feat: address review on self-hosted updates: the updater no longer re-locks its mutex when a job finishes (every job used to deadlock at completion and freeze the status API), the backend caches the updater view so the member version pill, the health checks and the admin poll share one read and an absent updater is reported as not running rather than broken, the bare-metal upgrade builds unprivileged and hands off to a root-owned fixed-path installer that refuses symlinks so sudoers allows one command instead of install/cp/rm/chown/chmod/systemctl/ln, the installer fails when the backend does not come back, the seed image gets the version build args, the dashboard gates the update action on manage_settings and stops polling a backend that answers 404, and revived timestamps are typed as Date 2026-09-03 05:04:30 -07:00
Matthew Meszaros b2ea1f1961 feat: add self-hosted update awareness and one-click updates: every binary is stamped with its version and commit, the backend polls GitHub Releases and a new host-side updater (cmd/updater, compose profile or systemd unit) reports the checkout's commit distance, the admin panel's top bar shows a version pill that turns into an update indicator and opens a dialog with confirmation, live step progress and log, restart tracking and result, the dashboard header shows the same pill to every member of a self-hosted instance with the full update flow for platform admins, Setup and health gains update_available and updater_unreachable checks, warmblyctl status prints the version, make upgrade and scripts/upgrade-bare-metal.sh cover the by-hand paths, and docs gain an Updates page plus configuration, health, deployment and API reference updates 2026-09-03 05:04:30 -07:00
Matthew Meszaros a3264f0d39 feat: address the review on the rebuilt engagement branch: a deferred human click's effects are now durable and once-only (announce_pending on the click row written before the event is marked processed, a claim in finishHumanClick and a minute-by-minute sweep that finishes what a restart cut off, a burst relabel clears the flag), the tracking service keys the source-address token with TRACKING_IP_HASH_KEY so it cannot be enumerated back into an IPv4 address, the migration no longer adds a blocking index to the existing click table, and timeline click and open rows carry task_id 2026-09-03 04:13:39 -07:00
Matthew Meszaros 31dabea0a4 feat: rebuild the click-without-open fix on top of the per-link click attribution from #298: a person's click now also counts as an open and a burst that withdraws the click withdraws the open it implied unless a real open is on record, routing readers ignore machine opens as the docs promised, every open gets its own log row and every open and click records the mail client or proxy, browser, device, OS, country, region and city (migration 000124: origin columns on email_link_clicks plus an email_opens table), the tracking service publishes only the address's network in a nullable client_ip field which the consumer resolves with GeoLite and drops, the contact Activity tab shows each open and the origin of opens and clicks, the campaign overview gains a who-engaged-from-where breakdown exposed as engagement in campaign analytics, live open and click events carry occurred_at, client and location, the leads table explains why an open is not always counted, both logs are pruned daily after a year, email_opens joins the export registry, the consumer reads GEODB_PATH optionally, and the guides and API references are updated (fixes #294) 2026-09-03 03:49:44 -07:00
Matthew Meszaros 290f1f00aa Merge remote-tracking branch 'origin/main' into feature/public-forms
# Conflicts:
#	cmd/backend/main.go
#	docs/content/docs/guides/workspace-export-import.mdx
2026-09-01 09:29:53 -07:00
Matthew Meszaros d6ddf1f170 feat: fix implicit-TLS SMTP on 465 and IMAP STARTTLS on 143 behind a stored per-mailbox security mode that accepts any port, stop worker ID churn orphaning mailbox assignments via flock-claimed persistent worker ids, give the unibox a standard mail-folder sidebar (inbox/sent/drafts/archive/spam/trash) backed by a provider-derived folder column, and expose the AI tool registry over REST for non-MCP function-calling agents (#283) 2026-09-01 03:53:19 -07:00
Matthew Meszaros 60c9e316d9 feat: hosted lead-capture forms end to end: drag-and-drop builder with field settings, design panel, embed/share and submissions tabs in the dashboard, a public TanStack form app (forms/) served by the new standalone forms service (cmd/forms + internal/formserver) on FORMS_DOMAIN with per-form frame-ancestors CSP, honeypot/fill-time/Turnstile/per-IP submit protection and a same-origin JSON API proxying the backend internal API, form submissions creating contacts with categories and campaign enrollment plus realtime, audit, webhook and org-transfer coverage, migration 000114, seed forms, CI jobs, Dockerfile, systemd/nginx/compose manifests and docs 2026-09-01 01:17:51 -07:00
Matthew Meszaros 111a5a4034 feat: read the forwarded client address from exactly one operator-named header (TRACKING_CLIENT_IP_HEADER, default x-forwarded-for with the proxy-appended last entry, cf-connecting-ip only when configured) so a client-supplied CF-Connecting-IP passed through a generic trusted proxy can no longer choose the rate-limit bucket or the stored page-hit location, with tests and the configuration, env example and compose entries 2026-08-29 05:11:22 -07:00
Matthew Meszaros 8daefbe8c4 feat: address the Greptile review on website tracking by believing forwarded-IP headers only from TRACKING_TRUSTED_PROXIES (socket peer otherwise, proxy-appended last X-Forwarded-For entry, applied to pixel, click and page-hit paths), making IdentifyVisitor report whether it claimed the row so a lost identification race re-reads the visitor and splits onto a fresh record instead of attributing the hit to the wrong contact, forgetting the edge dedupe entry when a forward to the backend fails so the retry is counted, and trimming the new Go and Rust comments to the one-line style 2026-08-29 04:16:28 -07:00
Matthew Meszaros 9ab42cfd65 feat: build the browser half of social sign-in, which was never wired: GOOGLE_CLIENT_ID was read at boot and made the login screen render a Google button, but the button opened a popup at /auth/google/login which no route served, and authService.GoogleAuth/AppleAuth had no caller anywhere in the codebase; internal/app/socialauth now runs Google and Apple through the flow generic OIDC already used (one-time state, PKCE, nonce, id_token verified against the provider JWKS, identity keyed on issuer and subject, JIT provisioning, the ban and 2FA gates), the redirect URI defaults to API_PUBLIC_URL/v1/auth/<provider>/callback and is logged at boot because registering the dashboard origin instead is the mistake that produces a valid OAuth client and a dead button, /auth/config advertises only providers the backend can actually complete, the SSO landing page no longer swallows a two_fa_required response, and OIDC_PROVIDER_NAME finally reaches the button it documents 2026-08-28 01:33:08 -07:00
Matthew Meszaros cbf4190f42 feat: make a custom tracking domain verifiable instead of permanently "Pending DNS": the CNAME target is now this install's TRACKING_DOMAIN rather than a hardcoded t.warmbly.com that resolves nowhere, matching is exact on the label boundary (or on shared addresses, so a provider-flattened CNAME stops reading as no record at all) instead of strings.Contains, and every outcome carries the reason plus what DNS actually returned, including when the tracking host the customer is told to point at has no record of its own; a pasted URL is normalized to its host and a malformed one is rejected up front instead of saved and left pending forever; only a VERIFIED mailbox domain is used at send time with the shared host as the fallback and a campaign-feed entry saying why; pixels and click tickets are built from the configured host, and with none configured mail ships untracked rather than carrying links to another deployment's tracking service; adds GET /emails/:id/track and POST /emails/:id/track/verify plus an hourly re-resolution sweep so a record that propagates later starts being used and one that breaks stops routing links; and scopes the tracking-domain write by organization like the read, which also fixes GET /emails/:id passing a user id to an org-scoped query and 404ing for every caller 2026-08-24 09:02:29 -07:00
Matthew Meszaros f0846eb034 feat: stop a freshly connected mailbox being silently excluded from every campaign send, by making an unset mailbox timezone representable as the empty string the campaign scheduler already checks for, since email_accounts.timezone defaulted to 'UTC' while campaigns.timezone defaults to 'Europe/London' and nothing in the OAuth or SMTP onboarding paths ever set either, so a brand new mailbox looked deliberately placed in UTC, was compared against the differing campaign zone and dropped by the hardcoded 8am-8pm business-hours gate whenever the current UTC hour fell outside it, emptying the candidate pool and failing the campaign start, adding a migration that changes the column default and converts existing 'UTC' rows because until now no API field, dashboard control or onboarding path could set that column at all so every such row is the old default rather than a choice, adding the missing Timezone field to UpdateEmail with IANA validation so the setting the sending-behaviour UI already tells people to change is finally reachable and an unloadable zone is rejected instead of being silently coerced to UTC by the scheduler, and replacing the misleading 'no active email accounts found for campaign's email tags' response for a pool that exists but is entirely gated out with a distinct message naming the real cause, via an ErrNoEligibleMailbox that wraps ErrNoEmailAccounts so the three callers that pause a campaign on it are unaffected (#126) (#125) 2026-08-16 07:54:45 +02:00
Matthew Meszaros 734cb5fe08 feat: make self-hosted onboarding survivable by fixing invite_only, which could not onboard anyone (the accept route is JWT-only, so redeeming the invitation that would create your account required already having one, making the self-host default silently identical to fully closed), threading the invitation token through registration so an invited person lands in the inviting organization instead of a stray workspace, gating SSO just-in-time provisioning behind DISABLE_REGISTRATION (it bypassed the gate entirely, so an instance set to true was still open to anyone the IdP would assert) with SSO_AUTO_PROVISION as the opt-out, correcting the OIDC redirect URL that pointed at /api/v1 against a route at /v1 and 404'd every SSO login, scoping the first-launch exemption so it no longer overrides an explicit lockdown, preserving the remaining TTL when restoring a losing setup token so a public endpoint cannot hold the claim window open forever, replacing a generic 403 with typed registration_invite_only, registration_closed, invitation_invalid, setup_token_invalid and setup_already_complete codes that name the next step, logging why no claim link was issued on an already-claimed instance instead of staying silent, adding a warmblyctl operator CLI (status with health checks and a non-zero exit, reissuable setup-link, user create/list/reset-password/grant-admin/revoke-admin/disable-2fa, hash-password) so a locked-out operator no longer needs hand-written psql, adding read-only instance configuration over 104 environment variables with structural secret redaction and fingerprints, 35 health checks, a database-backed settings tier for the three keys no environment variable owns, hiding the signup form when the config already says invite_only rather than failing the whole form with a toast, and documenting first run, accounts and access, configuration, instance health and troubleshooting alongside the root .env.example the README told operators to write but never shipped (#114) 2026-08-16 05:58:11 +02:00
Matthew Meszaros 0ae4db2c41 feat: make self-hosted auth work without a mail relay by rewriting the platform SMTP transport with real AUTH and TLS (it did neither, so SMTP_USERNAME/SMTP_PASSWORD were dead and every documented relay was unreachable), adding MAIL_TRANSPORT=smtp|log|ses with a log transport that prints codes so a fresh install can sign in with no relay, demoting the emailed login code to AUTH_LOGIN_CODE=always|new_device|off (off on self-host, per NIST SP 800-63B and OWASP ASVS), claiming the first owner through a single-use setup link or WARMBLY_BOOTSTRAP_* instead of register-then-psql, deriving every emailed URL from APP_URL rather than a hardcoded app.warmbly.com that leaked live reset tokens to the vendor, fixing the confirm hooks that read path params against paramless routes and broke login, register and reset confirmation in the dashboard everywhere, adding generic OIDC with PKCE, one-time state, verified nonce and (issuer,subject) identity binding, enforcing 2FA on the social paths that skipped it, adding a per-IP limiter and trusted-proxy handling to the unthrottled auth group, refusing boot on the published default secrets, and dropping mailpit from the default stack (#99) 2026-08-14 14:57:09 +02:00
Matthew Meszaros 8bd2c2b57a feat: make self-hosting work end to end and rewrite the guide around what was tested (#97) 2026-08-13 09:47:46 +02:00
Matthew Meszaros a7518a8558 docs: refresh the documentation site, fix inaccurate claims and contact addresses, add SEO primitives (#90)
* feat: rewrite the self-hosting docs against repo ground truth: turn the deployment guide into a full self-host guide (quick start with first-admin bootstrap via make grant-admin, .env secrets with exact key formats, PUBLIC_HOST derivation and HTTPS reverse-proxy vars, provider switches with build-tag caveats, mailbox OAuth, remote worker enrollment via SSH or wmenroll tokens, real CI image tags, upgrades and backups), rewrite the events page around the real NATS/Kafka bus topics and {type,body} envelopes, fix Kafka-era and make-target claims in architecture/local-development/deploy README, add API_PUBLIC_URL and drop the dead LOG_DISCORD_WEBHOOK_URL in env.example, and remove the docker-compose.kafka.yml comment pointing at a file that does not exist

* feat: make the self-hosting docs visual and skimmable by adding a Mermaid MDX component (client-rendered, theme-aware) to the docs site, condensing the self-host guide around a control-plane topology diagram, a worker enrollment sequence diagram, a dashboard screenshot, and symptom/check troubleshooting + optional-subsystem tables, and adding an execution-plane flowchart to the architecture page

* feat: stop the docs root flashing a 'Continue to the Warmbly docs' link before redirecting by navigating with an inline location.replace that runs during HTML parse, and demoting the visible link and meta refresh to no-JS fallbacks inside noscript

* feat: cut docs bulk and duplication by deleting three orphaned API pages that were stale forks of the reference section and were unreachable from the sidebar (porting their unique social sign-in, promo-code, and referral endpoints into api/reference/account-org.mdx as compact tables), condensing the deliverability and warmup guides to roughly half their length around tables instead of prose, replacing prose em dashes across the guides and MCP pages, and adding the required trailing slashes to internal links in 24 files

* feat: condense the sequences guide by about 40 percent, folding the switch-step deciders and branch conditions into tables and cutting restated prose while keeping every rule about threading, instant branches, reply matching, and stop on reply

* feat: condense the automations, unibox, advisor, and expressions guides by roughly 40 percent each, folding trigger lists, action catalogs, sending controls, and advisor checks into tables, adding a trigger-condition-action flow diagram to automations, and cutting restated prose while preserving every threshold, permission boundary, and rule

* feat: condense the mailboxes, campaigns, analytics, and team-roles guides by roughly 45 percent each, replacing prose walks through providers, rotation modes, lead statuses, counting rules, A/B confidence, and the permission matrix with compact tables and collapsing the four-way role grid into one capability table plus a one-line mapping

* feat: condense the AI-steps, security, and contacts-CRM guides by roughly 40 percent, turning sign-in methods, AI step modes, switch deciders, credit and failure behavior, import field mappings, and deal views into tables while keeping every safety boundary and dedupe rule

* feat: condense the meetings, notifications, AI-credits, and AI-assistant guides by roughly 40 percent, merging notification categories and their defaults into one table, collapsing credit costs, spend controls, and plan allowances into tables, and tightening the assistant page around its approval and permission boundaries

* feat: condense the integrations, collaboration, zapier, and make guides by roughly 35 percent, grouping the thirty-row Zapier and Make action lists into eight labelled areas, folding CRM default field mappings and presence indicators into tables, and promoting the destructive-action and unattended-delete warnings into callouts

* fix: correct three factual errors in the development docs: NOTIFICATION_EMAIL_DAILY_CAP=0 means uncapped rather than disabled (overEmailBudget returns false at limit<=0, so documenting it as a kill switch inverted the behavior), and the worker-SSH and warmup-pool migration citations in architecture.mdx pointed at pre-squash filenames that no longer exist or now belong to unrelated migrations, so both now cite the tables in 000001_baseline.up.sql

* feat: add the missing docs SEO primitives: a build-time sitemap.xml covering all 64 pages, a robots.txt that points at it and keeps the llms.mdx and og mirrors out of the index as duplicate content, and per-page canonical plus richer OpenGraph URL/title/description metadata

* fix: use the single real team@warmbly.com address everywhere a human is told to write in, replacing the invented hello/sales/legal/support inboxes across the marketing site, the transactional email footer, and the admin outreach composer default Reply-To (which pointed replies at a mailbox that does not exist), and collapse the contact page's two-inbox framing into one inbox with one published response time
2026-08-05 10:37:27 +02:00
Matthew Meszaros 79efbd4b5b feat: publish the web and admin images to GHCR in the release workflow and serve those production builds from the prod docker-compose instead of the Vite dev servers 2026-07-22 18:40:06 +02:00
Matthew Meszaros aeaa26d2d2 fix: use a valid local kms dev key and warn when sandbox port 8080 is taken 2026-07-21 17:34:59 +02:00
Matthew Meszaros af048b9d75 feat: restore the dovecot imap host behind a sandbox profile so seeded mailboxes really sync 2026-07-20 11:12:36 +02:00
Matthew Meszaros 9590bd63fb feat: add a no-cloud docker compose stack 2026-07-20 09:56:47 +02:00
Matthew Meszaros 7cfd9b26e9 feat: register the tracking-events Avro schema via a one-shot schema-registry-init infra service - the Rust tracking service encodes fetch-only against Schema Registry, so with no registered schema every open/click event failed serialization and was silently dropped in dev 2026-07-11 19:48:05 +02:00
Matthew Meszaros c8ed667aeb feat: add a dovecot IMAP service to the infra stack and accept any-credential SMTP auth on mailpit so local mailboxes can really send and sync (sandbox mail hosting), plus CODEC_PROVIDER/CREDENTIALS_ENCRYPTION_KEY env for backend, consumer, and workers and a host-gateway alias so the cloud-tasks emulator can call a natively run backend 2026-07-11 17:30:03 +02:00
Matthew Meszaros 956edabca3 feat: kafka no longer fails its first boot after an unclean stop - the container now waits for the stale ephemeral /brokers/ids/1 registration to expire in zookeeper before starting (with a 6s session timeout) instead of crashing on NodeExists and relying on the restart policy 2026-07-11 10:13:44 +02:00
Matthew Meszaros 2430095197 feat: move infrastructure state off dynamodb
Remove DynamoDB-backed storage paths, add Postgres/HTTP repositories for mailbox state maps, wire the internal message-map API, and add provisioning runner/migration plumbing.
2026-06-02 15:54:12 +02:00
Matthew Meszaros 559d05a458 feat: use dot-separated event-bus topic names for Kafka compatibility 2026-06-01 12:49:24 +02:00
Matthew Meszaros adedcc337e feat: support remote dev infra
Allow native dev services to target infrastructure running on another host by parameterizing local endpoints with INFRA_HOST and SELF_HOST.

Document the workflow and make Kafka advertise a configurable host for off-box clients.
2026-05-31 08:53:21 +02:00
Matthew Meszaros bf40834557 feat: update unibox experience 2026-05-30 16:37:50 +00:00
Matthew Meszaros 0740a8bbbe feat: fix local migrations and kafka topics 2026-05-30 14:11:47 +00:00
Matthew Meszaros e463df748f fix: add zookeeper healthcheck so kafka waits for sessions to be ready 2026-05-25 04:32:41 +00:00
Matthew Meszaros 6eb3a3bdfd merge: bring origin/main into branch, layer full seed on top of seedRich 2026-05-24 11:54:49 +00:00
Matthew Meszaros e5c0c8a448 chore(dev): standardize ports where they don't conflict
Most container ports go back to their natural defaults — the offsets
that existed weren't justified, they just made URLs harder to remember.
Now standard:

  backend         8080   (was always 8080)
  tracking        3000   (was 13000)
  realtime        4000   (was 14000)
  web             5173   (was 15173 — already changed)
  kafka           9092   (was 19092)
  schema-registry 8081   (was 18081)
  localstack      4566   (was 14566)
  cloud-tasks     8123   (was 18123)
  stripe-mock     12111  (always was)

Kept offset (the defaults conflict too often on real dev machines):

  postgres        15432  (system postgres / sibling project)
  redis           16379  (sibling docker projects with redis)
  mailpit ui      18025  (sibling docker projects with mailpit)
  mailpit smtp    11025  (same)
  kafka-ui        18090  (8080 already used by backend)

Touched: docker-compose.yml, Makefile (test-seed SEED_TEST_DB), READMEs
(root + deploy), local-development.md + deployment-guide.md. Internal
docker-network refs (kafka:29092, mailpit:1025, etc.) unchanged — only
host-port mappings moved. Compose validated, all default-profile
services come up healthy on the new ports.
2026-05-22 14:51:16 +00:00
Matthew Meszaros 33ea340660 fix(dev): set APP_URL on backend so CORS allows the web origin
The backend's CORS allow-list resolution falls back through:
  1. CORS_ALLOW_ORIGINS
  2. APP_URL
  3. origin derived from WEBSOCKET_URL

In compose only WEBSOCKET_URL was set, so allow-list landed at
http://localhost:14000 — the realtime port, not where the browser is
loaded from. The dev fallback that includes localhost:5173 only fires
when the list is empty, not when it's wrong-by-derivation. Result:
every POST to /auth/login etc. failed preflight with 403, which axios
surfaces as a generic "NetworkError" — login appeared to silently fail
after entering the password.

Setting APP_URL=http://localhost:5173 on the backend service makes the
allow-list match where the Vite dev server actually serves from. Sanity
checked with curl -X OPTIONS — preflight now returns 204.
2026-05-22 04:04:42 +00:00
Matthew Meszaros 34ab7ad266 fix(web): correct Request import depth + move web back to port 5173
The new admin API clients (audit, credentials, workers) imported Request
with four '..' segments instead of three. Vite's import-analysis failed
with "Failed to resolve import ../../../../Request" because that path
resolves to api/Request, not client/Request. tsc didn't catch it because
the resolver was permissive enough to keep going, but the runtime is
strict. Matched the existing pattern from roles/getRoles.ts (three dots
for Request, four for models).

Separately: web was on host port 15173, offset from the canonical 5173
to avoid colliding with a locally-running Vite outside Docker. Nobody
actually runs Vite locally in this setup, and the offset makes the URL
non-obvious. Moved back to 5173:5173 and updated VITE_APP_URL plus the
docs.

If a developer one day wants to run a host-side Vite alongside the
container, change the mapping back to "15173:5173" — the offset is the
escape hatch, not the default.
2026-05-22 03:55:34 +00:00
Matthew Meszaros 5a7fa6c71b fix(web): unbreak pnpm install in docker — drop rolldown-vite, fix retry loop
Four interlocking problems were causing the web container to spin in
its retry loop forever:

1. web/package.json aliased vite to "npm:rolldown-vite@7.1.14".
   rolldown-vite is being deprecated (its own warning told us to use
   7.3.1 for migration, or move to vite 8). vitest 4.x has a transitive
   `vite` dep that pnpm tried to resolve against the public registry,
   where vite@7.1.14 doesn't exist as a release (only 7.3.3 and 8.x).
   Result: ERR_PNPM_NO_MATCHING_VERSION on every retry.

2. The "resolutions" block was meant to force the alias on transitive
   deps. resolutions is Yarn syntax; pnpm doesn't read it. So the alias
   wasn't propagating, which is exactly why (1) blew up.

3. pnpm 11 stopped reading the "pnpm" field in package.json. Settings
   moved to pnpm-workspace.yaml. New file added with allowBuilds.esbuild
   set so pnpm doesn't refuse to compile esbuild's native binary at
   install time (it's transitively pulled in by vite + vitest).

4. The docker-compose web service had `until pnpm install; do echo
   "pnpm install retry..."; sleep 3; done` which spins forever on
   permanent dep-resolution errors and buries the actual message under
   thousands of retries. Replaced with fail-fast that prints a hint
   directing the admin to fix package.json and `make restart web`.

Also deleted the stale pnpm-lock.yaml so pnpm regenerates against the
fresh dep tree. Verified vite v7.3.3 boots, esbuild postinstall runs,
and Vite serves on http://localhost:15173 cleanly.

vite.config.ts has no rolldown-specific config, so the move from
rolldown-vite to plain vite is a no-op behaviourally.
2026-05-22 03:52:49 +00:00
Matthew Meszaros f95a12f2a1 Revert "feat(dev): hot reload for Go services in docker-compose"
This reverts commit 3a84e33155.
2026-05-22 03:37:34 +00:00
Matthew Meszaros 3a84e33155 feat(dev): hot reload for Go services in docker-compose
Until now, only the web service hot-reloaded (Vite HMR via the
node:22-alpine container + ./web mount). The Go services (backend,
consumer, worker) used their production multi-stage Dockerfiles, so
every code change meant `docker compose build <svc> && docker compose
up -d <svc>` — ~30s per service.

Switched all Go services to a shared dev image (go.dev.Dockerfile)
that ships:
  - full Go 1.25 toolchain on alpine
  - CGO deps for librdkafka (gcc, musl-dev, librdkafka-dev, pkgconf)
  - air v1.61.7 (the source watcher / hot-recompile tool)

docker-compose mounts the repo at /app and runs `air -c <config>`.
Each Go service has its own air.SERVICE.toml (build target +
exclusions). Named volumes for the Go module cache and build cache
so the first build is slow (~60s for module download) but subsequent
rebuilds after a save are ~2s.

Per-service compose changes:
  - backend, consumer: dockerfile, volumes, and command updated
  - worker-base (the YAML anchor used by all 3 workers): same

Production Dockerfiles in deploy/docker/{backend,consumer,worker}.
Dockerfile are unchanged and still used by release CI. The seed
one-shot in compose continues to use backend.Dockerfile (it's a
short-lived job, no benefit from the dev image).

Rust (tracking) and Elixir (realtime) still build-on-change. They
change far less often; documenting the workaround in
resources/local-development.md for now.
2026-05-22 03:37:05 +00:00
Matthew Meszaros d25eed3eb6 feat(dev): root docker-compose with profiles, LocalStack, richer seed
Hoist the dev/sim stack to a single docker-compose.yml at the repo root.
Adds profiles (default / sim / seed / tools) so you can opt into heavier
setups, and bundles dependencies that were previously missing:

- LocalStack (KMS + DynamoDB + S3) with a localstack-init one-shot that
  idempotently creates alias/master-key-dev, the UserEncryptedKeys and
  EmailMessageData tables, and the main S3 bucket. Backend and workers
  wait on it via service_completed_successfully.
- stripe-mock for billing flows
- kafka-ui under the tools profile

Three workers with deterministic UUIDv5 hostnames (shared / premium /
dedicated) so assignment, rebalancing, and per-pool routing all have
real targets to exercise.

Richer seed (cmd/seed/main.go) loads 3 orgs across tiers, 6 mailboxes
joined to free/premium warmup pools, a Beta campaign with a 2-step
sequence, and 10 contacts (2 unsubscribed) so suppression behaviour is
visible in the UI. Idempotent — safe to re-run.

Makefile targets:
  make dev    — infra + app + one worker
  make sim    — adds premium + dedicated workers
  make seed   — rich fixtures
  make tools  — kafka-ui at :18090
  make reset  — nuke volumes
2026-05-18 13:08:34 +00:00