Matthew Meszaros
ca66b25d8d
feat: register the release's bus schemas from the booting backend and hold the fleet on its current release until the backend runs the new one and the schema registry accepts them, check every published schema against a recorded snapshot in CI (make schemas) so a change the registry would refuse fails before merge, pin a FORWARD or FULL registry subject to BACKWARD on refusal, bound advisor narration so the run still lands, store unlabelled 8-bit mail with its invalid UTF-8 replaced, treat a pending or credential-less passkey ceremony as a cancellation, and drop link-scanner rejections from site error tracking
2026-09-29 19:20:07 +02:00
Matthew Meszaros
e58921484d
feat: rebuild mailbox import around column mapping and automatic host and sign-in detection (CSV, XLSX, pasted lists, saved mappings, retryable rows with fixes, migrations 000205-000206), connect whole Google Workspace domains and Microsoft 365 organizations through a proved administrator grant, import from inbox vendors (InboxKit, Zapmail, Mailforge, Infraforge, Maildoso, Cheap Inboxes, ScaledMail) with vendor-managed forwarding and DNS, add a sending domains page with per-domain tracking and verified root redirects, unify Add account into one Google and one Microsoft entry with per-method choices, mark per-mailbox Google sign-in as retiring with in-place moves to the admin grant or an app password, allow the loopback security mode in the credential columns (migration 000207), read semicolon-separated CSVs, and add a mock vendor API to the sandbox
2026-09-23 08:41:01 -07:00
Matthew Meszaros
28b3dc9f05
feat: move the CASA evidence pack out of this repository to CASA_EVIDENCE_DIR and make the generator refuse any destination inside the tree, because a pack that maps every control to its file and lists the advisories still open with their reachability conditions is a reconnaissance document for anyone attacking a self-hosted instance that has not updated yet
2026-09-19 13:08:33 +02:00
Matthew Meszaros
e668a2a36b
feat: complete the ADA CASA v2.1.1 AL1 control set across authentication, sessions, access control, cryptography, input validation and configuration, adding a breached-password denylist and per-account login throttling, enforced multi-factor authentication on the admin panel, step-up confirmation before an action that mints a lasting credential, purpose-scoped session tokens, single-use TOTP steps, tenant verification on every cross-referenced identifier, security headers on every surface, encrypted webhook signing secrets, per-organization idempotency, PKCE and a minimal two-scope Gmail consent on the mailbox OAuth flow, bounded spreadsheet and archive decoding, a patched Go toolchain with govulncheck in CI, and the evidence pack under compliance/casa
2026-09-19 08:18:35 +02:00
Matthew Meszaros
e3092f2322
feat: address the review by releasing the topic lock before the broker call so one slow creation cannot stall every publish, subscribe and close for the full admin timeout, refusing to open an admin connection once the bus is closed so a publish racing past the closed check cannot resurrect a client nothing will shut, building rather than vetting the tagged Kafka backend in CI because vet does not link and a CGO backend fails at link time, and correcting the Confluent tier wording to say auto topic creation is configurable only on Dedicated
2026-09-12 06:54:34 +02:00
Matthew Meszaros
6504d9958a
feat: create Kafka topics from the bus that uses them, because a worker's command topic is named after the node id issued at join time so the set is not knowable in advance, and the broker's auto-creation is off by default on Confluent Cloud and not configurable below Standard, which left a worker subscribed to a topic that did not exist receiving nothing and reporting no error, and compile the tagged Kafka build in CI so a backend nothing else builds cannot rot unnoticed
2026-09-12 06:02:49 +02:00
Matthew Meszaros
bbe9d9055a
feat: let web and admin be served from a static host by teaching each app's own entrypoint to render config.js wherever WARMBLY_CONFIG_OUT points, so one definition of the runtime key set serves both the container that renders it at start and a build:pages script that renders it into dist, ship a _redirects in each so a deep link stops 404ing without nginx try_files, and add scripts/check-pages-build.sh to make lint because a malformed config.js reads fine in a diff and leaves the app blank at runtime
2026-09-10 18:03:00 +02:00
Matthew Meszaros
9e689e6d91
feat: fix the three defects that stopped the split-deployment bus bundle from starting at all (NATS given store_dir twice on the command line and in nats.conf, which it refuses rather than reconciling; Redis unable to read its own private key because the image's entrypoint drops from root with gosu and discards the added group; and a healthcheck on localhost when the monitor binds IPv4 loopback, leaving the service unhealthy while serving fine), add bus/setup.sh so standing the box up is one idempotent command that verifies with real clients before printing the control plane's URLs, and add scripts/check-split-cloud.sh to make lint so a bundle that cannot start fails CI
2026-09-10 17:22:14 +02:00
Matthew Meszaros
92b811773e
feat: fail the build when a Dockerfile COPY names a path that is not in the repository, because build-push.yml runs only on push to main and nothing on a pull request builds an image, so a COPY of a file the same change deleted is green everywhere until it breaks main
2026-09-09 09:03:54 -07:00
Matthew Meszaros
6947aca9b3
feat: make the join-script check assert on what the script renders via a new --print-unit mode, after the previous version compared a heredoc copied into the checker and stayed green when the systemd command-substitution bug was put back, and make docker_mounts pure so the unit can be rendered without creating directories
2026-09-09 06:37:56 -07:00
Matthew Meszaros
29bf9b4320
feat: cover the join script with make join-check, wired into make lint, since nothing tested the highest-consequence non-Go file in the repo and three defects reached the branch through it; and fix the four the review found in the last round, matching the fs provider alias, creating blob parents under a 0022 umask rather than the 0700 one write_config leaves set, warning instead of silently mounting a blob root the node cannot write, and validating the path right after enrolment rather than halfway through the install
2026-09-09 06:28:30 -07:00
Matthew Meszaros
435dbb522f
feat: replace the worker tier/type/risk-pool/egress categories with a scored placement model and make the fleet pull-based, so a machine joins with one command, workers and consumers share one node registry with usage and liveness, nodes self-update to the version the control plane resolves, and the Hetzner provisioning, worker profiles and SSH orchestrator are removed
2026-09-09 04:54:01 -07:00
Matthew Meszaros
719a81866c
feat: gate the release on every published image being pullable with no credentials, because GHCR creates each package private and does not inherit the repository's visibility, so the authenticated imagetools inspect in create-release passed for four releases while ghcr.io/warmbly/warmbly/* returned unauthorized to everyone outside the org and no curl | sh self-host install could pull a byte ( #371 ); adds scripts/check-images-public.sh as the anonymous pull test, makes it the gate before create-release and the source of the digests in images.json, warns from build-push on main where a new service image first appears, and teaches install.sh to report a registry refusal as one instead of as a missing tag
2026-09-07 08:51:57 -07:00
Matthew Meszaros
4e94d2b47b
feat: install the CLI in one command from warmbly.com/cli.sh or cli.ps1, with checksum-verified archives, a Homebrew formula and Scoop manifest generated alongside them, a ghcr cli image, and a make cli-check that installs from a local mirror and proves a tampered checksum stops the install
2026-09-04 20:14:38 -07:00
Matthew Meszaros
d68bbcd2ab
feat: add a one-command self-host installer at warmbly.com/install.sh with an interactive data-control wizard, give docker-compose.yml image keys and per-store volume variables, add an image-mode updater, move engagement/form/audit retention into instance settings, and add warmblyctl backup/restore
2026-09-04 05:49:54 -07:00
Matthew Meszaros
701dc404a2
Merge remote-tracking branch 'origin/main' into fix/click-without-open-tracking
2026-09-03 05:22:05 -07:00
Matthew Meszaros
b2ea1f1961
feat: add self-hosted update awareness and one-click updates: every binary is stamped with its version and commit, the backend polls GitHub Releases and a new host-side updater (cmd/updater, compose profile or systemd unit) reports the checkout's commit distance, the admin panel's top bar shows a version pill that turns into an update indicator and opens a dialog with confirmation, live step progress and log, restart tracking and result, the dashboard header shows the same pill to every member of a self-hosted instance with the full update flow for platform admins, Setup and health gains update_available and updater_unreachable checks, warmblyctl status prints the version, make upgrade and scripts/upgrade-bare-metal.sh cover the by-hand paths, and docs gain an Updates page plus configuration, health, deployment and API reference updates
2026-09-03 05:04:30 -07:00
Matthew Meszaros
31dabea0a4
feat: rebuild the click-without-open fix on top of the per-link click attribution from #298 : a person's click now also counts as an open and a burst that withdraws the click withdraws the open it implied unless a real open is on record, routing readers ignore machine opens as the docs promised, every open gets its own log row and every open and click records the mail client or proxy, browser, device, OS, country, region and city (migration 000124: origin columns on email_link_clicks plus an email_opens table), the tracking service publishes only the address's network in a nullable client_ip field which the consumer resolves with GeoLite and drops, the contact Activity tab shows each open and the origin of opens and clicks, the campaign overview gains a who-engaged-from-where breakdown exposed as engagement in campaign analytics, live open and click events carry occurred_at, client and location, the leads table explains why an open is not always counted, both logs are pruned daily after a year, email_opens joins the export registry, the consumer reads GEODB_PATH optionally, and the guides and API references are updated ( fixes #294 )
2026-09-03 03:49:44 -07:00
Matthew Meszaros
05d74dbe9f
feat: forms v2, a full redesign of the hosted form and its builder: eight one-click themes, card/wide/split layouts with a brand cover panel, classic paged and Typeform-style focus modes split on a new page_break block, logo/cover/background uploads with size, fit and veil controls, an optional header bar that can span the page or sit with the form, a real HSV colour picker and font preview, per-contact personalized links that prefill and attribute without email verification, a render-token gate so the form JSON cannot be scraped without loading the page, funnel analytics with per-page drop-off and identified visitors, a leads-style forms list and responses table, and an organization custom forms domain verified by CNAME with hourly re-checks
2026-09-01 09:27:28 -07:00
Matthew Meszaros
60c9e316d9
feat: hosted lead-capture forms end to end: drag-and-drop builder with field settings, design panel, embed/share and submissions tabs in the dashboard, a public TanStack form app (forms/) served by the new standalone forms service (cmd/forms + internal/formserver) on FORMS_DOMAIN with per-form frame-ancestors CSP, honeypot/fill-time/Turnstile/per-IP submit protection and a same-origin JSON API proxying the backend internal API, form submissions creating contacts with categories and campaign enrollment plus realtime, audit, webhook and org-transfer coverage, migration 000114, seed forms, CI jobs, Dockerfile, systemd/nginx/compose manifests and docs
2026-09-01 01:17:51 -07:00
Matthew Meszaros
1247c712f3
feat: add make poollink-dev, a two-instance dev environment for the Warmbly Cloud link (a prod-like cloud with DEPLOYMENT_MODE=cloud, Stripe gates on with placeholder keys, Mailpit email verification and the Sunrise Labs pool seeded, next to a fresh unclaimed self-hosted instance whose WARMBLY_CLOUD_URL points at it), with status/setup-link/down/reset subcommands and a local-development docs section
2026-08-29 07:14:04 -07:00
Matthew Meszaros
cbf4190f42
feat: make a custom tracking domain verifiable instead of permanently "Pending DNS": the CNAME target is now this install's TRACKING_DOMAIN rather than a hardcoded t.warmbly.com that resolves nowhere, matching is exact on the label boundary (or on shared addresses, so a provider-flattened CNAME stops reading as no record at all) instead of strings.Contains, and every outcome carries the reason plus what DNS actually returned, including when the tracking host the customer is told to point at has no record of its own; a pasted URL is normalized to its host and a malformed one is rejected up front instead of saved and left pending forever; only a VERIFIED mailbox domain is used at send time with the shared host as the fallback and a campaign-feed entry saying why; pixels and click tickets are built from the configured host, and with none configured mail ships untracked rather than carrying links to another deployment's tracking service; adds GET /emails/:id/track and POST /emails/:id/track/verify plus an hourly re-resolution sweep so a record that propagates later starts being used and one that breaks stops routing links; and scopes the tracking-domain write by organization like the read, which also fixes GET /emails/:id passing a user id to an org-scoped query and 404ing for every caller
2026-08-24 09:02:29 -07:00
Matthew Meszaros
7c2d08f204
feat: renumber the campaign_send_outcome migration to 000091 so it stops colliding with 000089_email_auth_enforcement, which made golang-migrate refuse to init its source driver and left the backend restart-looping at boot on main, and guard the class of bug with scripts/check-migrations.sh (duplicate versions, gaps, unpaired up/down, unparseable filenames) wired into make check-migrations, make lint, a dedicated Migrations CI job and a TestEmbeddedMigrationsLoad that boots the real iofs source over the embedded FS
2026-08-23 21:26:14 -07:00
Matthew Meszaros
734cb5fe08
feat: make self-hosted onboarding survivable by fixing invite_only, which could not onboard anyone (the accept route is JWT-only, so redeeming the invitation that would create your account required already having one, making the self-host default silently identical to fully closed), threading the invitation token through registration so an invited person lands in the inviting organization instead of a stray workspace, gating SSO just-in-time provisioning behind DISABLE_REGISTRATION (it bypassed the gate entirely, so an instance set to true was still open to anyone the IdP would assert) with SSO_AUTO_PROVISION as the opt-out, correcting the OIDC redirect URL that pointed at /api/v1 against a route at /v1 and 404'd every SSO login, scoping the first-launch exemption so it no longer overrides an explicit lockdown, preserving the remaining TTL when restoring a losing setup token so a public endpoint cannot hold the claim window open forever, replacing a generic 403 with typed registration_invite_only, registration_closed, invitation_invalid, setup_token_invalid and setup_already_complete codes that name the next step, logging why no claim link was issued on an already-claimed instance instead of staying silent, adding a warmblyctl operator CLI (status with health checks and a non-zero exit, reissuable setup-link, user create/list/reset-password/grant-admin/revoke-admin/disable-2fa, hash-password) so a locked-out operator no longer needs hand-written psql, adding read-only instance configuration over 104 environment variables with structural secret redaction and fingerprints, 35 health checks, a database-backed settings tier for the three keys no environment variable owns, hiding the signup form when the config already says invite_only rather than failing the whole form with a toast, and documenting first run, accounts and access, configuration, instance health and troubleshooting alongside the root .env.example the README told operators to write but never shipped ( #114 )
2026-08-16 05:58:11 +02:00
Matthew Meszaros
0ae4db2c41
feat: make self-hosted auth work without a mail relay by rewriting the platform SMTP transport with real AUTH and TLS (it did neither, so SMTP_USERNAME/SMTP_PASSWORD were dead and every documented relay was unreachable), adding MAIL_TRANSPORT=smtp|log|ses with a log transport that prints codes so a fresh install can sign in with no relay, demoting the emailed login code to AUTH_LOGIN_CODE=always|new_device|off (off on self-host, per NIST SP 800-63B and OWASP ASVS), claiming the first owner through a single-use setup link or WARMBLY_BOOTSTRAP_* instead of register-then-psql, deriving every emailed URL from APP_URL rather than a hardcoded app.warmbly.com that leaked live reset tokens to the vendor, fixing the confirm hooks that read path params against paramless routes and broke login, register and reset confirmation in the dashboard everywhere, adding generic OIDC with PKCE, one-time state, verified nonce and (issuer,subject) identity binding, enforcing 2FA on the social paths that skipped it, adding a per-IP limiter and trusted-proxy handling to the unthrottled auth group, refusing boot on the published default secrets, and dropping mailpit from the default stack ( #99 )
2026-08-14 14:57:09 +02:00
Matthew Meszaros
8bd2c2b57a
feat: make self-hosting work end to end and rewrite the guide around what was tested ( #97 )
2026-08-13 09:47:46 +02:00
Matthew Meszaros
aeaa26d2d2
fix: use a valid local kms dev key and warn when sandbox port 8080 is taken
2026-07-21 17:34:59 +02:00
Matthew Meszaros
638c40c629
feat: make make sandbox a one-command demo stack and drop the stale kafka/localstack/worker-premium targets
2026-07-20 11:12:37 +02:00
Matthew Meszaros
49ffaedc9f
feat: run make dev on nats and make make up the one-command self-host
2026-07-20 09:56:47 +02:00
Matthew Meszaros
fa3672464a
feat: add one-command make dev target - brings up docker infra and polls real readiness (pg_isready plus kafka/schema-registry/localstack init one-shot exit codes, failing loudly with a make logs hint), applies migrations, loads seed fixtures (SEED=false to skip), installs web deps on first run, then runs backend + consumer + both shared workers + dashboard in one terminal with Ctrl-C teardown
2026-07-16 09:51:20 +02:00
Matthew Meszaros
647a1965f2
feat: replace the AI_LOCAL boolean with an AI_PROVIDER/AI_KEY/AI_MODEL/AI_BASE_URL/AI_FREE passthrough on the backend, consumer, and run targets so make run picks the provider too
2026-07-16 06:14:19 +02:00
Matthew Meszaros
0b1037843d
feat: add a make AI_LOCAL=1 toggle that runs the backend and consumer against a local Ollama model (OLLAMA_HOST/OLLAMA_MODEL overridable), off by default
2026-07-15 18:52:07 +02:00
Matthew Meszaros
7cfd9b26e9
feat: register the tracking-events Avro schema via a one-shot schema-registry-init infra service - the Rust tracking service encodes fetch-only against Schema Registry, so with no registered schema every open/click event failed serialization and was silently dropped in dev
2026-07-11 19:48:05 +02:00
Matthew Meszaros
f8cf7e930e
feat: native dev targets for the sandbox - make sandbox / sandbox-seed, a second native worker (worker-premium, required because paid orgs place strictly onto premium-tier workers) joined into make run, CODEC_PROVIDER=json + CREDENTIALS_ENCRYPTION_KEY + MAIL_TLS_INSECURE in the dev env, dovecot in INFRA_SVCS, and SELF_HOST defaulting to host.docker.internal so the dockerized cloud-tasks emulator can reach the host-run backend
2026-07-11 17:31:42 +02:00
Matthew Meszaros
3473d09bcc
feat: fix review findings in the roles redesign — GetMembers role_id column (members endpoint 500), TransferOwnership role_id hygiene, accept-time role re-resolution, race-free in-use delete guard covering invitations, assignment anti-escalation with self-role-change block, canManage-gated members UI, colored RolePills, fresh currentOrganization on refetch, dev JWT_SECRET wiring for make realtime, docs corrections
2026-06-11 10:45:21 +02:00
Matthew Meszaros
8732805934
feat: replace signed click redirects with server-side link tickets (tracked_links store, internal resolver API, opaque /c/<id> URLs, layered anti-probe caches with miss budget and circuit breaker) removing TRACKING_LINK_SECRET entirely
2026-06-11 09:30:21 +02:00
Matthew Meszaros
2c5e8b2cbd
feat: make TRACKING_LINK_SECRET a required boot-time secret on backend and tracking service with no unsigned mode and no rotation grace, so rotating the key revokes old links immediately
2026-06-11 09:00:04 +02:00
Matthew Meszaros
3236ffc459
feat: expand product docs and automation references
...
Restructure the docs navigation into product-focused pages, add automation expression reference content, and refresh related template and personalization surfaces.
2026-06-09 09:07:56 +02:00
Matthew Meszaros
6109601e87
feat: pin lint command to installed tool
...
Restore the golangci-lint config format expected by the pinned v1 CI tool and make the lint target invoke the installed binary directly.
2026-06-02 16:50:42 +02:00
Matthew Meszaros
3943b8a2e7
feat: update docs for postgres-backed secrets
...
Refresh developer docs, deployment notes, and public-site copy to describe the Postgres-backed encrypted key and message-map model after DynamoDB removal.
2026-06-02 15:55:01 +02:00
Matthew Meszaros
f2913249b4
feat: implement the two-step email-code admin login flow
2026-06-01 13:58:06 +02:00
Matthew Meszaros
76c2cb7d49
feat: run make seed natively so it no longer needs a docker backend
2026-06-01 13:19:30 +02:00
Matthew Meszaros
83eae84d49
feat: merge main into warmup branch
...
Resolve the main-branch conflict in realtime event publishing by keeping both warmup account-health and audit-created events. Renumber the warmup migrations after the current main migration tail.
2026-06-01 11:36:04 +02:00
Matthew Meszaros
a04d7450bc
feat: guard warmup reconciliation by access
...
Make the warmup task reconciler re-check account state and org warmup access before scheduling replacement tasks. Remove pool membership for accounts whose org can no longer use warmup to avoid repeated no-access task churn.
2026-06-01 11:19:44 +02:00
Matthew Meszaros
8b28b86b7f
feat: support exposed local dev hosts
...
Add PUBLIC_HOST wiring for local web, admin, site, realtime, and backend CORS so the native dev stack can be reached over Tailscale or LAN without changing localhost defaults.
2026-06-01 05:51:02 +02:00
Matthew Meszaros
adedcc337e
feat: support remote dev infra
...
Allow native dev services to target infrastructure running on another host by parameterizing local endpoints with INFRA_HOST and SELF_HOST.
Document the workflow and make Kafka advertise a configurable host for off-box clients.
2026-05-31 08:53:21 +02:00
Matthew Meszaros
0740a8bbbe
feat: fix local migrations and kafka topics
2026-05-30 14:11:47 +00:00
Matthew Meszaros
a227938778
feat: seed unibox trial fixtures
2026-05-30 14:06:36 +00:00
Matthew Meszaros
d08c984c42
feat: snapshot current dashboard changes
2026-05-30 13:56:27 +00:00
Matthew Meszaros
7f830b976f
feat: sync org session, harden campaign loading, add mailbox bulk-remove
2026-05-30 03:42:39 +00:00
Matthew Meszaros