Matthew Meszaros
0eb4e30fec
feat: record a Microsoft 365 grant only when the consenting sign-in holds the Global Administrator or Privileged Role Administrator role, validate a tracking host before any vendor DNS write, clear Graph delta cursors when an Outlook mailbox moves onto a grant, keep a switched-off mailbox off when it moves unless its sign-in stopped it, treat pool-link mailboxes as managed, park a delegated mailbox without a grant as inactive, honour MAILVENDOR_SANDBOX_URL only when APP_ENV is dev, drop a vendor's cached domains with its key, retry a failed import redirect setup, keep address:password pastes whose password has a comma, and move the vendor import components to import/vendors so the Go vendor ignore rule no longer drops them
2026-09-23 09:03:25 -07:00
Matthew Meszaros
e58921484d
feat: rebuild mailbox import around column mapping and automatic host and sign-in detection (CSV, XLSX, pasted lists, saved mappings, retryable rows with fixes, migrations 000205-000206), connect whole Google Workspace domains and Microsoft 365 organizations through a proved administrator grant, import from inbox vendors (InboxKit, Zapmail, Mailforge, Infraforge, Maildoso, Cheap Inboxes, ScaledMail) with vendor-managed forwarding and DNS, add a sending domains page with per-domain tracking and verified root redirects, unify Add account into one Google and one Microsoft entry with per-method choices, mark per-mailbox Google sign-in as retiring with in-place moves to the admin grant or an app password, allow the loopback security mode in the credential columns (migration 000207), read semicolon-separated CSVs, and add a mock vendor API to the sandbox
2026-09-23 08:41:01 -07:00
Matthew Meszaros
addb956ad6
feat: shared TypeSafe client under internal/pkg/typesafe with inbox tagging phases 2 and 3 (hold, stop, task, suppress behind workspace switches, reversible ones on by default), labels seeded at workspace creation and by the follow-up sweep, premade inbox views (hot leads, needs a reply, follow up, declined, automated), typed reply classification and a reply_intent branch condition, an inbox agent draft gate, Advisor copy judgment with a cached editor re-check, warmup content lint, bounce cause classification that keeps a blocked address sendable, and per-form submission triage
2026-09-19 23:33:37 -07:00
Matthew Meszaros
e8f14bb2fd
feat: address the review on the Gmail app-password connect by reading BOX_GOOGLE_OAUTH_CONNECT through config.GoogleOAuthConnect in the instance-settings table so a yes/on value cannot display true against a gate that parses it as false, dropping the coming-soon line from the walkthrough banner on deployments where Google sign-in is actually available, naming the 2-Step Verification app-password control an administrator still has rather than the Less secure apps page Google removed, saying the OAuth client re-authorizes existing mailboxes as well as refreshing them, and marking the marketing send trace as the Google sign-in path
2026-09-18 22:18:56 -07:00
Matthew Meszaros
ee46cb49e8
feat: route new Gmail and Google Workspace mailboxes through a guided three-step app-password connect over smtp.gmail.com and imap.gmail.com instead of Google sign-in, behind BOX_GOOGLE_OAUTH_CONNECT (off by default) and announced to clients as gmail_oauth_connect on /auth/config, refusing a new gmail OAuth start with 403 mailbox_gmail_oauth_disabled in both the direct and Warmbly Cloud broker paths while leaving mailboxes already connected that way sending, syncing and re-authorizable
2026-09-18 22:06:09 -07:00
Matthew Meszaros
dcf26a2361
feat: make automatic inbox tagging atomic live and reviewable in production
2026-09-17 04:57:24 -07:00
Matthew Meszaros
f106c8541d
feat: make the bus envelopes Avro-encodable ( #535 )
...
* feat: make both bus envelopes Avro-encodable by deriving each one's schema from a declared registry of body types, with a union branch per body and our own struct walk that skips unexported fields and honours avro:"-" before descending, so the schema describes exactly what encoding/json already puts on the wire, and narrow the two sync cursors on the wire DTOs to int64 because Avro has no unsigned 64-bit type
* feat: stop the instance health check, the config registry and the docs all claiming Avro cannot serialize a worker envelope, which stopped being true once the envelopes carried a declared union, and check the one thing that is still a real misconfiguration instead: avro selected with no SCHEMA_REGISTRY_URL to resolve against
* feat: emit a reference the second time a record appears in an envelope schema instead of defining it again, because Avro names a record once and a document that defines warmbly.events.Token three times is rejected outright, and keep the Schema Registry round-trip as a skip-by-default test since only a registry judges the document rather than the objects it was built from
* feat: carry uint64 as Avro fixed(8) rather than long, which lets the sync cursors keep their unsigned type instead of being narrowed, name every event field after its json tag so the schema and the JSON wire agree, and populate every field in the round-trip test because zero values are why a uint64 mapped to long passed in the first place
* feat: frame Avro in Confluent's wire format and encode through hamba's default API instead of going through avrov2, whose private avro.API holds a type resolver avro.Register cannot reach, so a union body failed there with unable to resolve type while encoding cleanly against the same schema, and keep the registry round-trip as a skip-by-default test
2026-09-15 10:50:30 -07:00
Matthew Meszaros
4784ee7d39
feat: fetch the MaxMind databases instead of requiring a mounted file ( #529 )
...
* feat: let the backend, consumer and tracking service fetch their own MaxMind databases from GEODB_URL and TRACKING_SCANNER_ASN_DB_URL, reading the archive shape from the content so a permalink tar.gz, a gzipped mmdb and a bare mmdb all work, never replacing a file already at the path, opening the bytes before installing them so a licence-key error page cannot become the database forever, skipping the AppleDouble sidecars a macOS tar writes ahead of the real file, and treating both URLs as secrets because the permalink carries the licence key
* feat: drop the trailing blank line cargo fmt --check rejects at the end of tracking/src/asndb.rs
* feat: stream the downloaded ASN archive instead of decompressing it whole, sizing each buffer from the gzip footer and the tar header so the member is allocated exactly once, which drops the peak of unwrapping a permalink tar.gz from 38 MB to 11.9 MB, essentially the database itself
* feat: stop the MaxMind licence key reaching the logs through net/http's and reqwest's own error text, which both print the URL they were given and so defeated the redaction beside them, drop userinfo as well as the query when redacting, refuse plain http for a URL carrying a credential and refuse an https-to-http redirect, and apply the size cap to the decoded database rather than the compressed transfer so a gzip bomb cannot fill the disk
* feat: strip basic-auth userinfo as well as the query when the tracking service redacts its database URL, parsing it rather than cutting at the first question mark so where a credential sits is the URL library's problem and not a guess
2026-09-15 03:06:34 -07:00
Matthew Meszaros
0a1ed6f04e
feat: resolve scanner ASNs from a GeoLite2-ASN database so the catalogue's asn: entries match without a Cloudflare transform rule, ship the Proofpoint, Mimecast and Cisco ASNs enabled behind a new probable certainty that widens the consumer's machine window instead of deciding the verdict, make the tracking event dedupe claim one coalesced operation, and report an ASN database that opened cleanly but resolves nothing ( #440 )
2026-09-15 01:48:45 -07:00
Matthew Meszaros
ced741e352
feat: make PostHog the default error tracker across every runtime while keeping Sentry fully supported alongside or instead of it, by turning internal/observability/errs into a two-sink fan-out with a local-log fallback, adding $exception capture to the Go services, the Rust tracking service, the Elixir realtime service and the dashboard, admin and form apps, reporting gin panics with their route, request id, workspace and user, attaching that identity plus a route and failed-request trail to browser exceptions, and wiring POSTHOG_ERROR_TRACKING, the node join env, compose, source-map upload and the docs to match
2026-09-10 19:11:32 +02:00
Matthew Meszaros
64dfaa8c60
Merge remote-tracking branch 'origin/main' into feat/split-cloud-hosting
2026-09-10 14:20:22 +02:00
Matthew Meszaros
47ba13083e
feat: make a split deployment work end to end by fixing the three defects that made an off-host node impossible to configure (nodeEnvKeys shipped S3_BUCKET and KMS_KEY_ID, which nothing reads, so an AWS-backed node silently used the default bucket and key alias; a joined consumer never received PRIMARY_DB and died at boot; and node.env was rewritten on every join with no file an operator could add to), then removing the need for cloud credentials on a node at all with brokered KMS and blob providers that renderNodeEnv hands out automatically, plus deploy/split-cloud, scripts/aws-bootstrap.sh, two fleet instance checks and the docs
2026-09-10 13:58:59 +02:00
Matthew Meszaros
6684dafe3a
feat: classify opens and clicks that arrive from a known mail-filtering network as automated, so Microsoft 365 Defender's delivery-time pixel fetches and Safe Links URL detonations stop counting as engagement (issue #410 ), with a shipped scanner catalogue, TRACKING_SCANNER_* overrides and optional ASN matching from a trusted edge header
2026-09-10 03:38:50 -07:00
Matthew Meszaros
704fecdcc0
feat: settle on WARMBLY_NODE_REGION as the one region variable for every role, because the join script wrote it while the worker still read WORKER_REGION, so a joined worker reported no region and the placement geography term stayed dead for exactly the machines that had been given one
2026-09-09 05:04:55 -07:00
Matthew Meszaros
435dbb522f
feat: replace the worker tier/type/risk-pool/egress categories with a scored placement model and make the fleet pull-based, so a machine joins with one command, workers and consumers share one node registry with usage and liveness, nodes self-update to the version the control plane resolves, and the Hetzner provisioning, worker profiles and SSH orchestrator are removed
2026-09-09 04:54:01 -07:00
Matthew Meszaros
3596a9770f
feat: make mailboxes truly unlimited under a fair-use allowance of one mailbox per daily send the plan includes (15,000 on Business), drop the 200 hard cap and the 5-per-day connect throttle, open a request-more dialog with inline limit requests and plan upgrade when the allowance is reached, add bulk CSV mailbox import with live progress and a downloadable failed-rows file, reserve attachment storage atomically under a per-org lock so concurrent uploads cannot pass the quota ( #326 ), report storage and the allowance on the billing usage meters, and honor approved daily-send overrides in the sender
2026-09-04 21:00:26 -07:00
Matthew Meszaros
d68bbcd2ab
feat: add a one-command self-host installer at warmbly.com/install.sh with an interactive data-control wizard, give docker-compose.yml image keys and per-store volume variables, add an image-mode updater, move engagement/form/audit retention into instance settings, and add warmblyctl backup/restore
2026-09-04 05:49:54 -07:00
Matthew Meszaros
b2ea1f1961
feat: add self-hosted update awareness and one-click updates: every binary is stamped with its version and commit, the backend polls GitHub Releases and a new host-side updater (cmd/updater, compose profile or systemd unit) reports the checkout's commit distance, the admin panel's top bar shows a version pill that turns into an update indicator and opens a dialog with confirmation, live step progress and log, restart tracking and result, the dashboard header shows the same pill to every member of a self-hosted instance with the full update flow for platform admins, Setup and health gains update_available and updater_unreachable checks, warmblyctl status prints the version, make upgrade and scripts/upgrade-bare-metal.sh cover the by-hand paths, and docs gain an Updates page plus configuration, health, deployment and API reference updates
2026-09-03 05:04:30 -07:00
Matthew Meszaros
60c9e316d9
feat: hosted lead-capture forms end to end: drag-and-drop builder with field settings, design panel, embed/share and submissions tabs in the dashboard, a public TanStack form app (forms/) served by the new standalone forms service (cmd/forms + internal/formserver) on FORMS_DOMAIN with per-form frame-ancestors CSP, honeypot/fill-time/Turnstile/per-IP submit protection and a same-origin JSON API proxying the backend internal API, form submissions creating contacts with categories and campaign enrollment plus realtime, audit, webhook and org-transfer coverage, migration 000114, seed forms, CI jobs, Dockerfile, systemd/nginx/compose manifests and docs
2026-09-01 01:17:51 -07:00
Matthew Meszaros
9ab42cfd65
feat: build the browser half of social sign-in, which was never wired: GOOGLE_CLIENT_ID was read at boot and made the login screen render a Google button, but the button opened a popup at /auth/google/login which no route served, and authService.GoogleAuth/AppleAuth had no caller anywhere in the codebase; internal/app/socialauth now runs Google and Apple through the flow generic OIDC already used (one-time state, PKCE, nonce, id_token verified against the provider JWKS, identity keyed on issuer and subject, JIT provisioning, the ban and 2FA gates), the redirect URI defaults to API_PUBLIC_URL/v1/auth/<provider>/callback and is logged at boot because registering the dashboard origin instead is the mistake that produces a valid OAuth client and a dead button, /auth/config advertises only providers the backend can actually complete, the SSO landing page no longer swallows a two_fa_required response, and OIDC_PROVIDER_NAME finally reaches the button it documents
2026-08-28 01:33:08 -07:00
Matthew Meszaros
a75ea012a0
feat: import a mailbox's recent history on connect and govern sync by fair use: a backfill on every provider (newest first, inside an operator-editable window and cap, resumable through a durable per-provider cursor relayed as SYNC_STATE), a per-mailbox sync governor with priority, live and backfill lanes on shared Redis windows that defers over-budget mail with the cursor held instead of dropping it and only deactivates a mailbox for a flood or chronic daily overage, sync.* budgets on the admin instance settings shipped inside ADD_EMAIL, saved IMAP folder cursors and last_synced_at finally written, a Sync card in the mailbox drawer fed by GET /emails/:id/sync, and docs
2026-08-18 09:09:52 -07:00
Matthew Meszaros
734cb5fe08
feat: make self-hosted onboarding survivable by fixing invite_only, which could not onboard anyone (the accept route is JWT-only, so redeeming the invitation that would create your account required already having one, making the self-host default silently identical to fully closed), threading the invitation token through registration so an invited person lands in the inviting organization instead of a stray workspace, gating SSO just-in-time provisioning behind DISABLE_REGISTRATION (it bypassed the gate entirely, so an instance set to true was still open to anyone the IdP would assert) with SSO_AUTO_PROVISION as the opt-out, correcting the OIDC redirect URL that pointed at /api/v1 against a route at /v1 and 404'd every SSO login, scoping the first-launch exemption so it no longer overrides an explicit lockdown, preserving the remaining TTL when restoring a losing setup token so a public endpoint cannot hold the claim window open forever, replacing a generic 403 with typed registration_invite_only, registration_closed, invitation_invalid, setup_token_invalid and setup_already_complete codes that name the next step, logging why no claim link was issued on an already-claimed instance instead of staying silent, adding a warmblyctl operator CLI (status with health checks and a non-zero exit, reissuable setup-link, user create/list/reset-password/grant-admin/revoke-admin/disable-2fa, hash-password) so a locked-out operator no longer needs hand-written psql, adding read-only instance configuration over 104 environment variables with structural secret redaction and fingerprints, 35 health checks, a database-backed settings tier for the three keys no environment variable owns, hiding the signup form when the config already says invite_only rather than failing the whole form with a toast, and documenting first run, accounts and access, configuration, instance health and troubleshooting alongside the root .env.example the README told operators to write but never shipped ( #114 )
2026-08-16 05:58:11 +02:00
Matthew Meszaros