Commit Graph
198 Commits
Author SHA1 Message Date
Matthew Meszaros 62ea7ef906 feat: gate the dashboard version pill's update actions on a session that presented a second factor, carry session_mfa_verified on the web User model, pass the API error code into the permission-denied event and give admin_mfa_required its own two-factor dialog variant linking to Settings > Security instead of the Roles & access advice, and document the rule on the updates page 2026-09-22 03:42:56 -07:00
Matthew Meszaros 9426c0da51 feat: validate every person, workspace and company name through internal/pkg/displayname on each write path (profile, onboarding, setup, IdP sign-in, org create and rename, org import, enterprise inquiry, admin testers, warmblyctl) with a 400 invalid_name code, render stored names in platform email through the same rules, mirror them in the web forms, check the org slug format, and document the rules in error-codes, security and AGENTS.md 2026-09-21 03:34:03 -07:00
Matthew Meszaros 0a5e7947b4 feat: return a failed warmup retention delete from the worker so the bus redelivers it up to five times, re-key a Graph message in the map on every move so the sender copy's body can be dropped, never expunge a whole IMAP folder for one message (UID EXPUNGE, else MOVE to Trash, else refuse), build the two retention indexes concurrently in their own migrations 000193 and 000194, keep the dashboard stepper off 1 and 2 days, and describe retention as applying wherever the placement files warmup mail 2026-09-21 01:11:22 -07:00
Matthew Meszaros 1513419a2a feat: delete warmup mail from each mailbox once past a per-mailbox retention window (email_accounts.warmup_retention_days, else retention.warmup_mail_days, default 30) via a consumer sweep that retires the receipt and sender copy and a worker delete action that trashes on Gmail, deletes on Graph, expunges on IMAP and drops the stored body, prune per-message warmup records after retention.warmup_event_days, and count a warmup deletion as tampering only within 24 hours of arrival and never for a retired message, judging Gmail's Trash label on the same rule 2026-09-21 00:52:02 -07:00
Matthew Meszaros 6026168334 feat: stop blocking boot on the GeoIP download and swap the database in when it lands, retry a refused mirror with backoff honouring Retry-After, revalidate a database older than a week with If-Modified-Since instead of keeping the first copy forever, and add a twice-weekly job mirroring both MaxMind editions to a private bucket so the fleet stops spending a 30-a-day allowance per boot 2026-09-20 17:55:38 +02:00
Matthew Meszaros 5b16a09b02 feat: write public objects without a canned ACL so a bucket whose ownership is owner-enforced still stores avatars, form assets, OAuth logos and email-body images, give the passkey login challenge its own per-IP budget separate from the one password sign-in draws on, and surface the API's own message at upload and passkey call sites instead of a generic sentence 2026-09-20 15:40:52 +02:00
Matthew Meszaros addb956ad6 feat: shared TypeSafe client under internal/pkg/typesafe with inbox tagging phases 2 and 3 (hold, stop, task, suppress behind workspace switches, reversible ones on by default), labels seeded at workspace creation and by the follow-up sweep, premade inbox views (hot leads, needs a reply, follow up, declined, automated), typed reply classification and a reply_intent branch condition, an inbox agent draft gate, Advisor copy judgment with a cached editor re-check, warmup content lint, bounce cause classification that keeps a blocked address sendable, and per-form submission triage 2026-09-19 23:33:37 -07:00
Matthew Meszaros 4847b5cc57 feat: never let the connection clamp hand out more than the server's own share, since a comfort floor of ten on a twenty-connection database would have promised forty, count the phase offset toward a non-boot job's recorded next run so the panel does not show it overdue, and say in the configuration docs that the quarter assumes four clients and applies only when the probe answers 2026-09-19 20:08:11 +02:00
Matthew Meszaros af8b551d28 feat: stop exhausting the database's connection slots by clamping each process's pool to the share of max_connections the server actually leaves free, returning pooled connections after a minute instead of holding the high-water mark for thirty, and giving every scheduled job a fixed place in a 45-second window so the nine hourly loops stop opening a connection in the same instant 2026-09-19 19:47:56 +02:00
Matthew Meszaros acecd62c88 feat: scope mailbox disconnect and warmup lifecycle to the workspace rather than the member who connected the mailbox so an admin can act on every mailbox the list already shows them, evict a mailbox whose row is gone from every live worker when its provider errors arrive so a deleted mailbox stops calling the provider once a sync interval forever, subscribe before publishing the credential-validation job and classify a socket deadline as the retryable timeout it is, give the worker's validation reply its own budget so a slow mail host no longer loses a finished verdict, guard every global key handler against a keydown carrying no key, drop exceptions whose whole message is an object's default toString, make the Postgres pool size configurable, and record the CASA and security invariants in AGENTS.md 2026-09-19 12:49:46 +02:00
Matthew Meszaros b09ec39907 Merge pull request #599 from warmbly/chore/casa-al1-security-assessment
feat: complete the ADA CASA AL1 control set and ship the assessment evidence pack
2026-09-19 06:39:12 +00:00
Matthew Meszaros e668a2a36b feat: complete the ADA CASA v2.1.1 AL1 control set across authentication, sessions, access control, cryptography, input validation and configuration, adding a breached-password denylist and per-account login throttling, enforced multi-factor authentication on the admin panel, step-up confirmation before an action that mints a lasting credential, purpose-scoped session tokens, single-use TOTP steps, tenant verification on every cross-referenced identifier, security headers on every surface, encrypted webhook signing secrets, per-organization idempotency, PKCE and a minimal two-scope Gmail consent on the mailbox OAuth flow, bounded spreadsheet and archive decoding, a patched Go toolchain with govulncheck in CI, and the evidence pack under compliance/casa 2026-09-19 08:18:35 +02:00
Matthew Meszaros e8f14bb2fd feat: address the review on the Gmail app-password connect by reading BOX_GOOGLE_OAUTH_CONNECT through config.GoogleOAuthConnect in the instance-settings table so a yes/on value cannot display true against a gate that parses it as false, dropping the coming-soon line from the walkthrough banner on deployments where Google sign-in is actually available, naming the 2-Step Verification app-password control an administrator still has rather than the Less secure apps page Google removed, saying the OAuth client re-authorizes existing mailboxes as well as refreshing them, and marking the marketing send trace as the Google sign-in path 2026-09-18 22:18:56 -07:00
Matthew Meszaros ee46cb49e8 feat: route new Gmail and Google Workspace mailboxes through a guided three-step app-password connect over smtp.gmail.com and imap.gmail.com instead of Google sign-in, behind BOX_GOOGLE_OAUTH_CONNECT (off by default) and announced to clients as gmail_oauth_connect on /auth/config, refusing a new gmail OAuth start with 403 mailbox_gmail_oauth_disabled in both the direct and Warmbly Cloud broker paths while leaving mailboxes already connected that way sending, syncing and re-authorizable 2026-09-18 22:06:09 -07:00
Matthew Meszaros 0e914ee390 feat: stop the password reset flow reporting success while sending nothing, by answering 200 only for an address with no account rather than for every cache and database fault, folding addresses to one case on every account lookup and write so a typed capital cannot miss the row or split an SSO account in two, refunding the two-per-four-hours budget when the failure was ours, retrying one transient send and quoting the link's real lifetime; and clear the rest of error tracking by grouping the engagement breakdown in a subquery so ORDER BY stops resolving opens against email_opens, dropping unibox_mailboxes and email_sync_state writes whose mailbox was deleted mid-sync instead of redelivering them forever, answering a corrupt argon2 hash with ErrCredentials rather than a 500, never filing a cancelled caller as a database incident, and reporting only the first websocket init failure of a streak 2026-09-18 11:42:49 +02:00
Matthew Meszaros a0a19edeae feat: register a schema document whose fixed defaults are code-point strings and whose nested nulls are null so the registered envelope parses back, keep the warmup unibox row until the filing action is on the bus and the mailbox lookup is not a transient failure, recheck the heartbeat key before evacuating a worker, match the IMAP namespace prefix case-insensitively, and state the BACKWARD direction correctly 2026-09-18 11:29:48 +02:00
Matthew Meszaros bd1837833e feat: give every derived Avro field its zero as a default and register the marshalled schema so adding a field cannot refuse the whole envelope and stop every publish, file historical warmup leaks out of the customer's own mailbox rather than only the unibox, and make a worker earn a 10-minute absence before its mailboxes are evacuated so a version rollout costs no migrations 2026-09-18 10:34:36 +02:00
Matthew Meszaros 8240f14e8b feat: fail closed on incomplete cloud mailbox revocation and document retry-safe deletion and TLS diagnostics 2026-09-17 21:21:34 -07:00
Matthew Meszaros 8ee1c50a1b feat: make a failed SMTP send name the step and the cause behind it instead of one bare SERVER_UNREACHABLE sentinel, give a refused warmup send its day back so sent_today can no longer climb past the target while the cap frees the slot, revoke a mailbox's Warmbly Cloud enrollment when it is deleted so the pool stops holding its password, and prefer warmup partners outside the sender's own workspace while showing the partner diversity a mailbox is actually getting (#574, #575) 2026-09-17 20:02:37 -07:00
Matthew Meszaros e36a070595 Merge remote-tracking branch 'origin/main' into fix/worker-capacity-account-distribution 2026-09-17 08:02:45 -07:00
Matthew Meszaros 8a60c6bab3 feat: build realtime on OTP 27 and update vulnerable Elixir dependencies 2026-09-17 07:54:34 -07:00
Matthew Meszaros 2a1e55354d feat: merge latest main before requeueing worker capacity fixes
# Conflicts:
#	internal/app/stripe/service_test.go
2026-09-17 06:57:20 -07:00
Matthew Meszaros c1e45b194a feat: merge latest main before worker capacity queueing 2026-09-17 06:26:07 -07:00
Matthew Meszaros 9a2c565abe feat: enable Stripe Tax for subscriptions, credit purchases, automatic top-ups, billing details, and failure alerts 2026-09-17 15:25:26 +02:00
Matthew Meszaros d6025ea4c0 feat: address worker capacity review findings with safe migrations and recovery reporting 2026-09-17 06:24:14 -07:00
Matthew Meszaros dcf26a2361 feat: make automatic inbox tagging atomic live and reviewable in production 2026-09-17 04:57:24 -07:00
Matthew Meszaros bab9f86727 feat: correct worker capacity, mailbox distribution, observed IPv4, fleet pagination, and premium pool promotion 2026-09-17 04:15:05 -07:00
Matthew Meszaros 03f813dd43 feat: let an admin create a tester account that joins an existing workspace with an explicitly chosen role instead of always minting an empty one, so an OAuth reviewer signing in lands in a workspace that shows the app doing real work 2026-09-16 19:37:32 +02:00
Matthew Meszaros 6872ebbb23 feat: fix dashboard Stripe checkout, portal eligibility, webhook recovery and audit rendering 2026-09-16 15:23:17 +02:00
Matthew Meszaros 31c1f101c2 feat: give EmailSentEvent and WarmupEmailSentEvent their own derived Avro schemas so the email-events and warmup-events analytics streams stop failing at serialize on every send and finally register a subject (#546) 2026-09-16 04:20:15 -07:00
Matthew Meszaros 140c7de436 feat: add the admin panel's Promo codes page and route the six /admin/discounts endpoints that existed as handlers but were never wired, so a launch offer is built in the operator UI instead of an INSERT against production, with caps that an explicit null can actually clear on PATCH 2026-09-16 04:04:09 -07:00
Matthew Meszaros 561f8ec671 feat: keep cloud and local warmup mail out of Unibox with durable verification and automatic cleanup 2026-09-16 03:55:36 -07:00
Matthew Meszaros f106c8541d feat: make the bus envelopes Avro-encodable (#535)
* feat: make both bus envelopes Avro-encodable by deriving each one's schema from a declared registry of body types, with a union branch per body and our own struct walk that skips unexported fields and honours avro:"-" before descending, so the schema describes exactly what encoding/json already puts on the wire, and narrow the two sync cursors on the wire DTOs to int64 because Avro has no unsigned 64-bit type

* feat: stop the instance health check, the config registry and the docs all claiming Avro cannot serialize a worker envelope, which stopped being true once the envelopes carried a declared union, and check the one thing that is still a real misconfiguration instead: avro selected with no SCHEMA_REGISTRY_URL to resolve against

* feat: emit a reference the second time a record appears in an envelope schema instead of defining it again, because Avro names a record once and a document that defines warmbly.events.Token three times is rejected outright, and keep the Schema Registry round-trip as a skip-by-default test since only a registry judges the document rather than the objects it was built from

* feat: carry uint64 as Avro fixed(8) rather than long, which lets the sync cursors keep their unsigned type instead of being narrowed, name every event field after its json tag so the schema and the JSON wire agree, and populate every field in the round-trip test because zero values are why a uint64 mapped to long passed in the first place

* feat: frame Avro in Confluent's wire format and encode through hamba's default API instead of going through avrov2, whose private avro.API holds a type resolver avro.Register cannot reach, so a union body failed there with unable to resolve type while encoding cleanly against the same schema, and keep the registry round-trip as a skip-by-default test
2026-09-15 10:50:30 -07:00
Matthew Meszaros ae012dd13f Clear the live error-tracking issues, and the workspace rename that renamed the wrong workspace (#533)
* feat: stop a managed Kafka cluster refusing topic creation from failing the publish, by treating a topic- or cluster-authorization failure from CreateTopics as a topic the cluster owns rather than one that is missing, which on Confluent Cloud dropped every warmup event and filed one issue per message because the topic never became known

* feat: drop a report whose error is a cancelled context in errs rather than at ninety call sites, so a browser navigating away or a container draining on deploy stops filing one issue per query that happened to be in flight, while a deadline this process set and blew through still reports

* feat: stop renaming one workspace from renaming another, by keying the workspace settings editor on the workspace id so a switch re-seeds the name field instead of leaving the previous workspace's name against the new workspace's autosave baseline, and pinning every save on the workspace, sending and tracking pages to the workspace its draft was hydrated from

* feat: drop Script error. and the ResizeObserver notice on the marketing site and the hosted form page the way the dashboard and admin panel already do, since those two carry no stack and no bug and between them were the largest issues in error tracking, all of it from warmbly.com

* feat: rename the forms Turnstile script module to turnstileScript.ts so it no longer differs only in case from the Turnstile.tsx component, which resolved both imports to one file on a case-insensitive filesystem and failed forms' typecheck with TS1149

* feat: upload source maps from the static build:pages build as well as the image build, so the dashboards served from a static host stop reporting every stack frame as a minified name beside 'Invalid source map: bad json', which is PostHog falling back to fetching the .map from a host that answers with its SPA fallback

* feat: build every admin list in pg_admin.go with make rather than declaring it nil, so an empty page serializes as [] instead of null, and guard the audit table's own empty check, which is what crashed admin.warmbly.com/audit with 'null is not an object (evaluating d.data.length)' whenever a filter matched nothing

* feat: match the whole broker description rather than a substring when deciding a topic create was refused for permissions, since that answer remembers the topic as present, and clear the cached promise and dead tag when the forms Turnstile script fails to load so a blocked first attempt no longer leaves every later mount with the same rejection and the captcha permanently missing
2026-09-15 09:05:53 -07:00
Matthew Meszaros 2f15fecaad feat: document the MaxMind download endpoint that a current licence key actually authenticates against, account id as the username and key as the password in the URL userinfo, because the app/geoip_download permalink every guide still shows refuses a key issued today with Invalid license key (#530) 2026-09-15 03:37:14 -07:00
Matthew Meszaros 4784ee7d39 feat: fetch the MaxMind databases instead of requiring a mounted file (#529)
* feat: let the backend, consumer and tracking service fetch their own MaxMind databases from GEODB_URL and TRACKING_SCANNER_ASN_DB_URL, reading the archive shape from the content so a permalink tar.gz, a gzipped mmdb and a bare mmdb all work, never replacing a file already at the path, opening the bytes before installing them so a licence-key error page cannot become the database forever, skipping the AppleDouble sidecars a macOS tar writes ahead of the real file, and treating both URLs as secrets because the permalink carries the licence key

* feat: drop the trailing blank line cargo fmt --check rejects at the end of tracking/src/asndb.rs

* feat: stream the downloaded ASN archive instead of decompressing it whole, sizing each buffer from the gzip footer and the tar header so the member is allocated exactly once, which drops the peak of unwrapping a permalink tar.gz from 38 MB to 11.9 MB, essentially the database itself

* feat: stop the MaxMind licence key reaching the logs through net/http's and reqwest's own error text, which both print the URL they were given and so defeated the redaction beside them, drop userinfo as well as the query when redacting, refuse plain http for a URL carrying a credential and refuse an https-to-http redirect, and apply the size cap to the decoded database rather than the compressed transfer so a gzip bomb cannot fill the disk

* feat: strip basic-auth userinfo as well as the query when the tracking service redacts its database URL, parsing it rather than cutting at the first question mark so where a credential sits is the URL library's problem and not a guess
2026-09-15 03:06:34 -07:00
Matthew Meszaros 0a1ed6f04e feat: resolve scanner ASNs from a GeoLite2-ASN database so the catalogue's asn: entries match without a Cloudflare transform rule, ship the Proofpoint, Mimecast and Cisco ASNs enabled behind a new probable certainty that widens the consumer's machine window instead of deciding the verdict, make the tracking event dedupe claim one coalesced operation, and report an ASN database that opened cleanly but resolves nothing (#440) 2026-09-15 01:48:45 -07:00
Matthew Meszaros 619eb2729a fix: read a mailbox's Gmail send-as addresses from the worker holding it rather than from the backend, which was decrypting a mailbox credential in the control plane and showing Google a second client address for a mailbox whose mail moves through a worker, by round-tripping a new MAILBOX_IDENTITY command answered on the process channel like a credential validation, leaving the OAuth handshake as the one place the control plane still calls the provider (#522) 2026-09-14 21:28:07 -07:00
Matthew Meszaros 1ca3bd19b3 feat: carry a unibox read or unread change out to the mailbox itself through a new MESSAGE_SEEN worker command, so a conversation read in Warmbly stops showing bold in Gmail, Outlook and IMAP, relaying the state the row holds rather than the one the request asked for, only for messages that actually changed, dispatched detached from the request and never retried (#515) 2026-09-14 21:20:42 -07:00
Matthew Meszaros a52a894110 feat: let the OpenAI provider adapt to a model that refuses function tools unless reasoning is off, by flipping a sticky reasoning_effort=none flag on the 400 that names it and widening the per-call compatibility retry budget to cover every flag, since gpt-5.6-luna rejects three parameters in a row and the old budget of two ended the call before the third adaptation (#513) 2026-09-14 08:20:08 -07:00
Matthew Meszaros c28f915648 feat: erase everything a disconnected mailbox leaves behind, revoking its OAuth grant at Google and deleting its stored message bodies through a durable retried queue, cascade the nine mailbox foreign keys that had none so warmup receipts, tampering events and provider message maps stop outliving the mailbox, clear thread labels and snoozes on conversations the delete emptied, make workspace deletion possible at all by cascading the four organization foreign keys with no delete action, and put Disconnect in the mailbox row menu and a Settings danger zone since it was only reachable from the selection bar (#506) 2026-09-14 07:55:01 -07:00
Matthew Meszaros 40506c4f05 fix: seed the two warmup pools on every instance under fixed ids and make one pool per type structural, since the baseline squash dropped the insert and a fresh self-hosted instance never warmed; move memberships onto the canonical pools, scope the standing mirror trigger to the columns it mirrors so a pool move keeps a retention window, commit the runtime and every seeder to the ids through MoveToPool, assert the pools at boot and in a warmup_pools_missing health check, and drop the guide's claim of cross-tier borrowing the health gate rejects (#493) 2026-09-13 21:37:17 -07:00
Matthew Meszaros 7300b3b021 feat: full PostHog coverage: identify the signed-in user and workspace in the dashboard and admin panel with autocapture, heatmaps, dead and rage clicks, web vitals, network timing, console capture and session replay masking only password fields, send server-side signup, trial and subscription events under the user id with the organization as a group, keep the marketing site and form pages cookieless while capturing everything stateless plus a form funnel, upload the form app's source maps, and add WARMBLY_POSTHOG_SESSION_REPLAY 2026-09-13 20:58:22 -07:00
Matthew Meszaros 1dc4aedc3c fix: retire the warmup invalid-token band with its table, metric query, service and repository methods and admin tab, since nothing has fed it since #481 and no attributable forged-token signal exists; key the live pool fixtures on the canonical pool ids so the warmup, repository routing and tasks routing suites run on a fresh database, and correct every doc, site and advisor line that still described the retired signal or a spam-score threshold nothing implements (#490) 2026-09-13 08:09:01 -07:00
Matthew Meszaros 8799680166 fix: never charge a mailbox for a warmup token that arrived in its inbox, hold a quarantine or block for its full term against fresh metrics, and keep a penalised address's standing across removal, pool exit and export through a trigger-maintained mirror, since the recipient never controlled the token, the bands read seven days against 30-day terms, and the pool row died on paths a snapshot at deletion never saw (#481) 2026-09-13 04:34:44 -07:00
Matthew Meszaros 43dcbde06c feat: tester accounts, creatable from the admin panel (#483)
* feat: excuse one named account from the emailed login code, so a vendor reviewer who cannot read this instance's mail can sign in without turning codes off for everyone, with the reason recorded beside it and every run of warmblyctl status naming the accounts that hold one

* feat: create and manage tester accounts from the admin panel, so letting a reviewer in is a form rather than a shell, with the password shown once and every live exemption listed on one page because forgetting one is the way this goes wrong

* fix: give tester management its own permission bit rather than borrowing ban_users, create the account and its exemption in one transaction so no invisible orphan survives a failure, require an accountable operator on the CLI grant, stop a halted row scan reading as the whole exempt list, and show a failed query as an error instead of as no testers

* chore: re-run CI after the aggregator tripped on a cancelled job from the branch update, with every underlying job green

* chore: retrigger CI, the previous run sat queued indefinitely while other branches ran

* feat: roll back a half-created tester when its workspace step fails and backfill the manage-testers bit onto admins already holding every other permission, so the address is not left taken by an unusable account and the new routes are not 403 for the existing admin

* feat: make the 000151 manage-testers backfill one-way, because clearing bit 22 on the way down would also revoke it from an admin granted it explicitly afterwards and the up migration would not restore that
2026-09-13 03:07:10 -07:00
Matthew Meszaros 7f74664c72 fix: a granted plan unlocks nothing, and Turnstile never renders (#474)
* fix: let a granted plan unlock the dashboard, since the client decided paid from the Stripe status a managed subscription never touches, and replace the Turnstile size Cloudflare removed so the widget renders and can issue a token at all

* feat: tell people on a preview deployment that it is a public beta, once in a dialog and thereafter as a header pill they can reopen, driven by a config value rather than a hostname so one image stays reusable, and bind both Turnstile modals through onLoad because the component is not forwardRef and execution=execute never fires without the widget instance

* fix: keep the beta pill outside the desktop-only header group so the notice stays reopenable on a phone, and say in the docs that the value is baked into config.js at container start rather than read per load
2026-09-12 21:45:05 -07:00
Matthew Meszaros d456bc48c6 feat: fix the Warmbly Cloud pool link across both roles (#262): take an enrolled mailbox out of this instance's own warmup pool so local partners stop writing to it and their unverifiable warmup stops landing in the owner's unibox, recognise the cloud's warmup mail whose verify header did not survive delivery through a new warmup-deliveries lookup that ignores consumed_at because instance and cloud read the same mailbox, move the managed-mailbox access token route behind NODE_BROKER_TOKEN so the internet-facing tracking and forms services can no longer mint a live provider token, scope pause and resume to the caller's workspace, keep an enrolled mailbox listed once it goes inactive, release the cloud copy when the local mirror row cannot be written, refuse the one-time handshake when CREDENTIALS_ENCRYPTION_KEY is missing, blank an expired code's plaintext instance token, and stop errx answering 200 for a status outside its table 2026-09-12 06:58:25 -07:00
Matthew Meszaros b0050507e4 fix: tell a joining node a tag that exists, falling back to the published prod tag instead of a latest this project has never published, and drop the WORKER_IMAGE docs left behind by the removed push-based provisioning (#455) 2026-09-12 01:00:11 -07:00
Matthew Meszaros a4f503f620 fix: realtime cannot complete either of its TLS connections (#452)
* fix: let the realtime service actually establish its TLS connections, adding the wildcard hostname match fun Redix needs for every managed Redis and a CA bundle option for an RDS Postgres that the system trust store cannot verify

* feat: drop the redundant wget package from the realtime image since busybox already provides it, and fix a comment that read as literal interpolation

* fix: treat a blank DATABASE_SSL_CA_FILE as unset in the realtime config, because compose passes every optional variable through as empty and cacertfile with an empty path is no CA source at all rather than a fallback to the system store
2026-09-12 00:29:34 -07:00