Commit Graph
184 Commits
Author SHA1 Message Date
Matthew Meszaros 0e914ee390 feat: stop the password reset flow reporting success while sending nothing, by answering 200 only for an address with no account rather than for every cache and database fault, folding addresses to one case on every account lookup and write so a typed capital cannot miss the row or split an SSO account in two, refunding the two-per-four-hours budget when the failure was ours, retrying one transient send and quoting the link's real lifetime; and clear the rest of error tracking by grouping the engagement breakdown in a subquery so ORDER BY stops resolving opens against email_opens, dropping unibox_mailboxes and email_sync_state writes whose mailbox was deleted mid-sync instead of redelivering them forever, answering a corrupt argon2 hash with ErrCredentials rather than a 500, never filing a cancelled caller as a database incident, and reporting only the first websocket init failure of a streak 2026-09-18 11:42:49 +02:00
Matthew Meszaros a0a19edeae feat: register a schema document whose fixed defaults are code-point strings and whose nested nulls are null so the registered envelope parses back, keep the warmup unibox row until the filing action is on the bus and the mailbox lookup is not a transient failure, recheck the heartbeat key before evacuating a worker, match the IMAP namespace prefix case-insensitively, and state the BACKWARD direction correctly 2026-09-18 11:29:48 +02:00
Matthew Meszaros bd1837833e feat: give every derived Avro field its zero as a default and register the marshalled schema so adding a field cannot refuse the whole envelope and stop every publish, file historical warmup leaks out of the customer's own mailbox rather than only the unibox, and make a worker earn a 10-minute absence before its mailboxes are evacuated so a version rollout costs no migrations 2026-09-18 10:34:36 +02:00
Matthew Meszaros 8240f14e8b feat: fail closed on incomplete cloud mailbox revocation and document retry-safe deletion and TLS diagnostics 2026-09-17 21:21:34 -07:00
Matthew Meszaros 8ee1c50a1b feat: make a failed SMTP send name the step and the cause behind it instead of one bare SERVER_UNREACHABLE sentinel, give a refused warmup send its day back so sent_today can no longer climb past the target while the cap frees the slot, revoke a mailbox's Warmbly Cloud enrollment when it is deleted so the pool stops holding its password, and prefer warmup partners outside the sender's own workspace while showing the partner diversity a mailbox is actually getting (#574, #575) 2026-09-17 20:02:37 -07:00
Matthew Meszaros e36a070595 Merge remote-tracking branch 'origin/main' into fix/worker-capacity-account-distribution 2026-09-17 08:02:45 -07:00
Matthew Meszaros 8a60c6bab3 feat: build realtime on OTP 27 and update vulnerable Elixir dependencies 2026-09-17 07:54:34 -07:00
Matthew Meszaros 2a1e55354d feat: merge latest main before requeueing worker capacity fixes
# Conflicts:
#	internal/app/stripe/service_test.go
2026-09-17 06:57:20 -07:00
Matthew Meszaros c1e45b194a feat: merge latest main before worker capacity queueing 2026-09-17 06:26:07 -07:00
Matthew Meszaros 9a2c565abe feat: enable Stripe Tax for subscriptions, credit purchases, automatic top-ups, billing details, and failure alerts 2026-09-17 15:25:26 +02:00
Matthew Meszaros d6025ea4c0 feat: address worker capacity review findings with safe migrations and recovery reporting 2026-09-17 06:24:14 -07:00
Matthew Meszaros dcf26a2361 feat: make automatic inbox tagging atomic live and reviewable in production 2026-09-17 04:57:24 -07:00
Matthew Meszaros bab9f86727 feat: correct worker capacity, mailbox distribution, observed IPv4, fleet pagination, and premium pool promotion 2026-09-17 04:15:05 -07:00
Matthew Meszaros 03f813dd43 feat: let an admin create a tester account that joins an existing workspace with an explicitly chosen role instead of always minting an empty one, so an OAuth reviewer signing in lands in a workspace that shows the app doing real work 2026-09-16 19:37:32 +02:00
Matthew Meszaros 6872ebbb23 feat: fix dashboard Stripe checkout, portal eligibility, webhook recovery and audit rendering 2026-09-16 15:23:17 +02:00
Matthew Meszaros 31c1f101c2 feat: give EmailSentEvent and WarmupEmailSentEvent their own derived Avro schemas so the email-events and warmup-events analytics streams stop failing at serialize on every send and finally register a subject (#546) 2026-09-16 04:20:15 -07:00
Matthew Meszaros 140c7de436 feat: add the admin panel's Promo codes page and route the six /admin/discounts endpoints that existed as handlers but were never wired, so a launch offer is built in the operator UI instead of an INSERT against production, with caps that an explicit null can actually clear on PATCH 2026-09-16 04:04:09 -07:00
Matthew Meszaros 561f8ec671 feat: keep cloud and local warmup mail out of Unibox with durable verification and automatic cleanup 2026-09-16 03:55:36 -07:00
Matthew Meszaros f106c8541d feat: make the bus envelopes Avro-encodable (#535)
* feat: make both bus envelopes Avro-encodable by deriving each one's schema from a declared registry of body types, with a union branch per body and our own struct walk that skips unexported fields and honours avro:"-" before descending, so the schema describes exactly what encoding/json already puts on the wire, and narrow the two sync cursors on the wire DTOs to int64 because Avro has no unsigned 64-bit type

* feat: stop the instance health check, the config registry and the docs all claiming Avro cannot serialize a worker envelope, which stopped being true once the envelopes carried a declared union, and check the one thing that is still a real misconfiguration instead: avro selected with no SCHEMA_REGISTRY_URL to resolve against

* feat: emit a reference the second time a record appears in an envelope schema instead of defining it again, because Avro names a record once and a document that defines warmbly.events.Token three times is rejected outright, and keep the Schema Registry round-trip as a skip-by-default test since only a registry judges the document rather than the objects it was built from

* feat: carry uint64 as Avro fixed(8) rather than long, which lets the sync cursors keep their unsigned type instead of being narrowed, name every event field after its json tag so the schema and the JSON wire agree, and populate every field in the round-trip test because zero values are why a uint64 mapped to long passed in the first place

* feat: frame Avro in Confluent's wire format and encode through hamba's default API instead of going through avrov2, whose private avro.API holds a type resolver avro.Register cannot reach, so a union body failed there with unable to resolve type while encoding cleanly against the same schema, and keep the registry round-trip as a skip-by-default test
2026-09-15 10:50:30 -07:00
Matthew Meszaros ae012dd13f Clear the live error-tracking issues, and the workspace rename that renamed the wrong workspace (#533)
* feat: stop a managed Kafka cluster refusing topic creation from failing the publish, by treating a topic- or cluster-authorization failure from CreateTopics as a topic the cluster owns rather than one that is missing, which on Confluent Cloud dropped every warmup event and filed one issue per message because the topic never became known

* feat: drop a report whose error is a cancelled context in errs rather than at ninety call sites, so a browser navigating away or a container draining on deploy stops filing one issue per query that happened to be in flight, while a deadline this process set and blew through still reports

* feat: stop renaming one workspace from renaming another, by keying the workspace settings editor on the workspace id so a switch re-seeds the name field instead of leaving the previous workspace's name against the new workspace's autosave baseline, and pinning every save on the workspace, sending and tracking pages to the workspace its draft was hydrated from

* feat: drop Script error. and the ResizeObserver notice on the marketing site and the hosted form page the way the dashboard and admin panel already do, since those two carry no stack and no bug and between them were the largest issues in error tracking, all of it from warmbly.com

* feat: rename the forms Turnstile script module to turnstileScript.ts so it no longer differs only in case from the Turnstile.tsx component, which resolved both imports to one file on a case-insensitive filesystem and failed forms' typecheck with TS1149

* feat: upload source maps from the static build:pages build as well as the image build, so the dashboards served from a static host stop reporting every stack frame as a minified name beside 'Invalid source map: bad json', which is PostHog falling back to fetching the .map from a host that answers with its SPA fallback

* feat: build every admin list in pg_admin.go with make rather than declaring it nil, so an empty page serializes as [] instead of null, and guard the audit table's own empty check, which is what crashed admin.warmbly.com/audit with 'null is not an object (evaluating d.data.length)' whenever a filter matched nothing

* feat: match the whole broker description rather than a substring when deciding a topic create was refused for permissions, since that answer remembers the topic as present, and clear the cached promise and dead tag when the forms Turnstile script fails to load so a blocked first attempt no longer leaves every later mount with the same rejection and the captcha permanently missing
2026-09-15 09:05:53 -07:00
Matthew Meszaros 2f15fecaad feat: document the MaxMind download endpoint that a current licence key actually authenticates against, account id as the username and key as the password in the URL userinfo, because the app/geoip_download permalink every guide still shows refuses a key issued today with Invalid license key (#530) 2026-09-15 03:37:14 -07:00
Matthew Meszaros 4784ee7d39 feat: fetch the MaxMind databases instead of requiring a mounted file (#529)
* feat: let the backend, consumer and tracking service fetch their own MaxMind databases from GEODB_URL and TRACKING_SCANNER_ASN_DB_URL, reading the archive shape from the content so a permalink tar.gz, a gzipped mmdb and a bare mmdb all work, never replacing a file already at the path, opening the bytes before installing them so a licence-key error page cannot become the database forever, skipping the AppleDouble sidecars a macOS tar writes ahead of the real file, and treating both URLs as secrets because the permalink carries the licence key

* feat: drop the trailing blank line cargo fmt --check rejects at the end of tracking/src/asndb.rs

* feat: stream the downloaded ASN archive instead of decompressing it whole, sizing each buffer from the gzip footer and the tar header so the member is allocated exactly once, which drops the peak of unwrapping a permalink tar.gz from 38 MB to 11.9 MB, essentially the database itself

* feat: stop the MaxMind licence key reaching the logs through net/http's and reqwest's own error text, which both print the URL they were given and so defeated the redaction beside them, drop userinfo as well as the query when redacting, refuse plain http for a URL carrying a credential and refuse an https-to-http redirect, and apply the size cap to the decoded database rather than the compressed transfer so a gzip bomb cannot fill the disk

* feat: strip basic-auth userinfo as well as the query when the tracking service redacts its database URL, parsing it rather than cutting at the first question mark so where a credential sits is the URL library's problem and not a guess
2026-09-15 03:06:34 -07:00
Matthew Meszaros 0a1ed6f04e feat: resolve scanner ASNs from a GeoLite2-ASN database so the catalogue's asn: entries match without a Cloudflare transform rule, ship the Proofpoint, Mimecast and Cisco ASNs enabled behind a new probable certainty that widens the consumer's machine window instead of deciding the verdict, make the tracking event dedupe claim one coalesced operation, and report an ASN database that opened cleanly but resolves nothing (#440) 2026-09-15 01:48:45 -07:00
Matthew Meszaros 619eb2729a fix: read a mailbox's Gmail send-as addresses from the worker holding it rather than from the backend, which was decrypting a mailbox credential in the control plane and showing Google a second client address for a mailbox whose mail moves through a worker, by round-tripping a new MAILBOX_IDENTITY command answered on the process channel like a credential validation, leaving the OAuth handshake as the one place the control plane still calls the provider (#522) 2026-09-14 21:28:07 -07:00
Matthew Meszaros 1ca3bd19b3 feat: carry a unibox read or unread change out to the mailbox itself through a new MESSAGE_SEEN worker command, so a conversation read in Warmbly stops showing bold in Gmail, Outlook and IMAP, relaying the state the row holds rather than the one the request asked for, only for messages that actually changed, dispatched detached from the request and never retried (#515) 2026-09-14 21:20:42 -07:00
Matthew Meszaros a52a894110 feat: let the OpenAI provider adapt to a model that refuses function tools unless reasoning is off, by flipping a sticky reasoning_effort=none flag on the 400 that names it and widening the per-call compatibility retry budget to cover every flag, since gpt-5.6-luna rejects three parameters in a row and the old budget of two ended the call before the third adaptation (#513) 2026-09-14 08:20:08 -07:00
Matthew Meszaros c28f915648 feat: erase everything a disconnected mailbox leaves behind, revoking its OAuth grant at Google and deleting its stored message bodies through a durable retried queue, cascade the nine mailbox foreign keys that had none so warmup receipts, tampering events and provider message maps stop outliving the mailbox, clear thread labels and snoozes on conversations the delete emptied, make workspace deletion possible at all by cascading the four organization foreign keys with no delete action, and put Disconnect in the mailbox row menu and a Settings danger zone since it was only reachable from the selection bar (#506) 2026-09-14 07:55:01 -07:00
Matthew Meszaros 40506c4f05 fix: seed the two warmup pools on every instance under fixed ids and make one pool per type structural, since the baseline squash dropped the insert and a fresh self-hosted instance never warmed; move memberships onto the canonical pools, scope the standing mirror trigger to the columns it mirrors so a pool move keeps a retention window, commit the runtime and every seeder to the ids through MoveToPool, assert the pools at boot and in a warmup_pools_missing health check, and drop the guide's claim of cross-tier borrowing the health gate rejects (#493) 2026-09-13 21:37:17 -07:00
Matthew Meszaros 7300b3b021 feat: full PostHog coverage: identify the signed-in user and workspace in the dashboard and admin panel with autocapture, heatmaps, dead and rage clicks, web vitals, network timing, console capture and session replay masking only password fields, send server-side signup, trial and subscription events under the user id with the organization as a group, keep the marketing site and form pages cookieless while capturing everything stateless plus a form funnel, upload the form app's source maps, and add WARMBLY_POSTHOG_SESSION_REPLAY 2026-09-13 20:58:22 -07:00
Matthew Meszaros 1dc4aedc3c fix: retire the warmup invalid-token band with its table, metric query, service and repository methods and admin tab, since nothing has fed it since #481 and no attributable forged-token signal exists; key the live pool fixtures on the canonical pool ids so the warmup, repository routing and tasks routing suites run on a fresh database, and correct every doc, site and advisor line that still described the retired signal or a spam-score threshold nothing implements (#490) 2026-09-13 08:09:01 -07:00
Matthew Meszaros 8799680166 fix: never charge a mailbox for a warmup token that arrived in its inbox, hold a quarantine or block for its full term against fresh metrics, and keep a penalised address's standing across removal, pool exit and export through a trigger-maintained mirror, since the recipient never controlled the token, the bands read seven days against 30-day terms, and the pool row died on paths a snapshot at deletion never saw (#481) 2026-09-13 04:34:44 -07:00
Matthew Meszaros 43dcbde06c feat: tester accounts, creatable from the admin panel (#483)
* feat: excuse one named account from the emailed login code, so a vendor reviewer who cannot read this instance's mail can sign in without turning codes off for everyone, with the reason recorded beside it and every run of warmblyctl status naming the accounts that hold one

* feat: create and manage tester accounts from the admin panel, so letting a reviewer in is a form rather than a shell, with the password shown once and every live exemption listed on one page because forgetting one is the way this goes wrong

* fix: give tester management its own permission bit rather than borrowing ban_users, create the account and its exemption in one transaction so no invisible orphan survives a failure, require an accountable operator on the CLI grant, stop a halted row scan reading as the whole exempt list, and show a failed query as an error instead of as no testers

* chore: re-run CI after the aggregator tripped on a cancelled job from the branch update, with every underlying job green

* chore: retrigger CI, the previous run sat queued indefinitely while other branches ran

* feat: roll back a half-created tester when its workspace step fails and backfill the manage-testers bit onto admins already holding every other permission, so the address is not left taken by an unusable account and the new routes are not 403 for the existing admin

* feat: make the 000151 manage-testers backfill one-way, because clearing bit 22 on the way down would also revoke it from an admin granted it explicitly afterwards and the up migration would not restore that
2026-09-13 03:07:10 -07:00
Matthew Meszaros 7f74664c72 fix: a granted plan unlocks nothing, and Turnstile never renders (#474)
* fix: let a granted plan unlock the dashboard, since the client decided paid from the Stripe status a managed subscription never touches, and replace the Turnstile size Cloudflare removed so the widget renders and can issue a token at all

* feat: tell people on a preview deployment that it is a public beta, once in a dialog and thereafter as a header pill they can reopen, driven by a config value rather than a hostname so one image stays reusable, and bind both Turnstile modals through onLoad because the component is not forwardRef and execution=execute never fires without the widget instance

* fix: keep the beta pill outside the desktop-only header group so the notice stays reopenable on a phone, and say in the docs that the value is baked into config.js at container start rather than read per load
2026-09-12 21:45:05 -07:00
Matthew Meszaros d456bc48c6 feat: fix the Warmbly Cloud pool link across both roles (#262): take an enrolled mailbox out of this instance's own warmup pool so local partners stop writing to it and their unverifiable warmup stops landing in the owner's unibox, recognise the cloud's warmup mail whose verify header did not survive delivery through a new warmup-deliveries lookup that ignores consumed_at because instance and cloud read the same mailbox, move the managed-mailbox access token route behind NODE_BROKER_TOKEN so the internet-facing tracking and forms services can no longer mint a live provider token, scope pause and resume to the caller's workspace, keep an enrolled mailbox listed once it goes inactive, release the cloud copy when the local mirror row cannot be written, refuse the one-time handshake when CREDENTIALS_ENCRYPTION_KEY is missing, blank an expired code's plaintext instance token, and stop errx answering 200 for a status outside its table 2026-09-12 06:58:25 -07:00
Matthew Meszaros b0050507e4 fix: tell a joining node a tag that exists, falling back to the published prod tag instead of a latest this project has never published, and drop the WORKER_IMAGE docs left behind by the removed push-based provisioning (#455) 2026-09-12 01:00:11 -07:00
Matthew Meszaros a4f503f620 fix: realtime cannot complete either of its TLS connections (#452)
* fix: let the realtime service actually establish its TLS connections, adding the wildcard hostname match fun Redix needs for every managed Redis and a CA bundle option for an RDS Postgres that the system trust store cannot verify

* feat: drop the redundant wget package from the realtime image since busybox already provides it, and fix a comment that read as literal interpolation

* fix: treat a blank DATABASE_SSL_CA_FILE as unset in the realtime config, because compose passes every optional variable through as empty and cacertfile with an empty path is no CA source at all rather than a fallback to the system store
2026-09-12 00:29:34 -07:00
Matthew Meszaros e8186e5f52 fix: the tracking service must not write Avro to a JSON consumer (#450)
* feat: make the Rust tracking publisher honour CODEC_PROVIDER on Kafka instead of always writing Avro, so a json consumer stops silently dropping every open and click, and refuse avro at boot when no Schema Registry is configured

* feat: build the tracking service's kafka feature in CI, because clippy on the default build never opens kafka.rs and that file now ships as the published tracking -kafka image

* feat: install libcurl and the rest of the librdkafka build dependencies for the tracking kafka clippy step, which fails at the first object without curl headers even with WITH_CURL=0
2026-09-11 23:00:15 -07:00
Matthew Meszaros 7d79dcc282 feat: append a -kafka image variant to every version the control plane hands a fleet node when the instance runs Kafka, so a joining worker pulls a build that can actually reach the bus instead of failing at boot (#449) 2026-09-11 22:46:02 -07:00
Matthew Meszaros 280a3e64ac feat: publish Kafka-linked image variants (#448)
* feat: publish Kafka-linked images for backend, consumer, worker and tracking as -kafka tag variants built per-arch on native runners, because the cgo librdkafka link cannot cross-compile on the build-go path

* feat: carry BUILT_AT inside the Go Kafka targets' build-args instead of the shared build-native block, so tracking and realtime stop warning about an unconsumed arg, and emit the matrix with printf because echo expands the separating backslash-n in some shells
2026-09-11 22:31:41 -07:00
Matthew Meszaros e3092f2322 feat: address the review by releasing the topic lock before the broker call so one slow creation cannot stall every publish, subscribe and close for the full admin timeout, refusing to open an admin connection once the bus is closed so a publish racing past the closed check cannot resurrect a client nothing will shut, building rather than vetting the tagged Kafka backend in CI because vet does not link and a CGO backend fails at link time, and correcting the Confluent tier wording to say auto topic creation is configurable only on Dedicated 2026-09-12 06:54:34 +02:00
Matthew Meszaros 6504d9958a feat: create Kafka topics from the bus that uses them, because a worker's command topic is named after the node id issued at join time so the set is not knowable in advance, and the broker's auto-creation is off by default on Confluent Cloud and not configurable below Standard, which left a worker subscribed to a topic that did not exist receiving nothing and reporting no error, and compile the tagged Kafka build in CI so a backend nothing else builds cannot rot unnoticed 2026-09-12 06:02:49 +02:00
Matthew Meszaros 131e9ff093 feat: give the JetStream stream a size ceiling from NATS_MAX_BYTES, accepting a byte count or a size like 2GiB, because a managed account can require every stream to declare one and Synadia's Max Bytes Required rejects creation without it, turning that refusal into an error naming the variable to set, and fix the retry path which passed a zero max age and would have recreated the stream with no age limit after a failure 2026-09-12 05:40:43 +02:00
Matthew Meszaros f1f5249153 feat: authenticate to NATS with a user JWT and nkey seed so a managed bus like Synadia Cloud can replace a self-run one, taking the credential from a file path for containers and from a single-line base64 value for the fleet, because a node receives environment variables rather than files and the env file docker reads cannot express the multi-line credentials format, in both the Go event bus and the Rust tracking publisher 2026-09-12 05:29:09 +02:00
Matthew Meszaros 80c3c79a31 feat: add live coverage that the machine windows survive the settings document's jsonb round trip and reach the classifier, a guard that every shipped scanner CIDR is written as its own network address since the loader truncates host bits silently, and correct the comments that claimed an edit lands on the very next event when the consumer reads through a thirty second cache in its own process 2026-09-11 03:11:35 -07:00
Matthew Meszaros c115b44d33 feat: correct the docs wording for the automated-engagement windows so the API reference and the campaigns guide name the dispatch-to-worker clock rather than the send, and describe the click window as independently configurable with thirty seconds as its default rather than as a fixed relationship to the open window 2026-09-11 03:02:58 -07:00
Matthew Meszaros 184a3dc08e feat: make the automated-open and automated-click windows operator-editable under Instance settings and raise their defaults to 60s and 30s, because the ten-second window was anchored on dispatch to the worker rather than on delivery and routinely expired before the recipient-side gateway it was meant to catch had even seen the message, and add Barracuda's published Email Gateway Defense blocks to the scanner catalogue with Proofpoint, Mimecast and Cisco shipped commented out because browser isolation renders a clicked page from the vendor's own network 2026-09-11 02:53:34 -07:00
Matthew Meszaros 170c33780a feat: pass ui_host to PostHog in the dashboard, admin panel and marketing site from its own environment variable so a proxied api_host stops breaking toolbar and session-replay links, which the SDK builds against whatever it sends events to and which a reverse proxy does not serve, defaulting to us.posthog.com so an install that does not proxy is unaffected 2026-09-11 11:34:02 +02:00
Matthew Meszaros 1866c45c67 feat: serve a PostHog reverse proxy at /ingest on the backend so the dashboard, admin panel and marketing site can report analytics and errors through this instance instead of posthog.com, which content blockers drop for a large share of visitors, splitting asset traffic to the bundle host because sending it to the ingestion host 404s, withholding the caller's cookies and Authorization from a third party, and preserving the trailing slash that path cleaning removes and PostHog's capture endpoint needs 2026-09-11 06:02:21 +02:00
Matthew Meszaros 853afe458e Merge pull request #430 from warmbly/fix/bootstrap-and-tracking-domain-check
fix: two aws-bootstrap defects, and a check for tracking on your own brand
2026-09-10 20:29:46 -07:00
Matthew Meszaros eb21d978a8 feat: stop aws-bootstrap from treating a bucket that exists in another region as done, since head-bucket answers globally and a second region silently kept its blobs in the first, stop it printing the database master password to stdout where a terminal, a CI log or an agent transcript keeps it forever, and add an instance check that reports a tracking domain sharing a registered domain with the product's own URLs 2026-09-11 05:24:57 +02:00