Commit Graph
106 Commits
Author SHA1 Message Date
Matthew Meszaros 8883a1df6b feat: connect Microsoft 365 organizations through Microsoft's v2.0 admin consent page followed by a sign-in on the same state, read the approving admin's directory role from the Graph token as well as the ID token, return an organization-wide approval link for single-mailbox Outlook sign-in (admin_consent_url) and show it in the connect panel, and document publisher verification and the admin approval path 2026-10-03 20:17:38 +02:00
Matthew Meszaros a844c506a0 feat: make Warmbly Cloud warmup start and explain itself for linked mailboxes: apply the instance's warmup settings, start warmup on adopted mailboxes, read the warmup day mask Monday-first, report the last refused warmup send and the partner cap to the instance and dashboard, hand changed SMTP credentials to the cloud, refuse cleartext mailboxes at enrollment, and show cloud warmup with the warming animation and a clear Cloud marker 2026-10-01 03:01:36 -07:00
Matthew Meszaros 75f764dc67 feat: require a verified Cloud Tasks token with a pinned audience on task webhooks, manage_settings on integration OAuth and a fresh membership check at every mailbox and integration OAuth finish, user verification for passkey sign-in, ended sessions before a password reset or ban reports success, and a revoked session when a rotated refresh token is replayed; remove the internal DEK delete route, log credential path parameters by name, hold remote images in received mail until the reader loads them, add Calendly and Cal.com signing keys with inbound URL rotation, keep automation signing secrets out of connection responses, and apply the password rules to the bootstrap password 2026-09-30 06:46:24 -07:00
Matthew Meszaros a268e3c54a feat: limit the Warmup plan to the premium pool and uncapped mailboxes by gating sending, the inbox, AI, integrations, placement tests and paid storage on a plan that sends (HasProductPlan, with HasPremiumWarmup for pool placement), report 0 daily sends and refuse non-mailbox limit requests for workspaces that do not send, lock Warmup workspaces in the dashboard like free ones, show only the mailbox meter for them, and show the yearly price on yearly subscriptions 2026-09-28 09:16:42 -07:00
Matthew Meszaros 6108ef8255 feat: show each mailbox's own profile photo in its drawer header (read at a Microsoft connect, through Google and Microsoft admin grants, and from Zapmail and InboxKit listings, stored as a re-encoded JPEG under avatars/mailboxes with a weekly refresh, migration 000227 and workspace export support), bring back the classic Accounts mailbox cell as two lines with provider-logo avatars and vendor, grant or host chips, and replace the pulsing status and health dots with a text shimmer on live and at-risk states 2026-09-28 08:04:48 -07:00
Matthew Meszaros caf1852217 feat: keep placement seeds out of warmup pools as recipients too, store a probe's Message-ID before the send, hold queued probes against the sender's day and size a test to what is left, claim due monitors so one replica runs each, write comparison halves in one transaction, scan each seed's mail once per tick, keep test history when a mailbox is deleted, default to 20 seeds a minute apart, and render cloud-panel tests without a real lead by default 2026-09-25 21:46:09 -07:00
Matthew Meszaros 7f324a38ac feat: open inbox placement tests to workspaces with probes rendered like the campaign send and paced as placement tasks, Message-ID matching instead of a subject token and warmup header, instance, workspace and Warmbly Cloud seed panels, a tracking comparison, scheduled campaign monitors with alerts and optional auto-pause, monthly allowances, realtime updates and org transfer registration 2026-09-25 20:48:15 -07:00
Matthew Meszaros 7557d97866 feat: run a saved mailbox's placement, events, webhook and load on their own context so an expired caller deadline cannot skip them, bound bulk connects per row instead of per batch, have the worker reconciler remember the worker a load actually placed or moved a mailbox onto, keep an ADD_EMAIL that follows a REMOVE_EMAIL while the earlier load is still dialing by running it after that load, and show an import waiting only on vendor authorization as being authorized rather than importing 2026-09-24 11:58:49 +02:00
Matthew Meszaros 549fb00be6 feat: keep mailbox credential checks and imports from timing out behind a worker's command queue by loading mailboxes off the bus loop (a republish never dials twice, commands wait for an in-flight load, a failed load raises the account's auth or server error), storing Kafka offsets for background commit instead of a synchronous commit per message, reconciling moved, unplaced and dead-worker mailboxes at once while spreading the safety-net republish over 30 minutes, sending checks over each worker's Redis channel with failover in placement order, retrying unanswered import rows within the lease, finishing a connect the browser left and an import row a restart interrupted, and connecting InboxKit Google and Microsoft mailboxes with no sign-in through a vendor-authorized grant that covers only the domains the vendor account lists 2026-09-24 11:44:58 +02:00
Matthew Meszaros 0eb4e30fec feat: record a Microsoft 365 grant only when the consenting sign-in holds the Global Administrator or Privileged Role Administrator role, validate a tracking host before any vendor DNS write, clear Graph delta cursors when an Outlook mailbox moves onto a grant, keep a switched-off mailbox off when it moves unless its sign-in stopped it, treat pool-link mailboxes as managed, park a delegated mailbox without a grant as inactive, honour MAILVENDOR_SANDBOX_URL only when APP_ENV is dev, drop a vendor's cached domains with its key, retry a failed import redirect setup, keep address:password pastes whose password has a comma, and move the vendor import components to import/vendors so the Go vendor ignore rule no longer drops them 2026-09-23 09:03:25 -07:00
Matthew Meszaros e58921484d feat: rebuild mailbox import around column mapping and automatic host and sign-in detection (CSV, XLSX, pasted lists, saved mappings, retryable rows with fixes, migrations 000205-000206), connect whole Google Workspace domains and Microsoft 365 organizations through a proved administrator grant, import from inbox vendors (InboxKit, Zapmail, Mailforge, Infraforge, Maildoso, Cheap Inboxes, ScaledMail) with vendor-managed forwarding and DNS, add a sending domains page with per-domain tracking and verified root redirects, unify Add account into one Google and one Microsoft entry with per-method choices, mark per-mailbox Google sign-in as retiring with in-place moves to the admin grant or an app password, allow the loopback security mode in the credential columns (migration 000207), read semicolon-separated CSVs, and add a mock vendor API to the sandbox 2026-09-23 08:41:01 -07:00
Matthew Meszaros 1520da2fa8 feat: look up moved mail against each skipped folder with one SELECT per folder and cap the reconciliation by searches rather than rows, never settle a row whose lookup failed, refuse a skip name that is a saved folder the matcher keeps and purge only what it will stop following, drop the map entries of parked arrivals too, ignore UIDVALIDITY 0 when matching a rename into the skipped subtree, carry the listing memo and pending mark across a rename, ask before ticking a folder in the drawer since imported mail is removed, share one inbox-deleted publisher between the consumer's removal paths, and say in the CLI help that an emptied list does not restore removed mail 2026-09-22 05:45:38 -07:00
Matthew Meszaros ee9f5bf84e feat: make every skipped-folder purge drop the message map entries and parked arrivals with the rows so a message can be imported again if it moves back, persist each folder's delimiter (migration 000197) so the backend purge matches the same subfolders and case the worker skips, fail the mailbox load closed when the skip list cannot be read, mark a folder renamed into the skipped subtree as skipped, never remove a row re-fetched this pass, cap the reconciliation at 50 searches per pass and continue next pass, publish one EMAIL_DELETED after a folder purge, and resolve the account once for GET /emails/:id/sync 2026-09-22 05:33:56 -07:00
Matthew Meszaros 5a967cc3ce feat: let an IMAP mailbox exclude folders from sync (email_accounts.sync_skip_folders, migration 000196) so a folder another tool fills never reaches the unified inbox: the worker drops skipped folders and their subfolders before the walk, retires an already-synced one with its stored mail, and removes mail that later moves into one only when its Message-ID is found there; PUT /emails/:id/sync and the drawer's Sync card set the list, GET reports it with the server's folder list, warmbly mailbox skip-folders mirrors it, with docs, OpenAPI and error-code invalid_sync_folder 2026-09-22 05:15:49 -07:00
Matthew Meszaros c20d1c99f2 feat: race a 587 STARTTLS dial against a mailbox's silent port 465 on every send and connect check so the fleet keeps sending where outbound 465 is blocked, store a connect that passed that way with 587, name the port actually used in a refusal, make the Google app-password hint say Google itself refused the pair and name the alias and wrong-account causes, and render long error toasts wide, dismissable and longer-lived instead of a narrow four-second column 2026-09-20 13:16:52 +02:00
Matthew Meszaros 392bcc0478 feat: run the mailbox credential check off the worker's bus loop so it no longer waits behind queued sends and mailbox loads until the backend's fourteen-second wait expires, have the worker always answer with an error verdict when it cannot unseal the credentials so an untested mailbox is a server error rather than a mail-server timeout, name the leg that stayed silent and say whether the other one signed in with a 587 hint when 465 hangs, word the no-reply case as the worker not reporting back, dial both probes from WORKER_BIND_IP like the send and sync clients, and connect Gmail app-password mailboxes over 587 with STARTTLS because many hosts block outbound 465 2026-09-20 01:54:12 -07:00
Matthew Meszaros abe6157d37 feat: tighten the mailbox connect probes after a self-review, turning every socket error and not only a failed dial into closed words so a net.OpError naming the worker's bound address never reaches a customer, keeping a server reply's reason when the deadline passes while it is read instead of reporting a refused password as a timeout, connecting TCP before the TLS handshake on implicit-TLS SMTP so a port that answers and then fails the handshake reads as a TLS problem on both legs, sending EHLO explicitly because net/smtp's Extension swallows a failed greeting and reported a server that hung up as a mailbox with nothing to verify, and widening the backend's wait to fourteen seconds so the worker's seven-second budget plus the bus fits inside it 2026-09-20 00:21:31 -07:00
Matthew Meszaros c8162966a3 feat: tell a person connecting a mailbox what the mail server actually said instead of "invalid credentials" for everything, by having the worker's SMTP and IMAP probes classify a refused sign-in, an unreachable host, a failed TLS handshake, a retry-later reply and a timeout and publish that verdict as JSON ahead of the legacy digit, mapping it on the backend to mailbox_auth_refused, mailbox_unreachable, mailbox_tls_failed and mailbox_server_declined with the server's reply and a Gmail app-password hint in the message, starting the probe budget after the credentials are unsealed and bounding the SMTP conversation so a silent server no longer parks the worker, and normalizing passwords on every connect path so a Google app password pasted with its spaces works from the form, the CSV import, the API and the re-authorize dialog alike 2026-09-19 23:53:40 -07:00
Matthew Meszaros 834da184d9 feat: clear every dependency advisory that has an upstream fix, dropping the AWS SDK's legacy-rustls-ring default feature that was pulling a second hyper 0.14, rustls 0.21 and rustls-webpki 0.101 into the tracking service alongside the current ones, moving async-nats to 0.50 for the last old webpki and reqwest to 0.12, boxing the NATS producer variant the bigger client made oversized, refreshing the node trees with overrides for the esbuild and postcss-selector-parser that fumadocs pins, recording why the two unpatched cowlib advisories cannot be reached from a service that sets no cookie, and deciding the credential-validation timeout from the subscription context's deadline rather than the error's shape 2026-09-19 13:25:33 +02:00
Matthew Meszaros 2c886a3202 feat: decide the credential-validation timeout from the subscription context rather than the error's shape so a Redis socket timeout inside a live deadline stays an internal error, and require a synthetic mechanism and an empty stack as well as a default object string before dropping an exception so a message that stringified an object alongside real text is still reported 2026-09-19 13:00:23 +02:00
Matthew Meszaros acecd62c88 feat: scope mailbox disconnect and warmup lifecycle to the workspace rather than the member who connected the mailbox so an admin can act on every mailbox the list already shows them, evict a mailbox whose row is gone from every live worker when its provider errors arrive so a deleted mailbox stops calling the provider once a sync interval forever, subscribe before publishing the credential-validation job and classify a socket deadline as the retryable timeout it is, give the worker's validation reply its own budget so a slow mail host no longer loses a finished verdict, guard every global key handler against a keydown carrying no key, drop exceptions whose whole message is an object's default toString, make the Postgres pool size configurable, and record the CASA and security invariants in AGENTS.md 2026-09-19 12:49:46 +02:00
Matthew Meszaros b09ec39907 Merge pull request #599 from warmbly/chore/casa-al1-security-assessment
feat: complete the ADA CASA AL1 control set and ship the assessment evidence pack
2026-09-19 06:39:12 +00:00
Matthew Meszaros e668a2a36b feat: complete the ADA CASA v2.1.1 AL1 control set across authentication, sessions, access control, cryptography, input validation and configuration, adding a breached-password denylist and per-account login throttling, enforced multi-factor authentication on the admin panel, step-up confirmation before an action that mints a lasting credential, purpose-scoped session tokens, single-use TOTP steps, tenant verification on every cross-referenced identifier, security headers on every surface, encrypted webhook signing secrets, per-organization idempotency, PKCE and a minimal two-scope Gmail consent on the mailbox OAuth flow, bounded spreadsheet and archive decoding, a patched Go toolchain with govulncheck in CI, and the evidence pack under compliance/casa 2026-09-19 08:18:35 +02:00
Matthew Meszaros 6428e6b3e9 Merge pull request #594 from warmbly/feature/disable-google-oauth-new-mailboxes
feat: route new Gmail mailboxes through a guided app-password connect instead of Google sign-in, behind BOX_GOOGLE_OAUTH_CONNECT, leaving existing OAuth mailboxes sending and re-authorizable
2026-09-19 06:11:15 +00:00
Matthew Meszaros 49acd51b64 feat: stop one recurring fault burying error tracking by reporting it once per five minutes with the count it stands for, keep a cache outage from answering every signed-in request with a 500 and from taking realtime down by treating an unreachable Redis as a miss and the websocket handshake nonce nothing reads as best-effort, answer a 5xx with a sentence the reader can act on while the call site's own words go to the log against the same request id, prefer the API's own message over the HTTP class in the admin and dashboard clients, and name the fix on a schema registry refusal, an SES sandbox rejection and a mailbox check that could not be run 2026-09-19 07:39:40 +02:00
Matthew Meszaros ee46cb49e8 feat: route new Gmail and Google Workspace mailboxes through a guided three-step app-password connect over smtp.gmail.com and imap.gmail.com instead of Google sign-in, behind BOX_GOOGLE_OAUTH_CONNECT (off by default) and announced to clients as gmail_oauth_connect on /auth/config, refusing a new gmail OAuth start with 403 mailbox_gmail_oauth_disabled in both the direct and Warmbly Cloud broker paths while leaving mailboxes already connected that way sending, syncing and re-authorizable 2026-09-18 22:06:09 -07:00
Matthew Meszaros 4b2b641f92 feat: pass the workspace id to the mailbox delete in the database-backed removal and erasure tests, which compiled with the owner's id but would answer not found now that Delete is scoped to the organization, and rename the cloud link stub's parameter to say what it carries 2026-09-19 06:10:32 +02:00
Matthew Meszaros f99ee57484 feat: scope mailbox disconnect to the workspace instead of the connecting member, so a teammate with manage_emails no longer gets 404 on a mailbox the list shows them, delete by id in the repository on the strength of that check while the worker removal still names the owner the consumer's unibox cleanup is keyed on, and read the API's own reason off the normalised AppError in the accounts page so a refused disconnect says why instead of "The mailbox couldn't be disconnected" on every failure 2026-09-19 06:01:03 +02:00
tunglambk cf7e530e15 feat: release a cloud-managed mirror's cloud link when it is deleted through the mailbox delete, using the delete-only revocation that never calls back into the mailbox delete, so the mailbox returns to the cloud workspace instead of staying claimed by an instance that no longer holds it (issue #582) 2026-09-18 07:17:40 +00:00
Matthew Meszaros 8240f14e8b feat: fail closed on incomplete cloud mailbox revocation and document retry-safe deletion and TLS diagnostics 2026-09-17 21:21:34 -07:00
Matthew Meszaros e37c5053c2 feat: make warmup refunds atomic, count confirmed partner diversity in local and cloud mailbox views, and document cloud-safe mailbox deletion 2026-09-17 21:15:28 -07:00
Matthew Meszaros 68babc30f3 feat: give the mailbox delete its own cloud revocation that calls the pool before dropping the local row and refuses an unreadable link, so a nil answer is proof the credential is gone rather than proof the local row went, and drive the greylisting evidence guard through the real handler with stub repositories so removing it fails CI where the live test skips 2026-09-17 20:52:54 -07:00
Matthew Meszaros d2095a8a70 feat: stop a greylisted RCPT reply from being filed as bounce evidence now that the send carries the server's own words, revoke a cloud enrollment after the worker removal and put the mailbox back when the revocation is refused so a failed delete really changes nothing, drop the unenroll-under-Settings advice that makes the same failing call, and only log a within-workspace pairing when there is a sibling to draw 2026-09-17 20:19:46 -07:00
Matthew Meszaros 8ee1c50a1b feat: make a failed SMTP send name the step and the cause behind it instead of one bare SERVER_UNREACHABLE sentinel, give a refused warmup send its day back so sent_today can no longer climb past the target while the cap frees the slot, revoke a mailbox's Warmbly Cloud enrollment when it is deleted so the pool stops holding its password, and prefer warmup partners outside the sender's own workspace while showing the partner diversity a mailbox is actually getting (#574, #575) 2026-09-17 20:02:37 -07:00
SUMAN JANA 2134c7a143 feat(analytics): report on mail written by hand, with opt-in open and click tracking per mailbox 2026-09-17 10:26:25 +00:00
Matthew Meszaros 619eb2729a fix: read a mailbox's Gmail send-as addresses from the worker holding it rather than from the backend, which was decrypting a mailbox credential in the control plane and showing Google a second client address for a mailbox whose mail moves through a worker, by round-tripping a new MAILBOX_IDENTITY command answered on the process channel like a credential validation, leaving the OAuth handshake as the one place the control plane still calls the provider (#522) 2026-09-14 21:28:07 -07:00
Caner 5700aef8e1 fix: accept a live refresh token as proof of the offline_access grant when checking Microsoft consent, because the v2.0 token response echoes only the scopes the access token is valid for and never lists offline_access, which was turning working Outlook mailboxes away at onboarding (#500) 2026-09-14 20:56:06 -07:00
Matthew Meszaros 8d790ede6c feat: send from any address Google has verified a Gmail mailbox to send as and import the signature its owner already wrote in Gmail, reading both through gmail.settings.basic at connect and on demand via GET/POST /emails/:id/identity, validating the choice against the provider's own list in the service and again inside the UPDATE, clearing it when the provider stops verifying it, and never applying it to warmup (#514) 2026-09-14 10:13:36 -07:00
Matthew Meszaros c28f915648 feat: erase everything a disconnected mailbox leaves behind, revoking its OAuth grant at Google and deleting its stored message bodies through a durable retried queue, cascade the nine mailbox foreign keys that had none so warmup receipts, tampering events and provider message maps stop outliving the mailbox, clear thread labels and snoozes on conversations the delete emptied, make workspace deletion possible at all by cascading the four organization foreign keys with no delete action, and put Disconnect in the mailbox row menu and a Settings danger zone since it was only reachable from the selection bar (#506) 2026-09-14 07:55:01 -07:00
Matthew Meszaros 1f7e036659 fix: mailbox onboarding accepts connections that cannot work, and hides why (#465)
* fix: report why a provider refused to name a mailbox during onboarding, since the status and body were read and then discarded and five distinct failures arrived as one opaque sentence with nothing in the logs or error tracking behind it

* fix: refuse a mailbox the provider only partly authorised, since a consent screen lets the person untick individual permissions and still issues a token, and refuse a first connect that returns no refresh token because that mailbox has an hour to live

* fix: never record a provider's successful profile payload in diagnostics because a decode failure carries status 200 and that body is the mailbox owner's address, and bound the provider read so a broken intermediary cannot make onboarding read arbitrary memory
2026-09-12 08:07:20 -07:00
Matthew Meszaros d456bc48c6 feat: fix the Warmbly Cloud pool link across both roles (#262): take an enrolled mailbox out of this instance's own warmup pool so local partners stop writing to it and their unverifiable warmup stops landing in the owner's unibox, recognise the cloud's warmup mail whose verify header did not survive delivery through a new warmup-deliveries lookup that ignores consumed_at because instance and cloud read the same mailbox, move the managed-mailbox access token route behind NODE_BROKER_TOKEN so the internet-facing tracking and forms services can no longer mint a live provider token, scope pause and resume to the caller's workspace, keep an enrolled mailbox listed once it goes inactive, release the cloud copy when the local mirror row cannot be written, refuse the one-time handshake when CREDENTIALS_ENCRYPTION_KEY is missing, blank an expired code's plaintext instance token, and stop errx answering 200 for a status outside its table 2026-09-12 06:58:25 -07:00
Matthew Meszaros c4aece241b feat: scope the tag, category and folder registries and unibox conversation labels to the organization instead of the creating user, so a teammate sees and can edit the labels the owner made, splitting a label two workspaces shared into one copy each and guarding every label write against ids from another workspace (#457) 2026-09-12 03:37:31 -07:00
Matthew Meszaros 6dcbc6d6a7 feat: stop sending prompt=consent on the Outlook/Microsoft 365 authorize request, which made Entra ID re-run the consent eligibility check per sign-in and refuse every non-admin with AADSTS90095 even under a tenant-wide admin grant (issue #409), and ask for prompt=select_account instead so the account picker survives while Google keeps access_type=offline plus forced consent for its refresh token 2026-09-10 03:24:16 -07:00
Matthew Meszaros 6ebf9cfdcf Merge remote-tracking branch 'origin/main' into fix/self-hosted-unsubscribe-domain 2026-09-09 08:24:04 -07:00
Matthew Meszaros 7d58b874b8 feat: keep every recipient-facing and self-host-facing address on the deployment's own domain: mint unsubscribe links on a workspace's verified tracking domain (served by the tracking service, proxied to the backend that owns the pages), attach RFC 8058 one-click only over https, resolve all branding through config.Brand() gated on SelfHosted() so a self-host's email footer, sign-in links, stats card, API example and public form badge name nobody else, drop the app.warmbly.com fallback from AppBaseURL, blank TRACKING_DOMAIN and FORMS_DOMAIN on core-only installs, and have install.sh offer to configure a fresh interactive install instead of silently defaulting to localhost 2026-09-09 06:34:43 -07:00
Matthew Meszaros bfdbd77b04 feat: fix the eight defects the second review pass found, including three where the previous fix did not land: the bind-mounted state dir was root-owned so the node running as uid 1000 still could not write its update target, elevating the reserved-worker eviction did nothing because the rotation loop bailed on target-equals-current before the urgency was consulted, and the warmup-pool assertion was vacuous which hid that warmupPoolFor checked the subscription repo before the billing provider and answered free on a self-host install 2026-09-09 05:32:30 -07:00
Matthew Meszaros 435dbb522f feat: replace the worker tier/type/risk-pool/egress categories with a scored placement model and make the fleet pull-based, so a machine joins with one command, workers and consumers share one node registry with usage and liveness, nodes self-update to the version the control plane resolves, and the Hetzner provisioning, worker profiles and SSH orchestrator are removed 2026-09-09 04:54:01 -07:00
Matthew Meszaros 766bd3ae6a feat: let a self-hosted instance connect a mail server on its own machine by adding a third mailbox security mode, "none", accepted only for a loopback literal and only where the worker shares a host with the relay, so Proton Bridge on 127.0.0.1:1143/1025 and a local Dovecot or Mailpit can be connected at all, with the rule enforced in onboarding and reauth validation, again by the worker against the peer it actually dialled rather than the name it was given, and hidden from the connect form on the hosted product where the worker is never the customer's machine 2026-09-07 08:07:40 -07:00
Matthew Meszaros 51dedc90ee feat: put every runtime behind one optional error-reporting story: a single internal/observability/errs wrapper that is now the only package importing sentry-go, InitSentry for cmd/forms, release and environment tags on every service from the existing build stamp, optional Sentry in the admin panel and the public forms app, the sentry crate in the Rust tracking service, release tagging in realtime, CI source-map upload that only runs when a Sentry token is configured, and docs covering the DSN for each service 2026-09-07 03:51:06 -07:00
Matthew Meszaros bf46839fb7 feat: address the CodeRabbit review on the mailbox allowance work by reserving the mailbox slot inside the account insert transaction under a per-organization lock so concurrent connects cannot both take the last one, resolving the storage limit under the quota lock so a plan change cannot be raced past, deleting refused attachment objects on a context that outlives the request, settling already-connected rows before the bulk batch spends allowance, leaving password columns out of the failed-rows CSV, stopping click propagation from the portalled allowance dialog, and counting bulk progress once 2026-09-04 21:11:25 -07:00