Commit Graph
129 Commits
Author SHA1 Message Date
Matthew Meszaros 2c156ec07e feat: hold the contact provider sweep lease by owner token with a compare-and-delete and end each run before it can expire, recognise Microsoft 365 tenant onmicrosoft.com domains for ESP matching, count checked domains with no known provider with other rather than undetected, and report an unknown-provider filter when saving it as a segment 2026-09-26 06:47:29 -07:00
Matthew Meszaros 8812cba439 feat: detect each contact's inbox provider from its domain's MX and SPF in a backend sweep, show it as a sortable Email provider column with the provider logo, filter and segment on it across contacts, campaign leads and segments, and use it for campaign ESP matching including Workspace and Microsoft 365 custom domains and the coverage panel's per-provider lead counts 2026-09-26 06:33:44 -07:00
Matthew Meszaros 7f324a38ac feat: open inbox placement tests to workspaces with probes rendered like the campaign send and paced as placement tasks, Message-ID matching instead of a subject token and warmup header, instance, workspace and Warmbly Cloud seed panels, a tracking comparison, scheduled campaign monitors with alerts and optional auto-pause, monthly allowances, realtime updates and org transfer registration 2026-09-25 20:48:15 -07:00
Matthew Meszaros 7f1bae8cac feat: default the OpenAI writing, agent and warmup content batch models to gpt-6-luna instead of gpt-4o-mini, gpt-4o and gpt-5-mini, and document the new preset default in the deployment guide 2026-09-25 20:18:26 -07:00
Matthew Meszaros cf83a13b6f Merge pull request #678 from warmbly/fix/vendor-authorizing-row-actions
feat: keep mailbox imports live and self-explaining, show vendor authorization progress with faster options, and finish sign-in rows through a later admin grant
2026-09-24 17:15:19 +00:00
Matthew Meszaros 256010725b feat: count only email steps in every sent total (contact drawer, Leads view and statuses, campaign progress, guardrails, org conduct, verification evidence, segments, admin, advisor), keep line breaks in synced body text and strip quoted history in any shape, classify delivery failures before out-of-office and tag them as bounces, record statusless failure notices from X-Failed-Recipients as permanent bounces, limit reply and inbox-tag opt-outs to people answering our outreach (never bounces, auto-replies or list mail), and recheck earlier reply opt-outs, lifting with an audit entry only those the message that wrote them no longer supports 2026-09-24 09:44:17 -07:00
Matthew Meszaros 4930c578df feat: tell the person watching a vendor authorization what to do when it waits, naming the Google Admin domain-wide delegation step with Warmbly's client ID and scopes when the vendor waits on it, and saying when the vendor has not moved a request for 20 minutes with the time and the request id to give its support 2026-09-24 18:14:10 +02:00
Matthew Meszaros 06f4a270cc feat: keep the mailbox imports menu live and self-explaining (deferred rather than dropped progress events, a five-second refresh while an import runs, each import's state and what it waits on in words, a badge for imports that need you, and an × that hides an import for the workspace through POST /emails/imports/:id/dismiss with migration 000211, stopping a running one first), show the vendor's own status and the Microsoft and Google time expectations on rows being authorized, record a vendor row's mail host so it gets its provider icon and a working Sign in, drop the Fix button from rows the vendor is authorizing, and count warming mailboxes rather than connected ones in the pool banner, refreshed when mailboxes change 2026-09-24 17:46:46 +02:00
Matthew Meszaros 549fb00be6 feat: keep mailbox credential checks and imports from timing out behind a worker's command queue by loading mailboxes off the bus loop (a republish never dials twice, commands wait for an in-flight load, a failed load raises the account's auth or server error), storing Kafka offsets for background commit instead of a synchronous commit per message, reconciling moved, unplaced and dead-worker mailboxes at once while spreading the safety-net republish over 30 minutes, sending checks over each worker's Redis channel with failover in placement order, retrying unanswered import rows within the lease, finishing a connect the browser left and an import row a restart interrupted, and connecting InboxKit Google and Microsoft mailboxes with no sign-in through a vendor-authorized grant that covers only the domains the vendor account lists 2026-09-24 11:44:58 +02:00
Matthew Meszaros 13b0ff85c3 feat: refuse an InboxKit key that reaches no workspace with mailbox_vendor_no_workspace, and have the per-workspace credential lookup move on only past a mailbox a workspace does not hold, reporting any other vendor failure instead of a missing mailbox 2026-09-23 21:21:21 -07:00
Matthew Meszaros e35baa7f58 feat: connect every inbox vendor with only its API key by discovering the workspaces and organizations the key reaches (InboxKit, Zapmail, Infraforge, ScaledMail), carry the workspace in vendor mailbox and domain ids with a lookup for ids stored before, skip a workspace the key may not read, refuse a key that reaches none with mailbox_vendor_no_workspace, and add a workspace switcher to the mailbox picker that filters, counts picks and selects a whole workspace 2026-09-23 21:12:09 -07:00
Matthew Meszaros e58921484d feat: rebuild mailbox import around column mapping and automatic host and sign-in detection (CSV, XLSX, pasted lists, saved mappings, retryable rows with fixes, migrations 000205-000206), connect whole Google Workspace domains and Microsoft 365 organizations through a proved administrator grant, import from inbox vendors (InboxKit, Zapmail, Mailforge, Infraforge, Maildoso, Cheap Inboxes, ScaledMail) with vendor-managed forwarding and DNS, add a sending domains page with per-domain tracking and verified root redirects, unify Add account into one Google and one Microsoft entry with per-method choices, mark per-mailbox Google sign-in as retiring with in-place moves to the admin grant or an app password, allow the loopback security mode in the credential columns (migration 000207), read semicolon-separated CSVs, and add a mock vendor API to the sandbox 2026-09-23 08:41:01 -07:00
Matthew Meszaros 331aeb4db3 Merge origin/main into feature/open-tracking-device-client 2026-09-23 02:06:04 -07:00
Matthew Meszaros 727ee432cb feat: keep a manual verdict set while a requested verification check runs, restore a lapsed evidence override from the row's own check status so a check landing mid-rescore wins, give member re-verify requests at most half of each verification batch while the backlog has work, and compare rescores against the stored verification status 2026-09-22 22:35:10 -07:00
Matthew Meszaros 19eb68ecd5 feat: re-read stored opens and clicks by the live origin rules in batched, resumable consumer passes under an advisory lock instead of a boot-time SQL backfill, never read a proxy string on a click as Apple Mail or Gmail, keep recognising Outlook, Proton Mail, Yahoo Mail and HEY by name, show the mail app behind a click in the expanded timeline row, and note that other iOS mail apps count as Apple Mail 2026-09-22 22:31:27 -07:00
Matthew Meszaros 72272cfbe7 feat: queue a contact re-verify ahead of the backlog whatever its evidence or manual verdict while the current verdict stands, count MillionVerifier renewing subscription credits, let a paid verifier's fresh answer outrank mail older than 30 days, keep what each check said, and show Verified with MillionVerifier plus a live Re-verify button on the contact Deliverability card 2026-09-22 22:17:04 -07:00
Matthew Meszaros d44fd38d90 feat: show the device and mail client behind every open and click (iPhone · Apple Mail app, Windows PC · Outlook app, Gmail · device hidden) from a new mailclient detector that recognises Gmail, Yahoo, Apple MPP, HEY, Fastmail and Seznam image proxies instead of reporting their fake browser and data-centre location, record app vs webmail and device_hidden on the open and click logs, backfill stored opens by the same rules, let the logs accept the scanner reason, and surface it on the contact timeline, a new How they read overview, the live campaign feed, recent activity, the campaign Device breakdown (surfaces), webhooks and realtime 2026-09-22 22:16:51 -07:00
Matthew Meszaros 4578980b34 feat: fall back to My Organization when the first name is blank in displayname.DefaultWorkspace, and only treat a scheme as a link when a non-space follows its colon so names like Big Data: EU pass in both the Go and web validators 2026-09-21 03:42:17 -07:00
Matthew Meszaros 9426c0da51 feat: validate every person, workspace and company name through internal/pkg/displayname on each write path (profile, onboarding, setup, IdP sign-in, org create and rename, org import, enterprise inquiry, admin testers, warmblyctl) with a 400 invalid_name code, render stored names in platform email through the same rules, mirror them in the web forms, check the org slug format, and document the rules in error-codes, security and AGENTS.md 2026-09-21 03:34:03 -07:00
Matthew Meszaros 6026168334 feat: stop blocking boot on the GeoIP download and swap the database in when it lands, retry a refused mirror with backoff honouring Retry-After, revalidate a database older than a week with If-Modified-Since instead of keeping the first copy forever, and add a twice-weekly job mirroring both MaxMind editions to a private bucket so the fleet stops spending a 30-a-day allowance per boot 2026-09-20 17:55:38 +02:00
Matthew Meszaros addb956ad6 feat: shared TypeSafe client under internal/pkg/typesafe with inbox tagging phases 2 and 3 (hold, stop, task, suppress behind workspace switches, reversible ones on by default), labels seeded at workspace creation and by the follow-up sweep, premade inbox views (hot leads, needs a reply, follow up, declined, automated), typed reply classification and a reply_intent branch condition, an inbox agent draft gate, Advisor copy judgment with a cached editor re-check, warmup content lint, bounce cause classification that keeps a blocked address sendable, and per-form submission triage 2026-09-19 23:33:37 -07:00
Matthew Meszaros e668a2a36b feat: complete the ADA CASA v2.1.1 AL1 control set across authentication, sessions, access control, cryptography, input validation and configuration, adding a breached-password denylist and per-account login throttling, enforced multi-factor authentication on the admin panel, step-up confirmation before an action that mints a lasting credential, purpose-scoped session tokens, single-use TOTP steps, tenant verification on every cross-referenced identifier, security headers on every surface, encrypted webhook signing secrets, per-organization idempotency, PKCE and a minimal two-scope Gmail consent on the mailbox OAuth flow, bounded spreadsheet and archive decoding, a patched Go toolchain with govulncheck in CI, and the evidence pack under compliance/casa 2026-09-19 08:18:35 +02:00
Matthew Meszaros 6aebfe7e63 feat: store IMAP-synced addresses as Name <addr> like the Gmail and Graph syncs instead of Name (addr), teach mailhdr.Bare, the reply path's sender and recipient checks and the warmup sender fallback to read the old form for existing rows and older workers, so a reply into an IONOS or any other IMAP mailbox is attributed to its lead again after the address checks added on 16 September refused every one of them, and add a consumer sweep that re-offers unclaimed inbound mail answering a campaign send or coming from a contact to reply processing at boot and daily so the replies missed that week are attributed without anyone touching the database 2026-09-18 14:37:35 +02:00
Matthew Meszaros 719d31b264 feat: cover Hostinger's hyphenated DKIM selectors in the dnsauth MX-hint tests so dropping hostingermail-a and hostingermail-b from providerSelectors fails the suite 2026-09-18 02:19:18 -07:00
joao-crm f6c7569615 fix: probe Hostinger's hyphenated DKIM selectors alongside the numbered pair, since hostingermail-a and hostingermail-b are what the provider actually publishes and hostingermail1 and hostingermail2 answer on no domain we have been able to test, leaving every Hostinger-hosted sender reported as unsigned 2026-09-18 03:09:49 +00:00
Matthew Meszaros bab9f86727 feat: correct worker capacity, mailbox distribution, observed IPv4, fleet pagination, and premium pool promotion 2026-09-17 04:15:05 -07:00
Suman Jana 94c7e414e2 feat: preserve private Unibox reply drafts and safely collapse quoted conversation history 2026-09-16 04:35:53 -07:00
Matthew Meszaros f72d1f8d5c feat: prevent sender views from tracking opens and keep sent messages out of the default Inbox (#542) 2026-09-16 03:42:58 -07:00
Matthew Meszaros 4784ee7d39 feat: fetch the MaxMind databases instead of requiring a mounted file (#529)
* feat: let the backend, consumer and tracking service fetch their own MaxMind databases from GEODB_URL and TRACKING_SCANNER_ASN_DB_URL, reading the archive shape from the content so a permalink tar.gz, a gzipped mmdb and a bare mmdb all work, never replacing a file already at the path, opening the bytes before installing them so a licence-key error page cannot become the database forever, skipping the AppleDouble sidecars a macOS tar writes ahead of the real file, and treating both URLs as secrets because the permalink carries the licence key

* feat: drop the trailing blank line cargo fmt --check rejects at the end of tracking/src/asndb.rs

* feat: stream the downloaded ASN archive instead of decompressing it whole, sizing each buffer from the gzip footer and the tar header so the member is allocated exactly once, which drops the peak of unwrapping a permalink tar.gz from 38 MB to 11.9 MB, essentially the database itself

* feat: stop the MaxMind licence key reaching the logs through net/http's and reqwest's own error text, which both print the URL they were given and so defeated the redaction beside them, drop userinfo as well as the query when redacting, refuse plain http for a URL carrying a credential and refuse an https-to-http redirect, and apply the size cap to the decoded database rather than the compressed transfer so a gzip bomb cannot fill the disk

* feat: strip basic-auth userinfo as well as the query when the tracking service redacts its database URL, parsing it rather than cutting at the first question mark so where a credential sits is the URL library's problem and not a guess
2026-09-15 03:06:34 -07:00
Matthew Meszaros e67b13e57e feat: stop reporting a mailbox's DKIM as missing when its selector was simply never probed, by deriving candidate selectors from the sending domain's own SPF and MX records on top of a wider default set, reporting a miss as the tri-state dkim_status undetermined rather than a red Missing row in the drawer, dropping DKIM from the Advisor's missing-records finding entirely, refusing a revoked p= key, holding the summary back from accusing anything when DNS never answered, and fixing the CLI auth-check table whose columns read mailbox fields the endpoint does not return (#528) 2026-09-15 02:27:38 -07:00
Matthew Meszaros 8d790ede6c feat: send from any address Google has verified a Gmail mailbox to send as and import the signature its owner already wrote in Gmail, reading both through gmail.settings.basic at connect and on demand via GET/POST /emails/:id/identity, validating the choice against the provider's own list in the service and again inside the UPDATE, clearing it when the provider stops verifying it, and never applying it to warmup (#514) 2026-09-14 10:13:36 -07:00
Matthew Meszaros a52a894110 feat: let the OpenAI provider adapt to a model that refuses function tools unless reasoning is off, by flipping a sticky reasoning_effort=none flag on the 400 that names it and widening the per-call compatibility retry budget to cover every flag, since gpt-5.6-luna rejects three parameters in a row and the old budget of two ended the call before the third adaptation (#513) 2026-09-14 08:20:08 -07:00
Matthew Meszaros 5ec367de8a fix: put the mailbox signature and the opt-out footer inside the container an HTML email was laid out in instead of after it, by locating that container with a new offset-keeping outline scan in internal/pkg/mailhtml and splicing into it, and centring the line on the card's own width when a builder export has no single container to sit in, so neither renders hard left in the page background any more (issue #462) (#505) 2026-09-14 08:11:52 -07:00
Matthew Meszaros c28f915648 feat: erase everything a disconnected mailbox leaves behind, revoking its OAuth grant at Google and deleting its stored message bodies through a durable retried queue, cascade the nine mailbox foreign keys that had none so warmup receipts, tampering events and provider message maps stop outliving the mailbox, clear thread labels and snoozes on conversations the delete emptied, make workspace deletion possible at all by cascading the four organization foreign keys with no delete action, and put Disconnect in the mailbox row menu and a Settings danger zone since it was only reachable from the selection bar (#506) 2026-09-14 07:55:01 -07:00
Matthew Meszaros 47defafa09 feat: fix the six self-host defects reported in issue #439 (#456)
* feat: fix the six defects reported in issue #439 by mapping the IMAP UNAVAILABLE, INUSE and NONEXISTENT response codes to retry-level errors instead of a critical reconnect prompt, synthesising a stable no-msgid key so one message with no Message-ID header can no longer 400 the internal map endpoint and wedge every later sync pass with its cursors held, adding mailhtml.FromText and HasContent so an API or agent-created step with a plain body stops shipping the composer's empty div placeholder as its text/html part (derived on create and plain-only update, exposed as body_html on update_campaign_step, dropped at send and preview time, and refused at campaign start with empty_step_body), honouring sender_strategy='explicit' in ResolveCampaignSenderPool and ValidateCampaignReady so an emptied explicit pool parks the campaign instead of widening it to every mailbox in the workspace, making the paused_no_accounts auto-pause loud with an error log line, an error-level activity-feed entry and an org-scoped CAMPAIGN_PAUSED realtime pulse, gating the admin sign-in's Turnstile widget on GET /v1/auth/config so a self-host with CAPTCHA_PROVIDER=none is not locked out, and parsing NATS_URL down to its host:port so a credentialed bus URL no longer reports NATS down

* feat: act on the self-review of the issue #439 fixes by dropping the campaign wizard's own escapeHtml body_html builder, which entity-escaped the quotes in a conditional and made the template fail to parse at send time, and letting the backend's FromText render that part instead so wizard-written steps also get their bare URLs linked for click tracking, correcting the docs and openapi description that claimed an explicit sender pool never falls back when it still unions its tags as migration 000013 designed, extracting the duplicated blank-HTML-part guard into dropBlankHTMLPart shared by the send path and the preview, and recording why the no-msgid key keeps the folder name despite a RENAME changing it

* feat: address the CodeRabbit review on the issue #439 fixes by holding the admin sign-in's Turnstile widget unmounted until /v1/auth/config resolves so an instance with no route to Cloudflare cannot raise a widget error on a screen nobody submitted, failing StartCampaign closed when the sequence read errors rather than skipping both the malformed-template and empty-body refusals, giving TCPCheck the default port its protocol assumes so a portless NATS_URL is no longer reported down, leaving a URL that carries a merge field unanchored because the send path renders bodies with text/template and a quoted contact value would break out of the href, and correcting the sequences guide and the Campaign and CampaignUpdate openapi descriptions that named the wrong tag field
2026-09-12 03:13:38 -07:00
Matthew Meszaros ea8d15374e Merge branch 'main' into feature/editor-image-links-and-buttons 2026-09-11 22:56:16 -07:00
Matthew Meszaros 6501cb599c feat: fix the "Edit with AI" rewrite in the campaign body and the unibox composer for issue #432 by running /generation/edit on a new generation.BuildEditRules system prompt through AIProvider.Complete instead of the cold-outreach writer prompt that redefined the model's role, capped it at 80 words and imposed a five-part email skeleton on every instruction, raising the completion cap so a full-body rewrite is no longer truncated at 1024 tokens and counting the request limits in runes rather than bytes, carrying merge variables, AI blocks, conditionals, form links and link destinations through the round trip in web/src/components/app/ai/richTextPassage.ts instead of deleting every atom node via doc.textBetween, replacing the passage with paste semantics so a phrase rewritten inside a sentence stops splitting its paragraph into three, saying "No change" when the model hands the passage back untouched, and clamping the floating AI card to the surface it is editing so it no longer draws outside the step drawer over the flow canvas 2026-09-11 20:04:32 -07:00
Matthew Meszaros 7ff9f4d2c6 feat: let a campaign body image carry a link and add a call-to-action button to the editor toolbar, both rendered as mail-safe markup (an anchor-wrapped img, a one-cell table whose cell holds the colour and padding for Outlook's Word engine), with the plain-text half now keeping every link destination 2026-09-11 20:01:16 -07:00
Matthew Meszaros 27c5df5072 feat: refuse a redirect that downgrades a verification request from https to http in both provider clients, since Go keeps the Authorization header across a redirect that stays on the same host and the pasted API key would go on the wire in the clear, and label the verification action from the connection id rather than the active provider, because a degraded connection reports the built-in checker while still being a connection to manage 2026-09-11 05:26:33 -07:00
Matthew Meszaros 4d0f0fb6b6 feat: hold an exhausted verification account that publishes no balance for a cooldown instead of re-deriving its health from an account check that cannot see exhaustion, since CleanMyList answers GET /v1/jobs identically whether or not there is allowance left, so the minute-long lookup cache retired every observed 402 and put the whole next batch back on doomed paid calls while Settings reported the service as healthy, and refuse a second verification connection while one is connected rather than letting creation order silently move every check onto a different bill 2026-09-11 02:57:08 -07:00
Chris Edington 59122e6c8a feat: add CleanMyList contact verification with API key setup and built-in fallback 2026-09-10 20:06:29 +01:00
Matthew Meszaros 4b93e849a1 feat: drop the occurrence struct left behind in the round-robin span emitter, which the map of per-term span lists replaced before it was ever used and which golangci-lint's unused check does not flag for an unreferenced type 2026-09-10 10:07:18 -07:00
Matthew Meszaros 77dfefcf88 feat: give every occurrence of a repeated trigger term its own span rather than only the first, since pointing at one 'free' out of three sends the writer back to hunt for the other two on the next re-check, and emit the spans round by round so every term shows once in every half before any shows twice, because the list is capped for display and a word written twenty times would otherwise fill it and hide the other terms that are also wrong 2026-09-10 10:01:57 -07:00
Matthew Meszaros d51de7db3a feat: address the CodeRabbit review by quoting a fragment in the copy's own casing rather than the model's retyping of it, extracting the case-fold offset map into internal/pkg/casefold so the AI half gets the same Unicode safety the rules half has, giving a trigger term a span in each half it appears in instead of losing the second one to deduplication, scanning subject links before body anchors so the display cap cannot drop the subject's own, requiring WRITE_TEMPLATES on the credit-spending analyze route so a read-only key cannot spend the workspace balance, refusing to tell a customer their credits came back when the refund is what failed, and no longer letting a stale analysis retire the newer rules request that was about to replace it 2026-09-10 09:50:28 -07:00
Matthew Meszaros e7b9491975 feat: drop an AI finding's category when the model returns one outside the closed set the API documents, so a client validating the response against that enum is never handed something outside it, and correct the parse comment that still described an unreadable response as falling back to the rules pass when Analyze now refuses it and refunds 2026-09-10 09:28:00 -07:00
Matthew Meszaros 427c9ce27c feat: pin the invariant the trigger-span offset map rests on, that foldIndex folds byte-for-byte identically to strings.ToLower and lands every recorded offset on a rune boundary, across Turkish dotted I, the Kelvin sign, ligatures, titlecase runes and invalid UTF-8 where a bad byte becomes a three-byte replacement rune 2026-09-10 09:28:00 -07:00
Matthew Meszaros f468f44e1d feat: let a content-check issue's field carry the location instead of repeating it in the message, so the launch dialog and campaign feed stop reading 'Body: 3 spam-trigger term(s) found in subject/body', and say 'Subject and body' for an issue whose fragments straddle both halves rather than dropping the location entirely, with the editor labelling each quoted word on such an issue 2026-09-10 09:28:00 -07:00
Matthew Meszaros ade18d1650 feat: leave an AI finding's field empty when the model labelled neither half and nothing in the finding could be anchored in the copy, instead of defaulting it to the body and rendering a badge that sends the writer to the wrong box on the one panel whose whole purpose is saying which box to open 2026-09-10 09:28:00 -07:00
Matthew Meszaros 3a9b1747dc feat: map a trigger term's offset back through the case fold before slicing the copy, so a rune that changes byte length when lowercased (U+0130, U+212A) no longer shifts every following span into bytes the writer never typed or cuts one in half into invalid UTF-8, read an issue's field from the whole span list before the display cap trims it so a subject-first list of more than eight can no longer relabel a both-halves issue, and refuse an AI response carrying a verdict with no score and no findings instead of pricing its empty finding list into a confident 100 out of 100 above a verdict saying the opposite 2026-09-10 09:28:00 -07:00
Matthew Meszaros 331db196d8 feat: locate every content-check issue in the subject or the body with the exact fragments that caused it and a one-line fix, add POST /templates/analyze running the configured LLM over a campaign template for located spam findings quoted verbatim from the copy plus a rewritten subject and an overall score, verify every model quote against the draft so an invented sentence is dropped rather than shown, pin the analysis temperature so re-checking unchanged copy returns the same number, and give the editor panel a Re-check button that re-runs both passes and reports the movement since the last check 2026-09-10 09:28:00 -07:00