Commit Graph
149 Commits
Author SHA1 Message Date
Matthew Meszaros 1513419a2a feat: delete warmup mail from each mailbox once past a per-mailbox retention window (email_accounts.warmup_retention_days, else retention.warmup_mail_days, default 30) via a consumer sweep that retires the receipt and sender copy and a worker delete action that trashes on Gmail, deletes on Graph, expunges on IMAP and drops the stored body, prune per-message warmup records after retention.warmup_event_days, and count a warmup deletion as tampering only within 24 hours of arrival and never for a retired message, judging Gmail's Trash label on the same rule 2026-09-21 00:52:02 -07:00
Matthew Meszaros 26594391c8 feat: judge tampering with received warmup mail on a ladder inside the health bands, one deletion warns, two pause for seven days and four or two spam flags block for thirty, instead of a review-required block on the first deletion (#635) 2026-09-20 09:50:42 -07:00
Matthew Meszaros 48ecca2400 feat: follow the provider's read state in the unibox by storing seen from the \Seen flag on every IMAP, Gmail and Graph arrival, carrying read-state changes onto unibox_emails.seen in the FLAGS_ADD/FLAGS_REMOVE and UPDATE_EMAIL handlers, and backfilling existing rows from their flags in 000190 2026-09-20 06:54:29 -07:00
Matthew Meszaros addb956ad6 feat: shared TypeSafe client under internal/pkg/typesafe with inbox tagging phases 2 and 3 (hold, stop, task, suppress behind workspace switches, reversible ones on by default), labels seeded at workspace creation and by the follow-up sweep, premade inbox views (hot leads, needs a reply, follow up, declined, automated), typed reply classification and a reply_intent branch condition, an inbox agent draft gate, Advisor copy judgment with a cached editor re-check, warmup content lint, bounce cause classification that keeps a blocked address sendable, and per-form submission triage 2026-09-19 23:33:37 -07:00
Matthew Meszaros f9db4f435f feat: actually deactivate a mailbox whose credentials have died, by writing its status against the mailbox id instead of through the organization-scoped update that was being handed the owner's user id and so matched no row, and stop labelling a mail error's own uuid as an email account in the message 2026-09-19 20:23:17 +02:00
Matthew Meszaros acecd62c88 feat: scope mailbox disconnect and warmup lifecycle to the workspace rather than the member who connected the mailbox so an admin can act on every mailbox the list already shows them, evict a mailbox whose row is gone from every live worker when its provider errors arrive so a deleted mailbox stops calling the provider once a sync interval forever, subscribe before publishing the credential-validation job and classify a socket deadline as the retryable timeout it is, give the worker's validation reply its own budget so a slow mail host no longer loses a finished verdict, guard every global key handler against a keydown carrying no key, drop exceptions whose whole message is an object's default toString, make the Postgres pool size configurable, and record the CASA and security invariants in AGENTS.md 2026-09-19 12:49:46 +02:00
Matthew Meszaros e668a2a36b feat: complete the ADA CASA v2.1.1 AL1 control set across authentication, sessions, access control, cryptography, input validation and configuration, adding a breached-password denylist and per-account login throttling, enforced multi-factor authentication on the admin panel, step-up confirmation before an action that mints a lasting credential, purpose-scoped session tokens, single-use TOTP steps, tenant verification on every cross-referenced identifier, security headers on every surface, encrypted webhook signing secrets, per-organization idempotency, PKCE and a minimal two-scope Gmail consent on the mailbox OAuth flow, bounded spreadsheet and archive decoding, a patched Go toolchain with govulncheck in CI, and the evidence pack under compliance/casa 2026-09-19 08:18:35 +02:00
Matthew Meszaros d46cfad597 feat: check the warmup daily target again at the moment a send executes rather than only when the next one is placed, so a spam placement, health band or partner loss that cuts the target while a send is pending holds it as skipped_daily_limit and parks the chain at the next opening with a reply-back's aim intact, fail closed when that count cannot be read, share one target resolver between the placer and the send-time gate, add the skipped_org_suspended task status the suspended-workspace hold has written since #233 without a migration so its write stops failing and leaving the task pending for the dispatcher to re-fire, and anchor the ramp live fixture in UTC off the day boundary so its assertions no longer depend on the host timezone 2026-09-18 22:39:52 -07:00
Matthew Meszaros ffe3159256 feat: order the repair sweep's fake inbox by id so the cursor assertion is deterministic like the query it stands in for 2026-09-18 14:51:18 +02:00
Matthew Meszaros bd092dcf04 Merge remote-tracking branch 'origin/main' into fix/reply-attribution-and-human-opens
# Conflicts:
#	internal/app/consumer/event_new_email.go
2026-09-18 14:44:41 +02:00
Matthew Meszaros 6aebfe7e63 feat: store IMAP-synced addresses as Name <addr> like the Gmail and Graph syncs instead of Name (addr), teach mailhdr.Bare, the reply path's sender and recipient checks and the warmup sender fallback to read the old form for existing rows and older workers, so a reply into an IONOS or any other IMAP mailbox is attributed to its lead again after the address checks added on 16 September refused every one of them, and add a consumer sweep that re-offers unclaimed inbound mail answering a campaign send or coming from a contact to reply processing at boot and daily so the replies missed that week are attributed without anyone touching the database 2026-09-18 14:37:35 +02:00
Matthew Meszaros e737506ba0 feat: recognise a reply typed by hand in a warmup thread by the message it answers (In-Reply-To against warmup sends, receipts and earlier recognised turns, locally and through the pool link), keep it out of the unibox, file it out of the customer's Gmail, Outlook or IMAP inbox with the same folder action, record each recognised turn in warmup_thread_messages so the turn after it is recognised too, and let the daily sweep repair replies that already leaked 2026-09-18 14:12:33 +02:00
Matthew Meszaros cd964aafa8 feat: stop a deleted mailbox living on inside its worker, by returning the worker assignments a scheduled org or user deletion destroys so the erasure path can evict them the way the single-mailbox delete already does, and by treating an assignment lookup that finds no row as the mailbox being gone rather than a failed lookup, which tells every live worker to drop it; and clear the rest of error tracking by retrying a mailbox delete that loses a deadlock to its own cascade instead of failing the person who clicked Disconnect, answering a daily-usage read for a mailbox that no longer exists with 404 rather than a reported 500, and guarding the inner data field on three list reads plus serialising an empty pool-link list as [] rather than null 2026-09-18 12:42:43 +02:00
Matthew Meszaros a0a19edeae feat: register a schema document whose fixed defaults are code-point strings and whose nested nulls are null so the registered envelope parses back, keep the warmup unibox row until the filing action is on the bus and the mailbox lookup is not a transient failure, recheck the heartbeat key before evacuating a worker, match the IMAP namespace prefix case-insensitively, and state the BACKWARD direction correctly 2026-09-18 11:29:48 +02:00
Matthew Meszaros bd1837833e feat: give every derived Avro field its zero as a default and register the marshalled schema so adding a field cannot refuse the whole envelope and stop every publish, file historical warmup leaks out of the customer's own mailbox rather than only the unibox, and make a worker earn a 10-minute absence before its mailboxes are evacuated so a version rollout costs no migrations 2026-09-18 10:34:36 +02:00
Matthew Meszaros e37c5053c2 feat: make warmup refunds atomic, count confirmed partner diversity in local and cloud mailbox views, and document cloud-safe mailbox deletion 2026-09-17 21:15:28 -07:00
Matthew Meszaros 177a0817c4 Merge remote-tracking branch 'origin/main' into fix/closiqode-reported-issues 2026-09-17 21:00:48 -07:00
Matthew Meszaros 68babc30f3 feat: give the mailbox delete its own cloud revocation that calls the pool before dropping the local row and refuses an unreadable link, so a nil answer is proof the credential is gone rather than proof the local row went, and drive the greylisting evidence guard through the real handler with stub repositories so removing it fails CI where the live test skips 2026-09-17 20:52:54 -07:00
Matthew Meszaros 68c3676717 feat: keep warmup out of the customer's own mailbox and off their deliverability record: Gmail foldering now removes INBOX and SENT instead of only labelling, sent copies and reply-backs are filed in both directions, filing is configurable per mailbox (folder/inbox/archive via warmup_placement + warmup_folder, migration 000177), IMAP relocates a moved message by Message-ID so read/important stop no-opping, and a warmup send's bounce notice no longer lands in the unibox or suppresses a pool partner 2026-09-17 20:46:03 -07:00
Matthew Meszaros 0bcc24cac5 feat: say in the retryable-evidence test why the mirrored condition exists alongside the live one, since the live handler test needs a database and skips in CI 2026-09-17 20:22:33 -07:00
Matthew Meszaros d2095a8a70 feat: stop a greylisted RCPT reply from being filed as bounce evidence now that the send carries the server's own words, revoke a cloud enrollment after the worker removal and put the mailbox back when the revocation is refused so a failed delete really changes nothing, drop the unenroll-under-Settings advice that makes the same failing call, and only log a within-workspace pairing when there is a sibling to draw 2026-09-17 20:19:46 -07:00
Matthew Meszaros 8ee1c50a1b feat: make a failed SMTP send name the step and the cause behind it instead of one bare SERVER_UNREACHABLE sentinel, give a refused warmup send its day back so sent_today can no longer climb past the target while the cap frees the slot, revoke a mailbox's Warmbly Cloud enrollment when it is deleted so the pool stops holding its password, and prefer warmup partners outside the sender's own workspace while showing the partner diversity a mailbox is actually getting (#574, #575) 2026-09-17 20:02:37 -07:00
Matthew Meszaros a900f1e04c feat: make worker moves atomic and preserve safe concentration and health state 2026-09-17 07:45:23 -07:00
Matthew Meszaros c1e45b194a feat: merge latest main before worker capacity queueing 2026-09-17 06:26:07 -07:00
Matthew Meszaros d6025ea4c0 feat: address worker capacity review findings with safe migrations and recovery reporting 2026-09-17 06:24:14 -07:00
Matthew Meszaros b733d09f89 feat: make inbox follow-up labels safe for manual labels and automated mail 2026-09-17 04:57:52 -07:00
SUMAN JANA dff6b42558 feat(inbox): follow-up labels for who owes whom a reply, computed in code and swept hourly 2026-09-17 04:57:52 -07:00
Matthew Meszaros dcf26a2361 feat: make automatic inbox tagging atomic live and reviewable in production 2026-09-17 04:57:24 -07:00
SUMAN JANA a42b67f2e3 fix(inbox): tell the dashboard when a tag lands, so the label appears without a reload 2026-09-17 04:57:24 -07:00
SUMAN JANA 7e556f49e3 feat(inbox): backfill historical mail through the tagger, and keep a confident kind when the intent cannot be read 2026-09-17 04:57:24 -07:00
SUMAN JANA 260898bf20 feat(inbox): automatic tagging and relevance scoring for inbound mail, optional and off by default 2026-09-17 04:57:24 -07:00
Matthew Meszaros ae1a324801 Merge pull request #562 from rocker1166/feat/direct-mail-analytics
feat: add accurate direct-mail analytics and opt-in engagement tracking
2026-09-17 11:47:50 +00:00
Matthew Meszaros d1aa3d4361 feat: classify ambiguous Outlook and Apple image-proxy opens by delivery timing and stop inventing recipient device metadata 2026-09-17 04:27:34 -07:00
Matthew Meszaros 8745dd988d feat: correct direct-mail analytics attribution and publish live engagement updates 2026-09-17 04:27:00 -07:00
Matthew Meszaros bab9f86727 feat: correct worker capacity, mailbox distribution, observed IPv4, fleet pagination, and premium pool promotion 2026-09-17 04:15:05 -07:00
SUMAN JANA 2134c7a143 feat(analytics): report on mail written by hand, with opt-in open and click tracking per mailbox 2026-09-17 10:26:25 +00:00
Matthew Meszaros d3cd675d80 Merge pull request #554 from warmbly/fix/worker-disconnect-error-reporting
Fix ResizeObserver error reporting and cloud worker outage notifications
2026-09-16 15:51:53 +00:00
Matthew Meszaros cadf8f849a Merge pull request #553 from warmbly/fix/mail-sync-rate-limit-handling
Fix provider mail throttles without deactivating mailboxes
2026-09-16 15:51:27 +00:00
Matthew Meszaros 0334d2c64a feat: prevent ResizeObserver feedback warnings from reaching PostHog and scope worker outage alerts per workspace on shared cloud workers 2026-09-16 17:42:38 +02:00
Matthew Meszaros 817599d0eb feat: keep provider mail throttles retryable without deactivating mailboxes or flooding error tracking 2026-09-16 17:42:12 +02:00
Matthew Meszaros 2255e2145d feat: reject outbound mailbox copies and mismatched campaign threads before they can mark contacts replied for stop-on-reply (issue #549) 2026-09-16 08:24:19 -07:00
Matthew Meszaros 561f8ec671 feat: keep cloud and local warmup mail out of Unibox with durable verification and automatic cleanup 2026-09-16 03:55:36 -07:00
Matthew Meszaros 0a1ed6f04e feat: resolve scanner ASNs from a GeoLite2-ASN database so the catalogue's asn: entries match without a Cloudflare transform rule, ship the Proofpoint, Mimecast and Cisco ASNs enabled behind a new probable certainty that widens the consumer's machine window instead of deciding the verdict, make the tracking event dedupe claim one coalesced operation, and report an ASN database that opened cleanly but resolves nothing (#440) 2026-09-15 01:48:45 -07:00
Matthew Meszaros 95885fa714 fix: save a contact's edited email address (#511) (#521)
* fix: save a contact's edited email address by giving models.UpdateContact the Email field the dashboard was already sending, normalizing it to a bare lowercased address, refusing one another contact in the workspace holds with a 409 contact_email_taken instead of letting the unique index 500, and dropping the verification verdict plus the delivery evidence that belonged to the old mailbox (issue #511)

* fix: reset a contact's verification evidence behind a watermark the delivery-credit job honours, clear the cached esp_provider the old domain produced, write a case-only edit in place instead of answering 200 and changing nothing, and normalize the address on create and import too so the two paths cannot disagree about what an address is

* fix: refuse a verification observation whose campaign step provably left before the contact's address was edited, so a hard bounce or an open for the old mailbox arriving afterwards cannot mark the corrected address invalid or valid, read the credit watermark against dispatched_at rather than the sent_at a worker result stamps later, and answer a racing address collision with the documented 409 instead of the unique index's 500

* fix: build the deliverability evidence step from the resolved campaign task so both halves name one real row, drop the email format hint from the OpenAPI schema now that the endpoint also accepts a display-name address, and drain the shared delivery-credit backlog in the live test instead of assuming one pass reaches this contact
2026-09-14 23:05:16 -07:00
Matthew Meszaros 92e298b6ec fix: clear every open issue in error tracking by fixing the bugs behind them rather than the reports: a document-level mouseleave handing RippleProvider the document itself, whose classList is undefined; the admin panel posting /getaway without the /v1 its baseURL omits, so its realtime socket 404d on every page; Gmail throttles classified from the 403 status alone and told to re-authorize instead of back off; consumer flag and folder events retrying forever on a message the unibox never stored; a lost token-refresh race answered 500 instead of the documented 401; a nil email_accounts slice crashing the admin user page; Turnstile mounted with an empty sitekey; conditional passkey autofill run after the user navigated away; a boot log filed as an issue; and one publish failure per message on a topic the broker refuses (#519) 2026-09-14 21:06:14 -07:00
Matthew Meszaros d74d5e6836 fix: retire the warmup spam score, a counter that grew with volume rather than misbehaviour and that no band could act on (#508)
* fix: retire the warmup spam score, a ratchet that grew with volume rather than misbehaviour and that no band ever read, dropping the column from the pool row and the reputation ledger and explaining a pool finding with the band's own reason instead (#491)

* test: pin the advisor snapshot's pool columns against the scan, since the band's reason now reaches the finding through that select alone (#491)

* fix: hold a warmup sentence's score and reason with the sentence itself, keep the retired spam_score key on the published analytics payload as a deprecated zero, seed the sandbox with severity-shaped scores, and record the raw spam report when the warmup service is absent (#491)
2026-09-14 07:44:34 -07:00
Matthew Meszaros 6b6efca865 fix: campaign follow-ups opened a new conversation instead of replying in the contact's thread, so carry In-Reply-To/References and the Gmail threadId from the previous send, give every step a reply-in-thread switch, and let a threading step inherit the conversation's subject (issue #472) (#489) 2026-09-13 20:51:41 -07:00
Matthew Meszaros 1dc4aedc3c fix: retire the warmup invalid-token band with its table, metric query, service and repository methods and admin tab, since nothing has fed it since #481 and no attributable forged-token signal exists; key the live pool fixtures on the canonical pool ids so the warmup, repository routing and tasks routing suites run on a fresh database, and correct every doc, site and advisor line that still described the retired signal or a spam-score threshold nothing implements (#490) 2026-09-13 08:09:01 -07:00
Matthew Meszaros 8799680166 fix: never charge a mailbox for a warmup token that arrived in its inbox, hold a quarantine or block for its full term against fresh metrics, and keep a penalised address's standing across removal, pool exit and export through a trigger-maintained mirror, since the recipient never controlled the token, the bands read seven days against 30-day terms, and the pool row died on paths a snapshot at deletion never saw (#481) 2026-09-13 04:34:44 -07:00
joaoppa fe20326815 fix: stop the warmup tampering detector charging a mailbox for re-reading its own mail, since its Sent copy carries the recipient's token and a re-synced or reconnected mailbox presents tokens that no longer resolve, while keeping a token that names another pair as signal in any folder at any age (#468) 2026-09-12 22:20:42 -07:00