Commit Graph
73 Commits
Author SHA1 Message Date
Matthew Meszaros addb956ad6 feat: shared TypeSafe client under internal/pkg/typesafe with inbox tagging phases 2 and 3 (hold, stop, task, suppress behind workspace switches, reversible ones on by default), labels seeded at workspace creation and by the follow-up sweep, premade inbox views (hot leads, needs a reply, follow up, declined, automated), typed reply classification and a reply_intent branch condition, an inbox agent draft gate, Advisor copy judgment with a cached editor re-check, warmup content lint, bounce cause classification that keeps a blocked address sendable, and per-form submission triage 2026-09-19 23:33:37 -07:00
Matthew Meszaros 03f59d4d6f docs: state the tenant and key-shape invariants in these comments as the constraints they are, rather than as an account of what each check replaced, since this repository is public and every self-hosted instance that has not updated yet reads the same text 2026-09-19 08:28:41 +02:00
Matthew Meszaros e668a2a36b feat: complete the ADA CASA v2.1.1 AL1 control set across authentication, sessions, access control, cryptography, input validation and configuration, adding a breached-password denylist and per-account login throttling, enforced multi-factor authentication on the admin panel, step-up confirmation before an action that mints a lasting credential, purpose-scoped session tokens, single-use TOTP steps, tenant verification on every cross-referenced identifier, security headers on every surface, encrypted webhook signing secrets, per-organization idempotency, PKCE and a minimal two-scope Gmail consent on the mailbox OAuth flow, bounded spreadsheet and archive decoding, a patched Go toolchain with govulncheck in CI, and the evidence pack under compliance/casa 2026-09-19 08:18:35 +02:00
Matthew Meszaros 6aebfe7e63 feat: store IMAP-synced addresses as Name <addr> like the Gmail and Graph syncs instead of Name (addr), teach mailhdr.Bare, the reply path's sender and recipient checks and the warmup sender fallback to read the old form for existing rows and older workers, so a reply into an IONOS or any other IMAP mailbox is attributed to its lead again after the address checks added on 16 September refused every one of them, and add a consumer sweep that re-offers unclaimed inbound mail answering a campaign send or coming from a contact to reply processing at boot and daily so the replies missed that week are attributed without anyone touching the database 2026-09-18 14:37:35 +02:00
Matthew Meszaros 81d46bdcc8 feat: attribute a campaign reply by the thread it answers rather than requiring the From address to equal the contact's, so a person replying from a Gmail send-as alias, a forward or a colleague's desk counts as replied for that lead, stamp replied_at whether or not reply-intent automation is switched on, suppress the mailed address too when an alias reply opts out, and count only a person's opens in every open count and open rate (campaign overview, per step, daily, hourly, dashboard, recent activity) with machine opens shown as a separate not-counted figure, matching how the Leads tab already reads opened 2026-09-18 14:29:16 +02:00
Matthew Meszaros 68c3676717 feat: keep warmup out of the customer's own mailbox and off their deliverability record: Gmail foldering now removes INBOX and SENT instead of only labelling, sent copies and reply-backs are filed in both directions, filing is configurable per mailbox (folder/inbox/archive via warmup_placement + warmup_folder, migration 000177), IMAP relocates a moved message by Message-ID so read/important stop no-opping, and a warmup send's bounce notice no longer lands in the unibox or suppresses a pool partner 2026-09-17 20:46:03 -07:00
Matthew Meszaros 62e346c8e0 feat: restore cross-mailbox campaign reply tracking while preserving sent-folder safeguards 2026-09-17 06:03:56 -07:00
Matthew Meszaros 5855d7e32d feat: fully fence inbound reply processing and repair issue 549 state across campaigns and verification 2026-09-16 22:03:46 -07:00
Matthew Meszaros 35b94252e6 feat: make inbound reply processing idempotent across duplicate events 2026-09-16 21:38:10 -07:00
Matthew Meszaros c91e89980f feat: make reply recording atomic and require exact repair recipient matches 2026-09-16 21:25:43 -07:00
Matthew Meszaros b01984bec2 feat: verify reply direction from persisted mailbox state and repair issue 549 false Sent-folder replies 2026-09-16 21:11:54 -07:00
Matthew Meszaros 2255e2145d feat: reject outbound mailbox copies and mismatched campaign threads before they can mark contacts replied for stop-on-reply (issue #549) 2026-09-16 08:24:19 -07:00
Matthew Meszaros dd98187231 fix: shorten recipient unsubscribe links to 22-character, 128-bit stored tickets (#498) (#525)
* feat: shorten every recipient unsubscribe link from a 96-character signed token to a 22-character stored ticket carrying 128 bits from crypto/rand, minted once per recipient per campaign and reused by every step, so the address the text/plain half of a cold email prints in full fits on one line and cannot be guessed, keeping the signed form working for links already in inboxes and as the fallback when the store cannot be written, and answering a failed lookup with a retryable 'try again shortly' instead of telling the recipient their opt-out is invalid (issue #498)

* fix: restore the disabled-signer guard in URLOn, which factoring the URL builder moved behind a token mint that dereferences the signing key, so a nil or origin-less signer returns the empty string every caller reads as 'no link can be minted' instead of panicking (PR #525 review)
2026-09-15 00:42:45 -07:00
Matthew Meszaros 95885fa714 fix: save a contact's edited email address (#511) (#521)
* fix: save a contact's edited email address by giving models.UpdateContact the Email field the dashboard was already sending, normalizing it to a bare lowercased address, refusing one another contact in the workspace holds with a 409 contact_email_taken instead of letting the unique index 500, and dropping the verification verdict plus the delivery evidence that belonged to the old mailbox (issue #511)

* fix: reset a contact's verification evidence behind a watermark the delivery-credit job honours, clear the cached esp_provider the old domain produced, write a case-only edit in place instead of answering 200 and changing nothing, and normalize the address on create and import too so the two paths cannot disagree about what an address is

* fix: refuse a verification observation whose campaign step provably left before the contact's address was edited, so a hard bounce or an open for the old mailbox arriving afterwards cannot mark the corrected address invalid or valid, read the credit watermark against dispatched_at rather than the sent_at a worker result stamps later, and answer a racing address collision with the documented 409 instead of the unique index's 500

* fix: build the deliverability evidence step from the resolved campaign task so both halves name one real row, drop the email format hint from the OpenAPI schema now that the endpoint also accepts a display-name address, and drain the shared delivery-credit backlog in the live test instead of assuming one pass reaches this contact
2026-09-14 23:05:16 -07:00
Matthew Meszaros e18f2efc05 feat: hold an out-of-office contact in every campaign they are a lead of instead of only the one the auto-reply was attributed to, through a new HoldLeadEverywhere applying the same per-row guard so a member's own pause and a longer running hold both survive, and leaving completed campaigns alone (issue #518) 2026-09-14 21:54:05 -07:00
Matthew Meszaros ffd27bc46d fix: stop every out-of-office notice and bounce opening a high-priority CRM follow-up by classifying machine replies from their headers and gating the task on a per-intent setting, and give the Tasks page multi-select with select-all-matching, bulk status, priority and delete over new PATCH and DELETE /crm/tasks endpoints (issue #471) 2026-09-14 12:20:47 -07:00
Matthew Meszaros 13e9ce8e10 feat: hold a lead whose mailbox answers out of office until they are back, resuming at the return date it names, plus a manual per-contact pause in one campaign that unsubscribing and the suppression list were the only stand-ins for 2026-09-14 09:26:06 -07:00
Matthew Meszaros 6b6efca865 fix: campaign follow-ups opened a new conversation instead of replying in the contact's thread, so carry In-Reply-To/References and the Gmail threadId from the previous send, give every step a reply-in-thread switch, and let a threading step inherit the conversation's subject (issue #472) (#489) 2026-09-13 20:51:41 -07:00
Matthew Meszaros c4aece241b feat: scope the tag, category and folder registries and unibox conversation labels to the organization instead of the creating user, so a teammate sees and can edit the labels the owner made, splitting a label two workspaces shared into one copy each and guarding every label write against ids from another workspace (#457) 2026-09-12 03:37:31 -07:00
Matthew Meszaros dc9ce403de feat: stop one un-sendable lead parking a whole campaign and stop the contact drawer's next-action time walking forward on every refresh (issue #437): route up to config.CampaignPlacementCandidates due leads per pass instead of one, classify a placement refusal that belongs to a single lead (ESP-strict finding no mailbox for that recipient's provider, a bound lead inside its own mailbox's minimum gap or waiting for it to reopen, a recipient's send-time-optimized hours) as the new ErrLeadDeferred so the pass moves to the lead behind them and only defers the campaign when every candidate is refused, log the ESP-strict deferral once a day rather than once per refused lead per tick, and make PreviewContactSend a pure read that answers unchanged state identically on every call by running placement with the even-distribution, jitter, conflict-resolution, distribution-curve and sub-minute layers off, taking a behaviour profile's gap at its floor instead of drawing it, picking the mailbox deterministically instead of re-rolling rotation, reporting the next sending day's first open minute instead of a jittered twenty-four-hours-from-now, and reporting a due step's time as the campaign chain's own stored wakeup 2026-09-12 02:58:48 -07:00
Matthew Meszaros 331db196d8 feat: locate every content-check issue in the subject or the body with the exact fragments that caused it and a one-line fix, add POST /templates/analyze running the configured LLM over a campaign template for located spam findings quoted verbatim from the copy plus a rewritten subject and an overall score, verify every model quote against the draft so an invented sentence is dropped rather than shown, pin the analysis temperature so re-checking unchanged copy returns the same number, and give the editor panel a Re-check button that re-runs both passes and reports the movement since the last check 2026-09-10 09:28:00 -07:00
Matthew Meszaros 426ea45013 feat: full HTML and CSS support for campaign bodies and mailbox signatures, adding a send-time CSS inliner and a structure-aware plain-text renderer in internal/pkg/mailhtml, persisting HTML mode on a step through the previously unused sequences.body_code so a designed email is no longer gutted by the editor schema on reopen, widening the TipTap schema to hold tables, div containers, colours, fonts, sizes and alignment with toolbar controls for each, folding alignment into a single style producer because TipTap merges two style attributes by splitting on the first colon and truncated background-image url(https://...) to url("https"), replacing the regex plain-text stripper that put a whole stylesheet into the text/plain part and cost a content score eight points for a .free-trial-banner class, locating </body> past Outlook conditional comments so the signature and opt-out footer stop landing inside one, forcing the signature source view for markup a contentEditable cannot host safely, and reporting per-client compatibility findings from the preview endpoint 2026-09-09 06:34:47 -07:00
Matthew Meszaros a5802b53cf feat: address the review on the unsubscribe-link change by closing the tag scanner over quoted attributes in both the send path and the editor, so a > inside an attribute can no longer split a tag and rewrite the href that follows it into a dead link, and by scanning only the copy a plain-text step actually ships, resolving the campaign's inherited opt-out mode before warning, checking the subject too, and reporting an unreadable step list as a failed check rather than an empty scan 2026-09-07 06:34:45 -07:00
Matthew Meszaros b25897fe67 feat: render a hand-placed {{.UnsubscribeLink}} as a real anchor labelled with the workspace's unsubscribe link text instead of shipping the bare signed API URL in the body, let the composer turn a text selection into that link so the wording stays the author's, and warn in the step editor and at preflight when a plain-text-only campaign carries a body opt-out link that has nowhere to hide its address 2026-09-07 06:34:45 -07:00
Matthew Meszaros 5d60fe9a21 feat: fix campaign restart dead end (issue #340): resolve preflight tracking senders through the same explicit/tags/all pool the scheduler uses so a connected mailbox is never reported missing, fetch the launch dialog's step count from the steps endpoint instead of a field the campaign API never returns, turn on Keep running for new leads when a form or an automation feeds a campaign (migration 000131 backfills existing ones) and when a member starts a campaign whose every lead has finished so it goes active and waits for leads instead of answering 400, return waiting_for_leads from the start endpoint for the dialog's success screen, and document the no_leads and no_remaining_leads codes and the new behaviour in the campaigns, forms, automations and API docs 2026-09-07 02:26:50 -07:00
Matthew Meszaros 85f3dad215 feat: send the files a step actually carries by honouring campaign_attachments.sequence_id in the send, test-send and preview paths instead of attaching every campaign file to every step, count them per step in the preflight content score, and refuse an upload naming another campaign's step 2026-09-04 06:16:16 -07:00
Matthew Meszaros 64f2d1637b feat: address the review on the unsubscribe PR: register the suppression family in warmblyctl, require the confirm field on the browser unsubscribe POST and cap its body, render a chosen A/B variant through the template engine so its merge fields and unsubscribe link resolve, fold curly apostrophes before opt-out phrase matching, write pasted suppression lists in one transaction, clamp copy by runes instead of bytes, add the constraints NOT VALID plus a lower(email) index in migration 000123, scope the unsubscribe link type-ahead to email bodies, and document DELETE /suppressions/:id 2026-09-03 02:20:47 -07:00
Matthew Meszaros 5eb92c601c feat: give every campaign email a working opt-out: a reply-to-opt-out line by default or an unsubscribe link (workspace setting under Settings > Sending with a per-campaign override and a {{.UnsubscribeLink}} variable), signed per-recipient unsubscribe links served on the API origin so the List-Unsubscribe header no longer points at a dead warmbly.com page, a confirm page on GET with RFC 8058 one-click on POST and a resubscribe button, reply opt-out detection through the whole-word compliance lexicon with quoted history stripped, a first-class suppression list (Contacts tab, GET/POST/DELETE /suppressions with address and domain entries, audited removal, contact drawer action), the contact Subscribed flag enforced in campaign routing, migration 000122 with a shared recipient_suppressed() predicate, and docs for all of it 2026-09-03 01:52:40 -07:00
Matthew Meszaros 7a7c6051bc Merge remote-tracking branch 'origin/main' into feature/contact-segments 2026-08-30 02:07:56 -07:00
Matthew Meszaros 52916ab60d feat: verification evidence engine and Greptile fixes for #264: contact_verification_evidence ledger (migration 000111) fed by clean deliveries, human opens, clicks, replies and recipient-naming bounces from the send, tracking, reply and bounce paths, a decaying score that lets real mail outrank a probe and a newer bounce outrank older engagement, verification_confidence on every contact with a reasons list and animated Deliverability card in the contact drawer, per-organization probe breakers, undeliverable counts that ignore finished leads, tighter wiring comments, and docs 2026-08-29 23:49:33 -07:00
Matthew Meszaros daf946fc45 feat: add the add_to_segment and remove_from_segment sequence action steps, executed on both the scheduled campaign path and the instant reply path, wired into the backend and consumer 2026-08-29 23:45:05 -07:00
Matthew Meszaros ec7b7bbed3 feat: count anchors and bare URLs together instead of taking the larger of the two, so three labeled links plus three written-out URLs no longer score a clean 100, clear the editor's pending flag when the draft is emptied mid-request since the cancelled request's finally can no longer do it, fail the preflight content check when a campaign's attachments cannot be read rather than scoring as if there were none and passing copy the send path then warns about, and condense the new comments to the one-line form the repo convention asks for 2026-08-28 21:59:53 -07:00
Matthew Meszaros 4ddcb24bcd feat: make the campaign content check actually fire, fixing a link cap that counted zero links on every HTML email because the URLs live in href attributes that tag-stripping discards, a preflight that scored wait and action nodes as copy and so failed every campaign using one at 55/100 because GetSequencesByCampaignID never selected the kind column, a content_warning log level of "warning" that never matched the dashboard's "warn" amber tier, an unvalidated min_content_score an API caller could set to 5000, attachments the send path scored but preflight ignored, and a docs claim of live editor scoring that was really a manual button 2026-08-28 21:54:29 -07:00
Matthew Meszaros 4b5686111e feat: refuse a launch whose list is known to be largely undeliverable (#235)
* feat: refuse a launch whose list is known to be largely undeliverable

* feat: count the deliverable audience in SQL instead of subtracting two overlapping totals, surface the unverified advice, and stop a query error skipping the gate silently: a contact can be both suppressed and unsubscribed, so Total minus both removed it twice and inflated every share computed against the remainder, which for a fully overlapping list produced a negative deliverable clamped to zero; the unverified-list branch set a summary and remediation but neither Block nor Warn, and a preflight report only surfaces checks that did not pass, so that advice could never be displayed; and a failed audience query was treated as a pass, which is fail-open on a safety gate without even a log line saying the check did not run
2026-08-28 11:39:53 -07:00
Matthew Meszaros 1b8c51ad06 feat: read spam complaints and domain-auth refusals, the last two open delivery signals (#232)
* feat: close the two halves of the delivery-signal loop that were still open, complaints and domain-auth refusals: a spam complaint arrives as mail long after the send succeeded, and nothing read those reports, so the strongest negative signal a sender gets never reached the complaint rate, the suppression list or the breaker; internal/pkg/arf parses RFC 5965 feedback reports worker-side alongside the existing DSN path, takes the LAST Message-ID because the reported mail's headers follow the report's own, and records only abuse-type reports so a not-spam report cannot be inverted into a complaint; separately a receiving server refusing mail because the SENDING DOMAIN failed its authentication (5.7.515, 5.7.26) was classified as SERVER_UNREACHABLE and retried forever, and it is now its own hard code that blames the domain rather than the address, leaves the recipient unsuppressed, and brings that domain's DNS re-check forward so the sweep confirms the verdict the send gate acts on

* feat: make the complaint path actually reachable, and stop it trusting the report: selectTextParts only ever picked text/plain and text/html, so an RFC 5965 report's message/feedback-report and message/rfc822 parts never reached the worker and both Feedback-Type and the reported Message-ID were invisible, which would have left this feature inert and has been quietly weakening DSN parsing too; report parts are now selected as plain text; the complainer is taken from the RESOLVED SEND rather than the report body, because a report is unauthenticated mail anyone able to reach the mailbox could forge and honouring the address it names would let a forger suppress a contact the send never went to; Original-Mail-From is no longer read as the complainer since that address is the sender; and a domain-auth refusal now releases the reservation instead of spending one of the lead's attempts, because the recipient received nothing and the problem is the mailbox's domain, so another mailbox in the pool picks the lead up

* feat: fix the same missing-parts gap in the Gmail adapter, and say plainly that Microsoft Graph cannot see reports at all: goog.extractBody took only text/plain and text/html exactly as the IMAP path did, so a feedback report synced from Gmail was as invisible as one synced over IMAP, and the tests that would have caught either called the parser directly rather than going through the adapter where it actually broke; the new tests exercise that seam, and Graph returns one rendered body with no parts so reports there are undetectable without a MIME fetch that is not built, which the docs now state rather than implying full coverage
2026-08-28 11:02:45 -07:00
Matthew Meszaros ca5acb5aaf feat: score campaign copy as sent, not just as written (#229)
* feat: score campaign copy where it actually matters, on the message the recipient receives rather than only the template in the editor: warmlint.Score already ran at save time through /templates/score, but nothing checked the copy after merge fields, spintax, A/B selection and AI blocks resolved, which is exactly where a clean template becomes Hi comma or picks the one spammy spintax branch; the send path now scores the rendered message and writes one campaign-feed warning per step per day rather than one per recipient, RunPreflight gains the same check across every step, warmlint gains image-heavy, many-images and attachment heuristics, and the whole thing is advisory behind Settings > Sending > Content checks; the launch dialog also finally calls POST /campaigns/:id/preflight, whose report had no UI at all and so never reached the person about to send

* feat: fix three defects the review found in the content lint: a preflight run that could not read the campaign's steps left the worst score at 100 and reported the check as PASSED, which is the one outcome a check that did not run must never produce, so it now reports failed with the reason; the send-time path read organization settings while preflight read campaign-effective ones, so a campaign that turned the check off or moved its floor was still warned, and EffectiveSettings is now exposed and used by both; and the check-then-insert dedupe let concurrent recipients of the same low-scoring step each find nothing and each write, so it becomes one advisory-locked conditional insert proven by twelve racers producing exactly one feed entry
2026-08-28 10:29:39 -07:00
Matthew Meszaros 222c9d2554 feat: scope campaign sender resolution to the campaign's organization instead of its owner so a multi-org user can no longer send organization A's campaign from an organization B mailbox: GetByTags/GetAllActiveInScope/GetByCampaignSenders now take a repository.AccountScope keyed on organization_id where a scope with no organization resolves to no mailboxes rather than widening to the owner (tags carry no organization of their own, so one user's tag legitimately spans workspaces and the predicate is the only thing holding the boundary), the campaign scheduler and the preflight tracking-domain check build that scope from campaign.OrganizationID, unibox compose scores only the current workspace's mailboxes, the 'all' lanes of AccountHasActiveCampaign/CountActiveCampaignsForAccount join ea.organization_id = c.organization_id instead of ea.user_id = c.user_id, dead and broken PauseAllByUserID (which wrote the reason string into status and had no callers) is removed, the campaigns/unibox guides now state that senders resolve inside the campaign's workspace, and TestLiveOrglessCampaignDoesNotSendToSuppressedRecipient is updated because an orgless campaign now finds no senders before routing is consulted while the send gate it covers still refuses; live-tested in TestLiveSenderResolutionStaysInsideTheCampaignOrg, TestLiveSenderSchedulerNeverPicksAnotherOrgMailbox, TestLiveSenderSchedulerPicksTheCampaignOrgMailbox, TestLiveSenderScopeWithoutAnOrganizationReachesNothing and TestLiveActiveCampaignLookupIsOrgScoped 2026-08-24 09:20:41 -07:00
Matthew Meszaros cbf4190f42 feat: make a custom tracking domain verifiable instead of permanently "Pending DNS": the CNAME target is now this install's TRACKING_DOMAIN rather than a hardcoded t.warmbly.com that resolves nowhere, matching is exact on the label boundary (or on shared addresses, so a provider-flattened CNAME stops reading as no record at all) instead of strings.Contains, and every outcome carries the reason plus what DNS actually returned, including when the tracking host the customer is told to point at has no record of its own; a pasted URL is normalized to its host and a malformed one is rejected up front instead of saved and left pending forever; only a VERIFIED mailbox domain is used at send time with the shared host as the fallback and a campaign-feed entry saying why; pixels and click tickets are built from the configured host, and with none configured mail ships untracked rather than carrying links to another deployment's tracking service; adds GET /emails/:id/track and POST /emails/:id/track/verify plus an hourly re-resolution sweep so a record that propagates later starts being used and one that breaks stops routing links; and scopes the tracking-domain write by organization like the read, which also fixes GET /emails/:id passing a user id to an org-scoped query and 404ing for every caller 2026-08-24 09:02:29 -07:00
Matthew Meszaros 94cf21d95e Fix truncated, unformatted and mis-encoded email content in the unibox (#137)
* feat: add internal/pkg/mailhtml, a mail-oriented HTML sanitizer and text flattener, because rendering a received message body means rendering the sender's markup: Sanitize builds on bluemonday's UGC policy but keeps what real email is made of (table layout attributes, inline CSS through the property-allowlisted style sanitizer, legacy font/center, data: and https: images) while dropping script, iframe, object and the text content of style/head blocks so a marketing email's stylesheet cannot render as body copy, forcing target=_blank plus nofollow/noreferrer on links and allowing only http, https, mailto and tel; ToText flattens the same input for previews, turning block boundaries into newlines and decoding entities back to the characters they stand for so an already-escaped body does not surface as literal &amp;; LooksLikeHTML reports whether a stored body is actually markup, which is how a body recorded as HTML by an older sync but containing no tag at all can be recognised as the plain text it really is

* feat: add internal/pkg/mailhdr for RFC 5322 header values, since headers are ASCII on the wire and every transport was writing raw UTF-8 into Subject and display names: Subject and AddressList RFC 2047-encode non-ASCII (a no-op on plain ASCII, and a bare address stays bare rather than being wrapped in angle brackets), DecodeWords reverses encoded-words with a charset hook wired to go-message so legacy encodings Go does not handle natively still decode, and Bare/BareList strip a display name down to the routable address for SMTP envelope commands where 'Ana <a@b.com>' in RCPT TO is a syntax error, promoting go-message from an indirect to a direct dependency

* feat: encode outbound Subject and address headers on all three transports, so a subject or sender name containing an accent, a currency sign or an emoji reaches the recipient as the characters the user typed instead of mojibake: SMTP and Graph were writing the raw string into Subject (only the Gmail transport encoded it) and Graph built its From by fmt.Sprintf rather than mail.Address, so a non-ASCII display name went out unencoded there too, and all three joined To/Cc/Bcc entries verbatim so an encoded display name never appeared even when the caller supplied one; additionally the SMTP envelope now takes bare addresses through mailhdr.BareList, because an API caller may pass 'Name <addr>' (the compose handler has a bareAddress helper precisely because that arrives) and passing that to RCPT TO gets the recipient rejected by the server

* feat: rewrite the IMAP body reader, which was the reason received mail from SMTP/IMAP mailboxes came back corrupted: it built one FetchItemBodySection with a hardcoded Part []int{1} and a comment saying it would adjust when recursing, which it never did, so on a multipart/alternative the text/plain bytes were fetched twice and the second copy was stored as the HTML body (plain text rendered as markup loses every line break, shows & as an entity and swallows anything inside angle brackets), and decodeIfNeeded never reversed Content-Transfer-Encoding at all, leaving quoted-printable bodies full of =E2=80=99 runs and = soft breaks and base64 bodies unreadable, while its charset detection parsed params off a media-type string that never carried any and its mail.ReadMessage call could silently eat leading body lines as headers; the reader now walks the body structure for real part paths, fetches every text leaf in a single FETCH with a server-side Partial size cap, decodes quoted-printable and base64 (tolerating a tail cut mid-quantum by the cap) then converts the part's charset to UTF-8 with go-message, skips attachment-disposition parts so a .txt attachment cannot stand in for the body, takes one part per type inside a multipart/alternative but treats sibling inline parts in mixed/related as additive, and is bounded at five text parts per message; the stored body cap also goes from 200 KB to 512 KB because 200 KB cuts real HTML newsletters mid-document

* feat: decode Gmail's raw headers and entity-escaped snippets, because the Gmail API hands header values back exactly as they arrived on the wire, so a message from a sender whose subject or display name was RFC 2047-encoded showed in the dashboard as =?utf-8?q?caf=C3=A9?= rather than as the text it stands for, and the API's own snippet field is HTML-escaped, so a preview containing an apostrophe surfaced as &#39; in the conversation list and, until the thread reader stopped rendering snippets as message bodies, inside the message itself; getSingleHeader now runs values through mailhdr.DecodeWords (a no-op unless the value actually contains an encoded-word, so Message-ID and the warmup token header are untouched), the comma-split fallback in getAddressList does the same for display names net/mail could not parse, and the snippet is unescaped once on the way in

* feat: fix the conversation-list snippet, which collapsed whitespace before splitting on newlines so the quoted-line and signature filters below it could never match a thing, stripped HTML with bluemonday's strict policy and then showed the escaped output verbatim so an ampersand in an HTML-only message read as &amp; and a marketing email's stylesheet text rendered as body copy, and cut at 100 bytes with text[:100] so a multi-byte character or emoji at the boundary became a replacement glyph; it now flattens HTML through mailhtml.ToText (entities decoded, style and script content dropped) including when a sender puts markup in their text/plain part, filters quoted history and everything past the RFC 3676 signature delimiter while the text still has lines, collapses whitespace afterwards, and truncates on a rune boundary at 200 characters

* feat: make GET /unibox/:id serve a display-safe body and stop it failing outright, sanitizing body_html through mailhtml before it leaves the API so every consumer gets markup that cannot execute rather than each call site having to defend itself, degrading a body blob that cannot be read to the message's preview text with a new body_truncated flag instead of returning 500 (which made a message with a missing blob unopenable, and hit every seed, sandbox and dev-history fixture row since only the '<seed-' prefix was recognised while the sandbox uses '<sbx-' and dev history '<dev-'), and treating a stored HTML body that contains no tag at all as the plain text it really is, because mail synced before the IMAP reader addressed parts individually recorded the plain part under both bodies and serving that as HTML is exactly what collapsed a ten-line message onto one line

* feat: escape composer text before turning it into the HTML part of an outgoing email, replacing body_html: trimmedBody.replace(/\n/g, '<br />') in both the compose window and the reply composer with a shared plainToHtml that escapes the five markup characters first, so an email containing 'Terms & conditions' no longer ships a broken entity and one containing anything in angle brackets ('<see attached>', 'a < b', a pasted tag) no longer has the rest of the paragraph swallowed by the recipient's mail client as an unclosed tag, while runs of spaces survive as non-breaking spaces and bare URLs become links without eating the sentence punctuation after them; the same unescaped plain-to-HTML pattern in the campaign step editor's applyTemplate now goes through promptToHtml, which escapes as it paragraph-wraps

* feat: render the real message body in the unibox thread reader instead of the list preview, which is the whole of the reported bug: ThreadView mapped each thread row to a UniboxEmail whose body was '<p>' + escapeHtml(m.snippet) + '</p>' and MessageBubble rendered that as the message, but a snippet is a preview capped at 100 characters with every run of whitespace collapsed to one space, so a ten-line email displayed as roughly two lines on a single continuous line, and Gmail's already-escaped snippet was escaped a second time so an apostrophe read as &#39;; each expanded message now loads its own body from GET /unibox/:id (the newest message and anything unread open on mount, older messages collapse to their preview line so a long thread does not fetch every body at once) and renders it in a sandboxed iframe carrying no allow-scripts, which keeps a sender's stylesheet from restyling the dashboard and means nothing in the message can run even though the API already sanitized it, sizing itself from the inner document as images load, with the preview kept as the fallback when a body cannot be fetched and a notice when only a preview is stored

* feat: document how a message body is read and returned, adding a 'Reading a message' section to the unibox guide covering the expand-on-open behaviour, that formatting and special characters are preserved as sent, that the conversation list preview is a summary and not the message, and that HTML mail renders in an isolated frame with links opening in a new tab, plus a paragraph in the API endpoint reference stating that GET /unibox and GET /unibox/thread return previews carrying snippet while GET /unibox/:id returns body_plain and a sanitized body_html, and what body_truncated means

* feat: add email_accounts.save_to_sent, the per-mailbox switch for filing a copy of outbound mail in the Sent folder, defaulting on because plain SMTP submission leaves nothing behind in the sender's account while Gmail and Outlook file their own copy through their APIs, making it a per-mailbox choice rather than a global one since a submission server that files the copy itself (Gmail's SMTP, Fastmail, Zoho) would otherwise end up with two of everything, which is exactly why every desktop mail client ships the same switch, and wiring the column through the Email model, the mailbox read paths and UpdateEmail so it is readable and writable from the dashboard and the API

* feat: teach the IMAP client to APPEND a sent message and the SMTP client to hand back the exact bytes it submitted, the two transport pieces the Sent-folder copy needs: AppendToSent resolves the folder from the RFC 6154 \\Sent special-use attribute first (requesting it only when the server advertises SPECIAL-USE) and falls back to matching the known names against both the full mailbox name and its leaf, since servers namespace as INBOX.Sent and localize the label, caches the result for the life of the connection, files the message flagged \\Seen and dated when it was sent, and returns a sentinel rather than an error when the account has no Sent folder at all; APPEND addresses its mailbox by argument and never touches the selected mailbox, so unlike the warmup MOVE/STORE actions it is safe to run while the sync loop is mid-fetch on the same connection

* feat: file a copy of every SMTP send in the mailbox's Sent folder, closing the gap where a message sent from Warmbly through an SMTP/IMAP mailbox existed only in the recipient's inbox: nothing appeared in the customer's own mail client, and nothing appeared in the unibox either, whose thread reader can only show messages the sync found in a folder, so a user who sent from the dashboard and then went looking for what they sent found no record of it at all; the worker now APPENDs the exact bytes the SMTP client submitted after a successful send, best effort so a failed append never turns a delivered message into a failed task, skipping warmup traffic because filing dozens of machine-generated messages a day would bury the customer's real sent mail, and skipping Gmail and Graph mailboxes entirely since their APIs file their own copy; the per-mailbox setting rides along on the add-email worker payload as a pointer so an older control plane that does not send the field is read as unset and takes the default rather than as an explicit no

* feat: expose the Sent folder copy as a mailbox setting in the dashboard, adding a 'Keep a copy of sent mail' toggle to the Settings tab of the mailbox drawer that only renders for SMTP/IMAP mailboxes (Gmail and Outlook file their own copy, so the control would be a lie there), tracked by the drawer's save bar alongside the other editable fields, and worded so the one case where it should be turned off is obvious: a provider that already saves its own copy, where leaving it on means seeing every sent message twice

* feat: document the Sent folder copy in the mailboxes guide and the API reference, explaining why the toggle exists at all (SMTP submission leaves nothing in the sender's own account, so without it a sent message shows in neither the customer's mail client nor the unibox thread), when to turn it off (a provider such as Gmail, Fastmail or Zoho that already files its own copy of anything submitted over SMTP, where leaving it on doubles every message), that OAuth Gmail and Outlook mailboxes never show the control because their APIs file the copy themselves, that warmup traffic is deliberately excluded, and that PATCH /emails/:id takes save_to_sent

* feat: add unibox_emails.body_text and its search index, because unibox search ran against search_tsv, a generated column built from subject and snippet, and a snippet is a truncated one-line preview, so searching for a phrase that appears in the third paragraph of an email returned nothing at all and read as broken search rather than as search that only covers the first line; message bodies stay in object storage where they belong, and what lands in Postgres is a bounded 16 KB plain-text rendering carried on the new-email worker event, indexed with a GIN expression index rather than a second stored generated column since adding one of those rewrites the whole table while this builds against a column that is empty on every existing row

* feat: index what a message actually says, adding mailhtml.SearchText (HTML flattened, entities decoded, whitespace collapsed, quoted history deliberately kept because a phrase someone quoted back at you should still find the conversation, truncated on a rune boundary) and computing it on all three sync paths so IMAP, Gmail and Graph mail all arrive with searchable text, writing it on insert, and widening the unibox search filter to match either the existing subject-and-preview vector or the body expression, written exactly as the new index declares it so the index is actually used

* feat: backfill the searchable text of messages that were synced before bodies were indexed, so search covers the archive a customer already has instead of only mail that arrives from now on, which would have made the feature useless on day one for exactly the people who need it; the sweep pages through unibox_emails by id, reads each body from object storage under the mailbox owner's key, renders it with the same helper the sync path uses and writes it back, at 100 rows per 30 seconds because nothing waits on it, and returns for good once a pass finds nothing left to visit, with rows whose stored body really is empty simply revisited after the next restart rather than needing a tried-and-failed marker in the schema

* feat: document that unibox search now covers message text and not just subjects and previews, in the search paragraph of the unibox guide where the old wording only promised that search stays inside the current scope

* feat: add generation.RenderThread, the shared way to put a conversation in front of a model, because every AI surface was grounding on preview snippets and a draft written from the first hundred characters of each email answers the greeting rather than the question; it strips quoted history and signatures (the earlier messages are already in the prompt on their own, so quoting them again spends the budget twice, though a reply written underneath the quote is kept rather than thrown away when there is nothing meaningful above the attribution line), spends a bounded character budget newest-message-first since the message being replied to matters most, degrades older messages to their preview line instead of dropping them once the budget runs low, and renders oldest-first so the transcript reads in order

* feat: add grounding reads to the unibox service and repository, returning message text (the stored body, falling back to the preview for mail synced before bodies were indexed) for a thread or for all correspondence with one address, kept deliberately separate from the preview queries and given their own result type so a 16 KB body can never leak into a list response by accident, capped at twenty messages whatever a caller asks for, and paired with a RenderGrounding helper so every AI surface formats a conversation the same way instead of each one rolling its own transcript loop

* feat: ground every AI writing surface in what the messages actually say, switching the unibox reply draft, the compose draft's correspondence history, the inbox agent's thread history and the assistant's read-thread tool from preview snippets to real message text through the new grounding reads, which is what makes a drafted reply answer the question that was asked rather than the first sentence of the email; the inbox agent's triviality gate also reads the reply's full text now, since a preview line cannot tell a one-word ack apart from a long message that happens to open with one, and the assistant tool returns a bounded body per message with quoted history stripped instead of a snippet field

* feat: say in the docs that AI drafting reads the messages and not their previews, in both the unibox reply-draft section (adding that quoted history is stripped and the newest messages get the most room, so a draft answers what was asked rather than the opening sentence) and the inbox agent's grounding section, where 'the full thread so far' was true of the message list but not of how much of each message the model actually saw

* feat: renumber the two new migrations to 000087 and 000088 after rebasing onto main, which landed its own 000085 (org data transfer) and 000086 (email sync state) in the meantime, so the sequence has no duplicate versions

* feat: add the two new API fields to the OpenAPI spec that landed on main while this branch was open, documenting save_to_sent on the Mailbox and MailboxUpdate schemas and body_truncated on UniboxEmail, and saying on body_html that what the API returns is already sanitized so a client can render it directly
2026-08-21 18:45:09 +02:00
Matthew Meszaros 66aaaa6584 feat: add an ai agent step to campaign sequences 2026-07-21 17:35:11 +02:00
Matthew Meszaros 93b64e2afa feat: add shared run time listing and arg resolving for ai agents 2026-07-21 17:35:11 +02:00
Matthew Meszaros e8de52ce30 refactor: put the task client behind a scheduler interface 2026-07-20 09:56:16 +02:00
Matthew Meszaros f36c1a5001 feat: replace the campaign AI step with a Switch step — a multi-way router whose cases are rows on the canvas node, each with its own draggable source dot (per-case ai_label branches wired by handle, retarget on re-drag, purple case edges with no IF boxes, bottom dot as the otherwise fallback), decided per contact by an AI prompt (1 credit, fenced context, same credit lifecycle) or a free deterministic template-value match, with a mode toggle + case list editor (stale case paths healed on save), drag-to-empty-canvas creating the target and wiring the case, and sequences/ai-credits/automation docs rewritten for the switch 2026-07-18 05:42:50 +02:00
Matthew Meszaros d18c5ebe4a feat: execute campaign AI steps in the scheduler path — one metered completion per contact (SetAI-wired provider + credits with per-step idempotency and refund-on-failure) that labels progress, fills contact custom fields which feed the chosen actions' templates, runs AI-picked pre-configured actions with per-decision campaign logging and tag/label choice resolution, grounds decisions in campaign history and the newest inbound reply via LatestInboundFromContact, and hands instant chains back to the scheduler at AI nodes 2026-07-17 16:26:23 +00:00
Matthew Meszaros d038635425 feat: inbox agent skips drafting trivial acknowledgement replies - reuse replyclassify.WorthModeling on the inbound reply snippet so a one-line 'thanks'/'ok, got it' never triggers a paid 5-credit reply draft, threading the reply snippet through the InboxAgentReply hook payload; the org is only charged when there is enough human text to warrant a real reply 2026-07-14 07:32:16 +02:00
Matthew Meszaros 56de65380b feat: inbox agent that drafts a suggested unibox reply on inbound human replies for human approve/edit/discard - paid + per-org opt-in (organizations.inbox_agent_enabled) feature where the consumer's reply hook, on a non-automated reply, detaches a goroutine (panic-contained, never blocks ingest) that checks entitlement, dedupes via ai_thread_drafts partial unique indexes (one pending per thread, unique source_message_id), pre-checks balance, grounds a reply in the thread history + org voice + skills via generation.Provider.Complete, reserves a draft row then charges 5 credits (idempotency inbox_agent:<draft.ID>, row unwound on a fresh context if the charge fails so no unpaid draft lingers), and emits an org-scoped AI_DRAFT_READY event gated on access_unibox; the agent never sends - only a human POST to /unibox/agent-drafts/:id/approve sends through the normal reply path, claiming pending->approved before send with an approved->pending revert on send failure, alongside list + discard endpoints; plus the unibox awaiting_agent_draft badge + agent_drafts search scope, an AgentDraftCard in the thread view with inline-editable approve-and-send/discard, the workspace settings opt-in toggle, CanUseInboxAgent paid gate, migration 000065, and docs (inbox-agent guide, endpoints, realtime) 2026-07-14 06:13:27 +02:00
Matthew Meszaros 041ae5afd5 feat: resolve inbound bounce events back to the original campaign send by message id and record them idempotently through the deliverability pipeline (suppression, campaign progress, breaker), and document permanent-only bounce detection 2026-07-04 11:36:47 +02:00
Matthew Meszaros 2c910dcdeb feat: make a campaign step's Original a first-class weighted A/B arm driven by a single draggable traffic-split bar, persisting the control share as an is_control variant row 2026-06-15 08:11:53 +02:00
Matthew Meszaros c5dfa5e4e7 feat: remove the HTTP-request action from campaign steps and automations in favor of signed webhooks, keep fire_event for custom payloads, and drop the now-unused outbound quota plumbing 2026-06-15 08:11:35 +02:00
Matthew Meszaros fcafac4195 refactor: remove the campaign 'notify (webhook)' step type now that the HTTP request step covers outbound calls and there is no webhook setup surface 2026-06-14 10:48:06 +02:00