Commit Graph
55 Commits
Author SHA1 Message Date
Matthew Meszaros 0a5e7947b4 feat: return a failed warmup retention delete from the worker so the bus redelivers it up to five times, re-key a Graph message in the map on every move so the sender copy's body can be dropped, never expunge a whole IMAP folder for one message (UID EXPUNGE, else MOVE to Trash, else refuse), build the two retention indexes concurrently in their own migrations 000193 and 000194, keep the dashboard stepper off 1 and 2 days, and describe retention as applying wherever the placement files warmup mail 2026-09-21 01:11:22 -07:00
Matthew Meszaros 1513419a2a feat: delete warmup mail from each mailbox once past a per-mailbox retention window (email_accounts.warmup_retention_days, else retention.warmup_mail_days, default 30) via a consumer sweep that retires the receipt and sender copy and a worker delete action that trashes on Gmail, deletes on Graph, expunges on IMAP and drops the stored body, prune per-message warmup records after retention.warmup_event_days, and count a warmup deletion as tampering only within 24 hours of arrival and never for a retired message, judging Gmail's Trash label on the same rule 2026-09-21 00:52:02 -07:00
Matthew Meszaros 7b93e48bd4 feat: make Archive mean something everywhere by keeping filed conversations out of every working unibox view except All mail and the Archive folder, read a mailbox address out of the raw From header so Awaiting reply stops missing every thread sent as "Name <addr>", file and mark read by thread id rather than by message id, and add per-row triage actions plus a multi-select selection bar to the conversation list 2026-09-20 07:16:35 -07:00
Matthew Meszaros addb956ad6 feat: shared TypeSafe client under internal/pkg/typesafe with inbox tagging phases 2 and 3 (hold, stop, task, suppress behind workspace switches, reversible ones on by default), labels seeded at workspace creation and by the follow-up sweep, premade inbox views (hot leads, needs a reply, follow up, declined, automated), typed reply classification and a reply_intent branch condition, an inbox agent draft gate, Advisor copy judgment with a cached editor re-check, warmup content lint, bounce cause classification that keeps a blocked address sendable, and per-form submission triage 2026-09-19 23:33:37 -07:00
Matthew Meszaros acecd62c88 feat: scope mailbox disconnect and warmup lifecycle to the workspace rather than the member who connected the mailbox so an admin can act on every mailbox the list already shows them, evict a mailbox whose row is gone from every live worker when its provider errors arrive so a deleted mailbox stops calling the provider once a sync interval forever, subscribe before publishing the credential-validation job and classify a socket deadline as the retryable timeout it is, give the worker's validation reply its own budget so a slow mail host no longer loses a finished verdict, guard every global key handler against a keydown carrying no key, drop exceptions whose whole message is an object's default toString, make the Postgres pool size configurable, and record the CASA and security invariants in AGENTS.md 2026-09-19 12:49:46 +02:00
Matthew Meszaros e668a2a36b feat: complete the ADA CASA v2.1.1 AL1 control set across authentication, sessions, access control, cryptography, input validation and configuration, adding a breached-password denylist and per-account login throttling, enforced multi-factor authentication on the admin panel, step-up confirmation before an action that mints a lasting credential, purpose-scoped session tokens, single-use TOTP steps, tenant verification on every cross-referenced identifier, security headers on every surface, encrypted webhook signing secrets, per-organization idempotency, PKCE and a minimal two-scope Gmail consent on the mailbox OAuth flow, bounded spreadsheet and archive decoding, a patched Go toolchain with govulncheck in CI, and the evidence pack under compliance/casa 2026-09-19 08:18:35 +02:00
Matthew Meszaros f99ee57484 feat: scope mailbox disconnect to the workspace instead of the connecting member, so a teammate with manage_emails no longer gets 404 on a mailbox the list shows them, delete by id in the repository on the strength of that check while the worker removal still names the owner the consumer's unibox cleanup is keyed on, and read the API's own reason off the normalised AppError in the accounts page so a refused disconnect says why instead of "The mailbox couldn't be disconnected" on every failure 2026-09-19 06:01:03 +02:00
Matthew Meszaros 0f5ca71155 feat: correct mailbox sending metrics and workspace analytics across dashboard surfaces 2026-09-16 21:44:42 -07:00
Matthew Meszaros 6762b4e491 feat: scope contact activity timelines to the selected organization so teammates can load contacts and campaigns created by other members 2026-09-16 08:20:31 -07:00
Matthew Meszaros def0a5e5f1 feat: restore workspace campaign access and enforce shared analytics permissions (#543) 2026-09-16 03:31:01 -07:00
Matthew Meszaros 50711a8e66 fix: a campaign's linked segments are its audience, so detaching one withdraws its leads (#510) (#523)
* fix: make a campaign's linked segments the audience rather than an accumulator, so detaching one withdraws the leads it enrolled instead of leaving the old list mixed in with the new, tracked by a new campaign_leads.source that keeps a hand-picked lead, an overlapping segment's member and anyone the campaign has already emailed out of the withdrawal, and reported back as withdrawn/contacted counts the dialog confirms and explains (issue #510)

* fix: serialize the linked-segment sweep against a link replacement by taking the same campaign lock, so a sweep that read the old set cannot re-enrol the audience the replacement just withdrew, and word the dialog's confirm and toast so the campaign, not the segment, is what has already emailed a lead

* fix: stop the one-shot segment enrol from re-stamping leads that are already in the campaign, since the Leads tab's Add back runs through it and pinning a whole linked audience as hand-picked because one held-out member was restored is the accumulation this change exists to end

* fix: lock the leads a detachment is about to withdraw before deciding, because a send is reserved by stamping campaign_contact_progress and only then locking the lead row, so a reservation committing mid-pass was invisible to the delete's snapshot and could withdraw a lead whose first email had already gone; and report the already-emailed count on every toast branch, since a detach where the whole audience had been emailed changed no count and said nothing after confirming a removal

* fix: add the campaign_leads.source check constraint NOT VALID, which still enforces every insert and update while skipping a full scan of the largest table in the product under ACCESS EXCLUSIVE to learn that a one-statement-old column holds its own default everywhere
2026-09-14 22:12:15 -07:00
Matthew Meszaros 13e9ce8e10 feat: hold a lead whose mailbox answers out of office until they are back, resuming at the return date it names, plus a manual per-contact pause in one campaign that unsubscribing and the suppression list were the only stand-ins for 2026-09-14 09:26:06 -07:00
Matthew Meszaros 6b6efca865 fix: campaign follow-ups opened a new conversation instead of replying in the contact's thread, so carry In-Reply-To/References and the Gmail threadId from the previous send, give every step a reply-in-thread switch, and let a threading step inherit the conversation's subject (issue #472) (#489) 2026-09-13 20:51:41 -07:00
Matthew Meszaros c4aece241b feat: scope the tag, category and folder registries and unibox conversation labels to the organization instead of the creating user, so a teammate sees and can edit the labels the owner made, splitting a label two workspaces shared into one copy each and guarding every label write against ids from another workspace (#457) 2026-09-12 03:37:31 -07:00
Matthew Meszaros 47defafa09 feat: fix the six self-host defects reported in issue #439 (#456)
* feat: fix the six defects reported in issue #439 by mapping the IMAP UNAVAILABLE, INUSE and NONEXISTENT response codes to retry-level errors instead of a critical reconnect prompt, synthesising a stable no-msgid key so one message with no Message-ID header can no longer 400 the internal map endpoint and wedge every later sync pass with its cursors held, adding mailhtml.FromText and HasContent so an API or agent-created step with a plain body stops shipping the composer's empty div placeholder as its text/html part (derived on create and plain-only update, exposed as body_html on update_campaign_step, dropped at send and preview time, and refused at campaign start with empty_step_body), honouring sender_strategy='explicit' in ResolveCampaignSenderPool and ValidateCampaignReady so an emptied explicit pool parks the campaign instead of widening it to every mailbox in the workspace, making the paused_no_accounts auto-pause loud with an error log line, an error-level activity-feed entry and an org-scoped CAMPAIGN_PAUSED realtime pulse, gating the admin sign-in's Turnstile widget on GET /v1/auth/config so a self-host with CAPTCHA_PROVIDER=none is not locked out, and parsing NATS_URL down to its host:port so a credentialed bus URL no longer reports NATS down

* feat: act on the self-review of the issue #439 fixes by dropping the campaign wizard's own escapeHtml body_html builder, which entity-escaped the quotes in a conditional and made the template fail to parse at send time, and letting the backend's FromText render that part instead so wizard-written steps also get their bare URLs linked for click tracking, correcting the docs and openapi description that claimed an explicit sender pool never falls back when it still unions its tags as migration 000013 designed, extracting the duplicated blank-HTML-part guard into dropBlankHTMLPart shared by the send path and the preview, and recording why the no-msgid key keeps the folder name despite a RENAME changing it

* feat: address the CodeRabbit review on the issue #439 fixes by holding the admin sign-in's Turnstile widget unmounted until /v1/auth/config resolves so an instance with no route to Cloudflare cannot raise a widget error on a screen nobody submitted, failing StartCampaign closed when the sequence read errors rather than skipping both the malformed-template and empty-body refusals, giving TCPCheck the default port its protocol assumes so a portless NATS_URL is no longer reported down, leaving a URL that carries a merge field unanchored because the send path renders bodies with text/template and a quoted contact value would break out of the href, and correcting the sequences guide and the Campaign and CampaignUpdate openapi descriptions that named the wrong tag field
2026-09-12 03:13:38 -07:00
Matthew Meszaros 470654f5b0 feat: say machine_clicks counts the contacts whose only clicks on a step were automated rather than counting steps, document the zero-send rule on all four step rates in the OpenAPI schema, and stop get_campaign_stats dropping the machine open and click counts from both the campaign totals and each step 2026-09-11 09:12:22 -07:00
Matthew Meszaros 5087023e48 feat: give every campaign step its own open, click, reply and bounce rate in Step performance, computed against that step's own sends, with the automated share of its opens and clicks carried alongside them through GetSequenceStats, the campaign analytics API, the get_campaign_stats AI tool and the docs 2026-09-11 08:56:03 -07:00
Matthew Meszaros 68b5d8f358 feat: bind a campaign lead to the mailbox that sends its first email so every follow-up leaves from the same address, holding a lead back while its mailbox is merely out of budget or outside its hours and moving it to another mailbox only when that one can no longer send for the campaign at all 2026-09-09 08:51:00 -07:00
Matthew Meszaros 0e7525866a feat: merge main into the select-all branch and rebuild the contact selection on top of its new keyset pagination, dropping this branch's own cursor repair now that Search carries the sort key and boundary value in a SortCursor, re-extracting the shared WHERE builder from that version, and pointing SearchIDs at the contactSorts spec and the conditional campaign-count lateral so a resolved selection orders and joins exactly the way the list does 2026-09-08 20:26:34 -07:00
Matthew Meszaros a6862477b3 feat: add a 'select all matching' selection to every contact list so a bulk action can cover the whole filtered set instead of only the loaded pages: a ContactSelection body (all + filters + exclude) on bulk update, bulk delete, verification, batch research, segment members and CRM push resolved server-side through a new SearchIDs that shares the search's WHERE builder, a Gmail-style banner and shared selection module in the contacts table, campaign Leads and segment members views, an uncapped select-all in the From contacts picker, and fixes for the keyset cursor comparing a row against itself (contact pagination repeated and skipped rows past page one) and the dashboard sending custom-field filters under the wrong wire name 2026-09-08 05:19:59 -07:00
Matthew Meszaros ed50ad9f2f feat: add a campaign entry delay so a contact's first email can wait a set time after they enter the campaign, with campaigns.entry_delay_minutes and a campaign_leads.added_at anchor (migration 000136), the delay applied in the router's per-lead due check and floored into the placer through ContactSequencePair.NotBefore, a distinct entry_delay constraint in the contact next-action preview, and the control surfaced on the Schedule tab, a new Trigger card at the top of the Steps canvas, the campaign wizard's Schedule step, the launch dialog, the update_campaign AI tool and the iOS schedule page, plus guides, API reference and live scheduler and repository tests 2026-09-08 05:06:04 -07:00
Matthew Meszaros 63d6637d0c feat: expose forms, segments and the suppression list to agents by adding twenty-three tools to the shared registry, so an MCP client, a Hermes-style REST agent, the dashboard assistant and warmblyctl can all build an audience, publish a lead-capture form, link one to a campaign and manage who is unreachable, each gated on the same contact and campaign permissions its HTTP route already uses 2026-09-07 03:39:42 -07:00
Matthew Meszaros 579b0ac04c feat: merge main into the unsubscribe branch again, carrying the UTM campaign columns alongside unsubscribe_mode in every campaign scanner, keeping unsubscribe links out of the moved link tracker and its UTM tagging, and renumbering the opt-out migration to 000124 because main released 000123 for link clicks 2026-09-03 03:43:47 -07:00
Matthew Meszaros 78c4021053 Merge remote-tracking branch 'origin/main' into feat/link-click-attribution
# Conflicts:
#	docs/content/docs/api/reference/campaigns.mdx
#	docs/content/docs/guides/campaigns.mdx
#	internal/repository/pg_campaign.go
#	internal/repository/pg_campaign_lifecycle.go
#	web/src/components/app/campaigns/NewCampaignDialog.tsx
2026-09-03 03:14:19 -07:00
Matthew Meszaros 9485503e03 feat: merge main into the unsubscribe branch, keeping the new campaigns.kind column alongside unsubscribe_mode in every campaign scanner and renumbering the opt-out migration to 000123 because main took 000122 for campaign kind 2026-09-03 01:56:38 -07:00
Matthew Meszaros 5eb92c601c feat: give every campaign email a working opt-out: a reply-to-opt-out line by default or an unsubscribe link (workspace setting under Settings > Sending with a per-campaign override and a {{.UnsubscribeLink}} variable), signed per-recipient unsubscribe links served on the API origin so the List-Unsubscribe header no longer points at a dead warmbly.com page, a confirm page on GET with RFC 8058 one-click on POST and a resubscribe button, reply opt-out detection through the whole-word compliance lexicon with quoted history stripped, a first-class suppression list (Contacts tab, GET/POST/DELETE /suppressions with address and domain entries, audited removal, contact drawer action), the contact Subscribed flag enforced in campaign routing, migration 000122 with a shared recipient_suppressed() predicate, and docs for all of it 2026-09-03 01:52:40 -07:00
Matthew Meszaros 24b7e26fbb feat: add the one-time email campaign preset from #288: campaigns.kind column with sequence and one_time values and a kind field on POST /campaigns, GET /campaigns?kind= and an one_time count on /campaigns-overview, a POST /campaigns-estimate endpoint projecting deduplicated segment recipients against the mailbox pool's per-day capacity and finish date, a wizard type picker whose one-time flow runs Basics, Email, Audience, Sending and Send with send-now or scheduled start and the estimate panel and then creates, links segments and starts the campaign, a guard refusing a second email step on a one-time campaign, draft/scheduled/sending/sent wording with a One-time badge and Type filter in the campaigns list, kind on the AI create_campaign_draft tool, honouring text_only in the send and test-email paths so plain-text campaigns ship no HTML part or tracking, and the campaigns, segments, API reference and endpoint docs 2026-09-03 01:42:00 -07:00
Matthew Meszaros b785d3b86b feat: address review on per-link click attribution: read bare href values and ignore data-href when tracking anchors, compare the destination host instead of substring-matching the tracking domain, tag bare URLs in plain-text bodies when UTM tagging is on, count UTM limits in characters, expose the UTM overrides in the AI campaign tool, dedupe clicks by ticket so two links sharing a destination are two clicks, count machine_clicks only for steps with machine and no human clicks, never withdraw a click stamp that predates per-link logging, keep the coarse timeline click unless a logged click stands for it, resolve the link once per event, give auto-clicks their own tooltip and make the wizard's discard guard notice toggled settings 2026-09-03 01:20:48 -07:00
Matthew Meszaros 99be92f159 feat: attribute every campaign click to the exact link with a per-link click log (email_link_clicks) behind the contact activity timeline, campaign live feed, recent activity and the email_clicked webhook, add per-campaign automatic UTM tagging (utm_tracking with source, medium and campaign overrides, utm_content from the link text) applied at send time to the stored ticket destination, and classify opens and clicks as machine when they arrive within ten seconds of dispatch or when several links of one email are followed within five seconds, so scanner clicks are logged but never count as engagement, fire automations or send webhooks 2026-09-03 01:05:34 -07:00
Matthew Meszaros d6ddf1f170 feat: fix implicit-TLS SMTP on 465 and IMAP STARTTLS on 143 behind a stored per-mailbox security mode that accepts any port, stop worker ID churn orphaning mailbox assignments via flock-claimed persistent worker ids, give the unibox a standard mail-folder sidebar (inbox/sent/drafts/archive/spam/trash) backed by a provider-derived folder column, and expose the AI tool registry over REST for non-MCP function-calling agents (#283) 2026-09-01 03:53:19 -07:00
Matthew Meszaros d7a17a0149 feat: make the per-mailbox daily campaign cap configurable up to 5000 (issue #276): raise campaign_limit, campaign daily_limit and ramp start/ceiling validation to config.LimitMax, warn in the dashboard above 100/day, and update aitools, zapier and docs copy to match 2026-08-31 03:50:36 -07:00
Matthew Meszaros a02ff7c936 feat: address verification overhaul for #264: MillionVerifier as a pay-as-you-go integration plugin with org-sealed key and automatic fallback to the built-in check, built-in prober gains domain cache, Microsoft/Yahoo fingerprinting, MX fallback, disposable/role sub-statuses and a self-check breaker, imports and POST /contacts accept verification results in any known provider vocabulary with auto-detected columns, verdict provenance and expiry columns (migration 000110), campaigns park at paused_undeliverable with re-verify/send-anyway instead of finishing, POST/GET /contacts/verification bulk actions and overview, launch gate override via acknowledge_list_risk, animated verification marks, banner and settings card in the dashboard, and docs 2026-08-29 23:11:22 -07:00
Matthew Meszaros a4739b63ec feat: add first-touch contact source attribution (migration 000106: contacts.source with a CHECK, source_detail, first_seen_at, existing rows stamped unknown) stamped at every creation site (dashboard manual/campaign, file import, Google Sheets sync, API key, AI assistant), write contact_created / campaign_added / campaign_removed / category_added / category_removed lifecycle events into contact_activities inside the same transactions as the links and merge them into the contact timeline, refactor FindNextRoutedPair's routing into a campaignRouter shared with a per-contact RouteContact and split CalculateNextCampaignTime into campaignSenders + placeCampaignSend so a read-only PreviewContactSend derives a contact's next step through the scheduler's own constraints, expose it as GET /contacts/:id/campaigns behind the contact service, render a per-campaign state panel, Campaigns/Lifecycle filter chips and expandable event rows in the Activity tab plus a Source section on Overview, cover it with TestLive* for the timeline events and the next-action preview (step wait, sending window, paused), and document it in the contacts, campaigns and API reference docs (issue #255) 2026-08-29 03:28:16 -07:00
Matthew Meszaros d396f53fc4 feat: add Delete and Duplicate campaign actions to the dashboard (issue #185): every campaign row and the detail header get a ⋯ menu (Edit, Duplicate, Start/Pause, Delete) plus a Delete card at the bottom of Settings, all permission-gated with a confirm that spells out what goes; DELETE /campaigns/:id is now organization-scoped instead of user_id-scoped so teammates can delete, runs in one transaction that also deletes the campaign's pending tasks and cancels a wakeup tick claimed at that moment (campaign_tasks only nulls its link, so those rows kept firing), removes attachment objects and publishes CAMPAIGN_DELETED so a teammate's open detail page is sent back to the list; new POST /campaigns/:id/duplicate copies the campaign row as a draft with steps and their branch graph rewired onto new step ids, tags, folders, senders with rotation reset, A/B variants, advanced settings and attachments (quota-checked, blobs undone if the copy fails) and none of the leads, progress, logs, counters, ramp level, guardrail trip or past dates, naming it (copy)/(copy N) inside the 50 byte cap without splitting runes; a claimed campaign tick whose campaign vanished now ends the chain instead of staying active forever; covered by TestLiveCampaignLifecycle* against real SQL, RemapBranchTargets and duplicateName unit tests and a react-query vitest for the list cache, with API reference, endpoint map and campaigns guide updated 2026-08-25 08:54:51 -07:00
Matthew Meszaros cbf4190f42 feat: make a custom tracking domain verifiable instead of permanently "Pending DNS": the CNAME target is now this install's TRACKING_DOMAIN rather than a hardcoded t.warmbly.com that resolves nowhere, matching is exact on the label boundary (or on shared addresses, so a provider-flattened CNAME stops reading as no record at all) instead of strings.Contains, and every outcome carries the reason plus what DNS actually returned, including when the tracking host the customer is told to point at has no record of its own; a pasted URL is normalized to its host and a malformed one is rejected up front instead of saved and left pending forever; only a VERIFIED mailbox domain is used at send time with the shared host as the fallback and a campaign-feed entry saying why; pixels and click tickets are built from the configured host, and with none configured mail ships untracked rather than carrying links to another deployment's tracking service; adds GET /emails/:id/track and POST /emails/:id/track/verify plus an hourly re-resolution sweep so a record that propagates later starts being used and one that breaks stops routing links; and scopes the tracking-domain write by organization like the read, which also fixes GET /emails/:id passing a user id to an org-scoped query and 404ing for every caller 2026-08-24 09:02:29 -07:00
Matthew Meszaros 94cf21d95e Fix truncated, unformatted and mis-encoded email content in the unibox (#137)
* feat: add internal/pkg/mailhtml, a mail-oriented HTML sanitizer and text flattener, because rendering a received message body means rendering the sender's markup: Sanitize builds on bluemonday's UGC policy but keeps what real email is made of (table layout attributes, inline CSS through the property-allowlisted style sanitizer, legacy font/center, data: and https: images) while dropping script, iframe, object and the text content of style/head blocks so a marketing email's stylesheet cannot render as body copy, forcing target=_blank plus nofollow/noreferrer on links and allowing only http, https, mailto and tel; ToText flattens the same input for previews, turning block boundaries into newlines and decoding entities back to the characters they stand for so an already-escaped body does not surface as literal &amp;; LooksLikeHTML reports whether a stored body is actually markup, which is how a body recorded as HTML by an older sync but containing no tag at all can be recognised as the plain text it really is

* feat: add internal/pkg/mailhdr for RFC 5322 header values, since headers are ASCII on the wire and every transport was writing raw UTF-8 into Subject and display names: Subject and AddressList RFC 2047-encode non-ASCII (a no-op on plain ASCII, and a bare address stays bare rather than being wrapped in angle brackets), DecodeWords reverses encoded-words with a charset hook wired to go-message so legacy encodings Go does not handle natively still decode, and Bare/BareList strip a display name down to the routable address for SMTP envelope commands where 'Ana <a@b.com>' in RCPT TO is a syntax error, promoting go-message from an indirect to a direct dependency

* feat: encode outbound Subject and address headers on all three transports, so a subject or sender name containing an accent, a currency sign or an emoji reaches the recipient as the characters the user typed instead of mojibake: SMTP and Graph were writing the raw string into Subject (only the Gmail transport encoded it) and Graph built its From by fmt.Sprintf rather than mail.Address, so a non-ASCII display name went out unencoded there too, and all three joined To/Cc/Bcc entries verbatim so an encoded display name never appeared even when the caller supplied one; additionally the SMTP envelope now takes bare addresses through mailhdr.BareList, because an API caller may pass 'Name <addr>' (the compose handler has a bareAddress helper precisely because that arrives) and passing that to RCPT TO gets the recipient rejected by the server

* feat: rewrite the IMAP body reader, which was the reason received mail from SMTP/IMAP mailboxes came back corrupted: it built one FetchItemBodySection with a hardcoded Part []int{1} and a comment saying it would adjust when recursing, which it never did, so on a multipart/alternative the text/plain bytes were fetched twice and the second copy was stored as the HTML body (plain text rendered as markup loses every line break, shows & as an entity and swallows anything inside angle brackets), and decodeIfNeeded never reversed Content-Transfer-Encoding at all, leaving quoted-printable bodies full of =E2=80=99 runs and = soft breaks and base64 bodies unreadable, while its charset detection parsed params off a media-type string that never carried any and its mail.ReadMessage call could silently eat leading body lines as headers; the reader now walks the body structure for real part paths, fetches every text leaf in a single FETCH with a server-side Partial size cap, decodes quoted-printable and base64 (tolerating a tail cut mid-quantum by the cap) then converts the part's charset to UTF-8 with go-message, skips attachment-disposition parts so a .txt attachment cannot stand in for the body, takes one part per type inside a multipart/alternative but treats sibling inline parts in mixed/related as additive, and is bounded at five text parts per message; the stored body cap also goes from 200 KB to 512 KB because 200 KB cuts real HTML newsletters mid-document

* feat: decode Gmail's raw headers and entity-escaped snippets, because the Gmail API hands header values back exactly as they arrived on the wire, so a message from a sender whose subject or display name was RFC 2047-encoded showed in the dashboard as =?utf-8?q?caf=C3=A9?= rather than as the text it stands for, and the API's own snippet field is HTML-escaped, so a preview containing an apostrophe surfaced as &#39; in the conversation list and, until the thread reader stopped rendering snippets as message bodies, inside the message itself; getSingleHeader now runs values through mailhdr.DecodeWords (a no-op unless the value actually contains an encoded-word, so Message-ID and the warmup token header are untouched), the comma-split fallback in getAddressList does the same for display names net/mail could not parse, and the snippet is unescaped once on the way in

* feat: fix the conversation-list snippet, which collapsed whitespace before splitting on newlines so the quoted-line and signature filters below it could never match a thing, stripped HTML with bluemonday's strict policy and then showed the escaped output verbatim so an ampersand in an HTML-only message read as &amp; and a marketing email's stylesheet text rendered as body copy, and cut at 100 bytes with text[:100] so a multi-byte character or emoji at the boundary became a replacement glyph; it now flattens HTML through mailhtml.ToText (entities decoded, style and script content dropped) including when a sender puts markup in their text/plain part, filters quoted history and everything past the RFC 3676 signature delimiter while the text still has lines, collapses whitespace afterwards, and truncates on a rune boundary at 200 characters

* feat: make GET /unibox/:id serve a display-safe body and stop it failing outright, sanitizing body_html through mailhtml before it leaves the API so every consumer gets markup that cannot execute rather than each call site having to defend itself, degrading a body blob that cannot be read to the message's preview text with a new body_truncated flag instead of returning 500 (which made a message with a missing blob unopenable, and hit every seed, sandbox and dev-history fixture row since only the '<seed-' prefix was recognised while the sandbox uses '<sbx-' and dev history '<dev-'), and treating a stored HTML body that contains no tag at all as the plain text it really is, because mail synced before the IMAP reader addressed parts individually recorded the plain part under both bodies and serving that as HTML is exactly what collapsed a ten-line message onto one line

* feat: escape composer text before turning it into the HTML part of an outgoing email, replacing body_html: trimmedBody.replace(/\n/g, '<br />') in both the compose window and the reply composer with a shared plainToHtml that escapes the five markup characters first, so an email containing 'Terms & conditions' no longer ships a broken entity and one containing anything in angle brackets ('<see attached>', 'a < b', a pasted tag) no longer has the rest of the paragraph swallowed by the recipient's mail client as an unclosed tag, while runs of spaces survive as non-breaking spaces and bare URLs become links without eating the sentence punctuation after them; the same unescaped plain-to-HTML pattern in the campaign step editor's applyTemplate now goes through promptToHtml, which escapes as it paragraph-wraps

* feat: render the real message body in the unibox thread reader instead of the list preview, which is the whole of the reported bug: ThreadView mapped each thread row to a UniboxEmail whose body was '<p>' + escapeHtml(m.snippet) + '</p>' and MessageBubble rendered that as the message, but a snippet is a preview capped at 100 characters with every run of whitespace collapsed to one space, so a ten-line email displayed as roughly two lines on a single continuous line, and Gmail's already-escaped snippet was escaped a second time so an apostrophe read as &#39;; each expanded message now loads its own body from GET /unibox/:id (the newest message and anything unread open on mount, older messages collapse to their preview line so a long thread does not fetch every body at once) and renders it in a sandboxed iframe carrying no allow-scripts, which keeps a sender's stylesheet from restyling the dashboard and means nothing in the message can run even though the API already sanitized it, sizing itself from the inner document as images load, with the preview kept as the fallback when a body cannot be fetched and a notice when only a preview is stored

* feat: document how a message body is read and returned, adding a 'Reading a message' section to the unibox guide covering the expand-on-open behaviour, that formatting and special characters are preserved as sent, that the conversation list preview is a summary and not the message, and that HTML mail renders in an isolated frame with links opening in a new tab, plus a paragraph in the API endpoint reference stating that GET /unibox and GET /unibox/thread return previews carrying snippet while GET /unibox/:id returns body_plain and a sanitized body_html, and what body_truncated means

* feat: add email_accounts.save_to_sent, the per-mailbox switch for filing a copy of outbound mail in the Sent folder, defaulting on because plain SMTP submission leaves nothing behind in the sender's account while Gmail and Outlook file their own copy through their APIs, making it a per-mailbox choice rather than a global one since a submission server that files the copy itself (Gmail's SMTP, Fastmail, Zoho) would otherwise end up with two of everything, which is exactly why every desktop mail client ships the same switch, and wiring the column through the Email model, the mailbox read paths and UpdateEmail so it is readable and writable from the dashboard and the API

* feat: teach the IMAP client to APPEND a sent message and the SMTP client to hand back the exact bytes it submitted, the two transport pieces the Sent-folder copy needs: AppendToSent resolves the folder from the RFC 6154 \\Sent special-use attribute first (requesting it only when the server advertises SPECIAL-USE) and falls back to matching the known names against both the full mailbox name and its leaf, since servers namespace as INBOX.Sent and localize the label, caches the result for the life of the connection, files the message flagged \\Seen and dated when it was sent, and returns a sentinel rather than an error when the account has no Sent folder at all; APPEND addresses its mailbox by argument and never touches the selected mailbox, so unlike the warmup MOVE/STORE actions it is safe to run while the sync loop is mid-fetch on the same connection

* feat: file a copy of every SMTP send in the mailbox's Sent folder, closing the gap where a message sent from Warmbly through an SMTP/IMAP mailbox existed only in the recipient's inbox: nothing appeared in the customer's own mail client, and nothing appeared in the unibox either, whose thread reader can only show messages the sync found in a folder, so a user who sent from the dashboard and then went looking for what they sent found no record of it at all; the worker now APPENDs the exact bytes the SMTP client submitted after a successful send, best effort so a failed append never turns a delivered message into a failed task, skipping warmup traffic because filing dozens of machine-generated messages a day would bury the customer's real sent mail, and skipping Gmail and Graph mailboxes entirely since their APIs file their own copy; the per-mailbox setting rides along on the add-email worker payload as a pointer so an older control plane that does not send the field is read as unset and takes the default rather than as an explicit no

* feat: expose the Sent folder copy as a mailbox setting in the dashboard, adding a 'Keep a copy of sent mail' toggle to the Settings tab of the mailbox drawer that only renders for SMTP/IMAP mailboxes (Gmail and Outlook file their own copy, so the control would be a lie there), tracked by the drawer's save bar alongside the other editable fields, and worded so the one case where it should be turned off is obvious: a provider that already saves its own copy, where leaving it on means seeing every sent message twice

* feat: document the Sent folder copy in the mailboxes guide and the API reference, explaining why the toggle exists at all (SMTP submission leaves nothing in the sender's own account, so without it a sent message shows in neither the customer's mail client nor the unibox thread), when to turn it off (a provider such as Gmail, Fastmail or Zoho that already files its own copy of anything submitted over SMTP, where leaving it on doubles every message), that OAuth Gmail and Outlook mailboxes never show the control because their APIs file the copy themselves, that warmup traffic is deliberately excluded, and that PATCH /emails/:id takes save_to_sent

* feat: add unibox_emails.body_text and its search index, because unibox search ran against search_tsv, a generated column built from subject and snippet, and a snippet is a truncated one-line preview, so searching for a phrase that appears in the third paragraph of an email returned nothing at all and read as broken search rather than as search that only covers the first line; message bodies stay in object storage where they belong, and what lands in Postgres is a bounded 16 KB plain-text rendering carried on the new-email worker event, indexed with a GIN expression index rather than a second stored generated column since adding one of those rewrites the whole table while this builds against a column that is empty on every existing row

* feat: index what a message actually says, adding mailhtml.SearchText (HTML flattened, entities decoded, whitespace collapsed, quoted history deliberately kept because a phrase someone quoted back at you should still find the conversation, truncated on a rune boundary) and computing it on all three sync paths so IMAP, Gmail and Graph mail all arrive with searchable text, writing it on insert, and widening the unibox search filter to match either the existing subject-and-preview vector or the body expression, written exactly as the new index declares it so the index is actually used

* feat: backfill the searchable text of messages that were synced before bodies were indexed, so search covers the archive a customer already has instead of only mail that arrives from now on, which would have made the feature useless on day one for exactly the people who need it; the sweep pages through unibox_emails by id, reads each body from object storage under the mailbox owner's key, renders it with the same helper the sync path uses and writes it back, at 100 rows per 30 seconds because nothing waits on it, and returns for good once a pass finds nothing left to visit, with rows whose stored body really is empty simply revisited after the next restart rather than needing a tried-and-failed marker in the schema

* feat: document that unibox search now covers message text and not just subjects and previews, in the search paragraph of the unibox guide where the old wording only promised that search stays inside the current scope

* feat: add generation.RenderThread, the shared way to put a conversation in front of a model, because every AI surface was grounding on preview snippets and a draft written from the first hundred characters of each email answers the greeting rather than the question; it strips quoted history and signatures (the earlier messages are already in the prompt on their own, so quoting them again spends the budget twice, though a reply written underneath the quote is kept rather than thrown away when there is nothing meaningful above the attribution line), spends a bounded character budget newest-message-first since the message being replied to matters most, degrades older messages to their preview line instead of dropping them once the budget runs low, and renders oldest-first so the transcript reads in order

* feat: add grounding reads to the unibox service and repository, returning message text (the stored body, falling back to the preview for mail synced before bodies were indexed) for a thread or for all correspondence with one address, kept deliberately separate from the preview queries and given their own result type so a 16 KB body can never leak into a list response by accident, capped at twenty messages whatever a caller asks for, and paired with a RenderGrounding helper so every AI surface formats a conversation the same way instead of each one rolling its own transcript loop

* feat: ground every AI writing surface in what the messages actually say, switching the unibox reply draft, the compose draft's correspondence history, the inbox agent's thread history and the assistant's read-thread tool from preview snippets to real message text through the new grounding reads, which is what makes a drafted reply answer the question that was asked rather than the first sentence of the email; the inbox agent's triviality gate also reads the reply's full text now, since a preview line cannot tell a one-word ack apart from a long message that happens to open with one, and the assistant tool returns a bounded body per message with quoted history stripped instead of a snippet field

* feat: say in the docs that AI drafting reads the messages and not their previews, in both the unibox reply-draft section (adding that quoted history is stripped and the newest messages get the most room, so a draft answers what was asked rather than the opening sentence) and the inbox agent's grounding section, where 'the full thread so far' was true of the message list but not of how much of each message the model actually saw

* feat: renumber the two new migrations to 000087 and 000088 after rebasing onto main, which landed its own 000085 (org data transfer) and 000086 (email sync state) in the meantime, so the sequence has no duplicate versions

* feat: add the two new API fields to the OpenAPI spec that landed on main while this branch was open, documenting save_to_sent on the Mailbox and MailboxUpdate schemas and body_truncated on UniboxEmail, and saying on body_html that what the API returns is already sanitized so a client can render it directly
2026-08-21 18:45:09 +02:00
Matthew Meszaros 5e6287c920 feat: add the Advisor, continuous sending checks surfaced on the row they are about (#86)
* feat: index advisor findings by subject and parent entity so a list page fetches its whole surface once and every row resolves its own advice from the shared cache instead of firing a request per row

* feat: rebuild the advisor fix drawer as a three-screen resolution flow (why it fired with the measured evidence, the exact before and after, then an animated outcome with undo) with a progress rail and direction-aware transitions, and deep-link manual fixes to the screen where they are made

* feat: add AdvisorRowFlag, the inline per-row advisor indicator that renders on the mailbox or campaign the problem is about and opens that row's findings in an anchored panel instead of making the reader join a card list against a table

* feat: add AdvisorSummaryBar, a one-line collapsible page summary that replaces the stack of advisor cards above a list, counts the distinct rows implicated rather than the findings, and forces itself open only for critical or workspace-level advice no row flag can carry

* feat: put advisor advice on the mailbox row it is about in the accounts list, replace the card stack above the table with the collapsible summary bar, and support ?mailbox=<id> so a finding can deep-link straight to the mailbox detail instead of the top of the list

* feat: flag advisor findings on the campaign row in the campaigns list, including step-level copy problems which index onto their parent campaign since a step has no row of its own, and add the collapsible summary bar above the list

* feat: move the deliverability and contacts pages onto the collapsible advisor summary bar so their findings stop pushing the numbers they describe below the fold

* feat: add an ordered Steps field to advisor findings, persisted as text[] and always refreshed from the current build, and write real how-to steps for the deliverability checks that have no one-click fix (bounce rate, spam placement, tracking domain, and per-record SPF/DKIM/DMARC instructions)

* feat: write ordered how-to steps for the manual advisor findings where the remedy alone leaves someone stuck (broken template syntax, missing first-name fallback, unsubscribed contacts still enrolled, a campaign with no resolvable sender, and a mailbox that lost warmup pool standing) and correct the personalization detail that named a merge syntax this product does not use

* feat: show a mailbox's advisor findings at the top of its detail drawer, which is where both the row flag and the ?mailbox deep link now land

* feat: open the resolution flow from findings that have no one-click fix too, since the ordered how-to lives there and a card with no Fix button previously left the steps unreachable

* docs: document the per-row advisor flags, the collapsible page summary, the three-screen resolution flow, and the ordered manual steps for findings with no one-click fix

* feat: align the advisor summary bar to the px-5 page gutter used by SectionBar and the list rows on all four surfaces, instead of sitting flush against the edge while the table it describes is indented

* fix: stop the resolution drawer collapsing to zero height between screens by switching the step transition to popLayout with a layout-animated container, so the dialog resizes into the next screen instead of snapping shut and reopening

* feat: wire the advisor repository, narrator, service, tool registration, and background runner into the backend boot path so findings evaluate on a schedule and the assistant can read them

* docs: register the advisor guide in the sidebar, add its endpoint scope table to the API reference, and document the sandbox advisor showcase

* fix: darken the advisor nav badge to solid orange-600 on white instead of a pale amber-100 chip that read as a disabled control beside the sidebar's saturated indicators, and drop the critical badge to rose-600 so the two stay in the same weight class

* fix: use orange-500 for the advisor nav badge, matching the high-severity dot on the row it points at, rather than the darker orange-600

* feat: add an Auto safety class to advisor actions and mark the seven fixes autopilot may apply unattended (the cap cuts, the send-gap widen, the campaign limit matches, and the unsubscribe header), with a test pinning the boundary so nothing that halts sending or generates new outbound mail can drift into it

* feat: add advisor autopilot, which applies the auto-safe fixes unattended as the member who switched it on, resolving their live permissions each run so it fails closed when they leave the org, bounded to 10 changes per evaluation and audited per fix like any hand-made change

* feat: add the advisor agent fix, a bounded per-finding agent run that resolves the problems a settings change cannot (broken template syntax, bulk-reading copy, shared-inbox lists) as the calling member inside a tool allowlist scoped to the finding's category, metered per iteration and marked applied only when it actually called a write tool

* feat: surface autopilot and the agent fix in the dashboard, adding the workspace toggle that names exactly which changes it may make, an Auto chip on the findings it is allowed to take, and an agent-fix path in the resolution drawer that reports the tools it actually called rather than only its own account of them

* docs: document the agent fix and autopilot, naming the exact set of changes autopilot may make, that it acts as the member who enabled it and stops when they leave, and why the agent-fix endpoint is JWT only

* fix: gate the agent fix per detector instead of per category, so a missing DMARC record no longer offers a Fix-with-agent button it can never satisfy and then reports failure; findings whose fix lives in DNS or a provider console now show their manual steps, and the client is told which is which via agent_fixable

* feat: soften the advisor surfaces to translucent washes, replacing the filled nav badge with a tinted pill that carries its colour in the text, frosting the row panel and the resolution drawer, and turning the severity chips and cards into layers the page shows through

* docs: correct the agent-fix scope to name the findings it cannot resolve, and why a DNS record shows steps instead of a button

* feat: ship the actual DNS records for the findings that live outside the platform, with the provider's SPF include resolved, the DMARC record scoped to the sending domain and starting at p=none, the DKIM host plus the console that generates its value, and a tracking CNAME pointing at this install's own tracking host

* feat: render advisor snippets as labelled copy-button rows so a DNS record is one click per field rather than a text-selection exercise, with no copy affordance on a value the server could not supply

* docs: document the pasteable DNS records and the guarantee that every check offers a fix, an agent, or ordered steps

* fix: bump golang.org/x/text to 0.39.0 to clear CVE-2026-56852, a HIGH-severity infinite loop in norm.Iter that Trivy started failing the security scan on
2026-07-30 17:15:09 +02:00
Matthew Meszaros fe127c9d8d feat: add automation agent tools to list, read, update, enable or disable, and delete automations while preserving the node graph 2026-07-22 17:05:37 +02:00
Matthew Meszaros 67c4dbfdfb feat: add API key and webhook management agent tools gated on manage-api-keys and manage-settings 2026-07-22 17:05:37 +02:00
Matthew Meszaros 1efc722458 feat: add JWT-only team, org-settings, voice-profile, and read-only billing agent tools that API keys and MCP cannot reach 2026-07-22 17:05:37 +02:00
Matthew Meszaros 49507b0244 feat: add mailbox management agent tools for reading, updating sending limits and warmup, tracking domains, warmup appeals, and disconnecting a mailbox 2026-07-22 17:05:37 +02:00
Matthew Meszaros 64ccd1b273 feat: add send_reply and compose_email agent tools as RiskSend with a recipient-suppression gate, so the assistant can send with per-action approval and never over MCP 2026-07-22 17:05:37 +02:00
Matthew Meszaros ae99322f29 feat: add unified-inbox agent tools for mark-seen, thread labels, snooze, and scheduled-send review and cancel 2026-07-22 17:05:24 +02:00
Matthew Meszaros 6d8385c89e feat: add campaign sequence-step agent tools to list, add, update, and delete steps 2026-07-22 17:05:24 +02:00
Matthew Meszaros 2efffda0f6 feat: add campaign get, update, delete, sender-set, tracking-domain verify, and logs agent tools 2026-07-22 17:05:24 +02:00
Matthew Meszaros 66c49bddc0 feat: add full CRM agent tools covering deals, tasks, task types, notes, pipelines, and stages end to end 2026-07-22 17:05:24 +02:00
Matthew Meszaros 2fba098313 feat: add contact create, delete, bulk-edit, timeline, and sent-email agent tools, each gated by the caller's contact permission and org 2026-07-22 17:05:24 +02:00
Matthew Meszaros 97461cd0cb feat: expand aitools Deps with mailbox, send, compose, warmup, sequence, org, api-key, webhook, subscription, and suppression services and register the new tool groups 2026-07-22 17:05:24 +02:00
Matthew Meszaros 0f86917880 feat: add a JWTOnly gate to the aitools registry so tools whose HTTP route has no API-key scope are hidden from API-key and MCP callers 2026-07-22 17:05:24 +02:00
Matthew Meszaros b465f30d42 fix: scope account analytics by org id not user id 2026-07-21 17:34:59 +02:00