Commit Graph
194 Commits
Author SHA1 Message Date
Matthew Meszaros a3989f9d9c feat: turn automations into a lead-intake path so a Facebook, Instagram, LinkedIn or TikTok lead form pushed by Zapier, Make, n8n or any webhook becomes a tagged, campaign-enrolled contact without leaving Warmbly: add the warmbly.upsert_contact and warmbly.add_to_campaign built-in actions with templated field mapping, custom fields, tags, campaign and an if-exists policy, fire a rich contact.created event from the one contact write path (silent for file imports, sheet syncs and API batches over 100) and expose contact.created and form.submitted as automation triggers with condition fields, variables and sample data, carry an automation depth through events raised by an action so a flow that creates a contact cannot re-trigger itself past five hops, stamp automation-created contacts with the new automation source (migration 000129), share the campaign picker between the sheet sync wizard and the automation builder, document lead intake in the automations, Zapier, Make, contacts, forms, integrations, expressions and webhook pages plus a new n8n guide, mirror the new triggers and actions on the marketing automations page, and drop the 34 MB cli binary that was committed by mistake 2026-09-06 00:36:05 -07:00
Matthew Meszaros 3bf8d4bdfa Merge remote-tracking branch 'origin/main' into feature/mailbox-fair-use-allowance 2026-09-04 21:25:53 -07:00
Matthew Meszaros 3596a9770f feat: make mailboxes truly unlimited under a fair-use allowance of one mailbox per daily send the plan includes (15,000 on Business), drop the 200 hard cap and the 5-per-day connect throttle, open a request-more dialog with inline limit requests and plan upgrade when the allowance is reached, add bulk CSV mailbox import with live progress and a downloadable failed-rows file, reserve attachment storage atomically under a per-org lock so concurrent uploads cannot pass the quota (#326), report storage and the allowance on the billing usage meters, and honor approved daily-send overrides in the sender 2026-09-04 21:00:26 -07:00
Matthew Meszaros 028689fd2e feat: device-code sign-in for the CLI, with a browser approval page at /cli that mints a scoped API key, self-revocation at DELETE /api-keys/self so a read-only credential can always end itself, and app_url plus websocket_url on /auth/config so a client can find the dashboard and the realtime gateway on a self-hosted layout 2026-09-04 20:14:27 -07:00
Matthew Meszaros eb85224cc8 Merge remote-tracking branch 'origin/main' into fix/contact-timeline-opaque-cursor 2026-09-04 02:57:59 -07:00
Matthew Meszaros ab0c756aa2 feat: show a campaign's linked segments on the Leads tab as chips with live enrolled and held-out counts that filter the list, explain an empty list caused by an empty segment or hand-removed members and add them back in one click, add a scope-aware Export to campaign leads and segment members with a campaign-ready preset that carries lead status and engagement, write the links and the enrolment of PUT /campaigns/:id/segments in one transaction so a failed enrolment changes nothing instead of answering added 0, restart a completed campaign when a one-shot enrol adds leads, clear the manual-removal record when a contact is added by hand, count every link's members in one scan, audit sweep enrolments so open Leads tabs refresh, and document it in the campaigns and segments guides and the API reference 2026-09-03 20:38:27 -07:00
Matthew Meszaros 5dbbaad5b5 feat: add the pagination envelope with its opaque next_cursor to the dashboard's ContactTimelineResult type and note that the top-level has_more only mirrors it 2026-09-03 20:19:16 -07:00
Matthew Meszaros 701dc404a2 Merge remote-tracking branch 'origin/main' into fix/click-without-open-tracking 2026-09-03 05:22:05 -07:00
Matthew Meszaros 31dabea0a4 feat: rebuild the click-without-open fix on top of the per-link click attribution from #298: a person's click now also counts as an open and a burst that withdraws the click withdraws the open it implied unless a real open is on record, routing readers ignore machine opens as the docs promised, every open gets its own log row and every open and click records the mail client or proxy, browser, device, OS, country, region and city (migration 000124: origin columns on email_link_clicks plus an email_opens table), the tracking service publishes only the address's network in a nullable client_ip field which the consumer resolves with GeoLite and drops, the contact Activity tab shows each open and the origin of opens and clicks, the campaign overview gains a who-engaged-from-where breakdown exposed as engagement in campaign analytics, live open and click events carry occurred_at, client and location, the leads table explains why an open is not always counted, both logs are pruned daily after a year, email_opens joins the export registry, the consumer reads GEODB_PATH optionally, and the guides and API references are updated (fixes #294) 2026-09-03 03:49:44 -07:00
Matthew Meszaros 579b0ac04c feat: merge main into the unsubscribe branch again, carrying the UTM campaign columns alongside unsubscribe_mode in every campaign scanner, keeping unsubscribe links out of the moved link tracker and its UTM tagging, and renumbering the opt-out migration to 000124 because main released 000123 for link clicks 2026-09-03 03:43:47 -07:00
Matthew Meszaros 78c4021053 Merge remote-tracking branch 'origin/main' into feat/link-click-attribution
# Conflicts:
#	docs/content/docs/api/reference/campaigns.mdx
#	docs/content/docs/guides/campaigns.mdx
#	internal/repository/pg_campaign.go
#	internal/repository/pg_campaign_lifecycle.go
#	web/src/components/app/campaigns/NewCampaignDialog.tsx
2026-09-03 03:14:19 -07:00
Matthew Meszaros 9485503e03 feat: merge main into the unsubscribe branch, keeping the new campaigns.kind column alongside unsubscribe_mode in every campaign scanner and renumbering the opt-out migration to 000123 because main took 000122 for campaign kind 2026-09-03 01:56:38 -07:00
Matthew Meszaros 5eb92c601c feat: give every campaign email a working opt-out: a reply-to-opt-out line by default or an unsubscribe link (workspace setting under Settings > Sending with a per-campaign override and a {{.UnsubscribeLink}} variable), signed per-recipient unsubscribe links served on the API origin so the List-Unsubscribe header no longer points at a dead warmbly.com page, a confirm page on GET with RFC 8058 one-click on POST and a resubscribe button, reply opt-out detection through the whole-word compliance lexicon with quoted history stripped, a first-class suppression list (Contacts tab, GET/POST/DELETE /suppressions with address and domain entries, audited removal, contact drawer action), the contact Subscribed flag enforced in campaign routing, migration 000122 with a shared recipient_suppressed() predicate, and docs for all of it 2026-09-03 01:52:40 -07:00
Matthew Meszaros 1cb2b65cd7 Merge remote-tracking branch 'origin/main' into feat/link-click-attribution 2026-09-03 01:42:44 -07:00
Matthew Meszaros 24b7e26fbb feat: add the one-time email campaign preset from #288: campaigns.kind column with sequence and one_time values and a kind field on POST /campaigns, GET /campaigns?kind= and an one_time count on /campaigns-overview, a POST /campaigns-estimate endpoint projecting deduplicated segment recipients against the mailbox pool's per-day capacity and finish date, a wizard type picker whose one-time flow runs Basics, Email, Audience, Sending and Send with send-now or scheduled start and the estimate panel and then creates, links segments and starts the campaign, a guard refusing a second email step on a one-time campaign, draft/scheduled/sending/sent wording with a One-time badge and Type filter in the campaigns list, kind on the AI create_campaign_draft tool, honouring text_only in the send and test-email paths so plain-text campaigns ship no HTML part or tracking, and the campaigns, segments, API reference and endpoint docs 2026-09-03 01:42:00 -07:00
Matthew Meszaros f579497c81 feat: make profile and workspace avatar changes survive a refresh: user avatar upload and remove go through the user service so the cached /auth/me copy is dropped instead of serving the old avatar for 30 minutes, the org list mapper carries avatar_url into the persisted workspace pointer it previously discarded, the avatar mutation hooks patch the cached user, org pointer and org queries with the server's answer before invalidating, all four handlers delete the replaced or removed object under our own avatars/ prefix with millisecond keys so quick replacements cannot collide on an immutable URL, the org avatar removal is audited, the uploader copy stops promising WebP and GIF, and the API docs note that remove deletes the stored file (fixes #295) 2026-09-03 01:06:36 -07:00
Matthew Meszaros 99be92f159 feat: attribute every campaign click to the exact link with a per-link click log (email_link_clicks) behind the contact activity timeline, campaign live feed, recent activity and the email_clicked webhook, add per-campaign automatic UTM tagging (utm_tracking with source, medium and campaign overrides, utm_content from the link text) applied at send time to the stored ticket destination, and classify opens and clicks as machine when they arrive within ten seconds of dispatch or when several links of one email are followed within five seconds, so scanner clicks are logged but never count as engagement, fire automations or send webhooks 2026-09-03 01:05:34 -07:00
Matthew Meszaros 290f1f00aa Merge remote-tracking branch 'origin/main' into feature/public-forms
# Conflicts:
#	cmd/backend/main.go
#	docs/content/docs/guides/workspace-export-import.mdx
2026-09-01 09:29:53 -07:00
Matthew Meszaros 05d74dbe9f feat: forms v2, a full redesign of the hosted form and its builder: eight one-click themes, card/wide/split layouts with a brand cover panel, classic paged and Typeform-style focus modes split on a new page_break block, logo/cover/background uploads with size, fit and veil controls, an optional header bar that can span the page or sit with the form, a real HSV colour picker and font preview, per-contact personalized links that prefill and attribute without email verification, a render-token gate so the form JSON cannot be scraped without loading the page, funnel analytics with per-page drop-off and identified visitors, a leads-style forms list and responses table, and an organization custom forms domain verified by CNAME with hourly re-checks 2026-09-01 09:27:28 -07:00
Matthew Meszaros 128e314103 Merge remote-tracking branch 'origin/main' into fix/issue-277 2026-09-01 04:03:37 -07:00
Matthew Meszaros d6ddf1f170 feat: fix implicit-TLS SMTP on 465 and IMAP STARTTLS on 143 behind a stored per-mailbox security mode that accepts any port, stop worker ID churn orphaning mailbox assignments via flock-claimed persistent worker ids, give the unibox a standard mail-folder sidebar (inbox/sent/drafts/archive/spam/trash) backed by a provider-derived folder column, and expose the AI tool registry over REST for non-MCP function-calling agents (#283) 2026-09-01 03:53:19 -07:00
Matthew Meszaros 60c9e316d9 feat: hosted lead-capture forms end to end: drag-and-drop builder with field settings, design panel, embed/share and submissions tabs in the dashboard, a public TanStack form app (forms/) served by the new standalone forms service (cmd/forms + internal/formserver) on FORMS_DOMAIN with per-form frame-ancestors CSP, honeypot/fill-time/Turnstile/per-IP submit protection and a same-origin JSON API proxying the backend internal API, form submissions creating contacts with categories and campaign enrollment plus realtime, audit, webhook and org-transfer coverage, migration 000114, seed forms, CI jobs, Dockerfile, systemd/nginx/compose manifests and docs 2026-09-01 01:17:51 -07:00
Matthew Meszaros 66a105fb9f feat: make segments a live campaign audience and round out lead management (issue #277): a campaign_segments link table with GET/PUT /campaigns/:id/segments (max 20, replace semantics), immediate enrolment of linked-segment members as leads plus targeted re-syncs on contact writes, segment edits, pin-ins and imports and a 2-minute backend sweep for drift, waking active campaigns and restarting completed ones through the full launch checks; a Segments manager dialog and toolbar button on the campaign Leads tab; Remove from campaign as the row action and a selection-bar bulk action so leads can leave a campaign without deleting the contact; campaign and segment pickers in the CSV import wizard backed by a new segment_ids import option that pins imported rows as include overrides; a delete guard naming linked campaigns; orgtransfer registration, live repo tests and docs for all of it 2026-08-31 08:45:27 -07:00
Matthew Meszaros 7a7c6051bc Merge remote-tracking branch 'origin/main' into feature/contact-segments 2026-08-30 02:07:56 -07:00
Matthew Meszaros 52916ab60d feat: verification evidence engine and Greptile fixes for #264: contact_verification_evidence ledger (migration 000111) fed by clean deliveries, human opens, clicks, replies and recipient-naming bounces from the send, tracking, reply and bounce paths, a decaying score that lets real mail outrank a probe and a newer bounce outrank older engagement, verification_confidence on every contact with a reasons list and animated Deliverability card in the contact drawer, per-organization probe breakers, undeliverable counts that ignore finished leads, tighter wiring comments, and docs 2026-08-29 23:49:33 -07:00
Matthew Meszaros 15dface385 feat: add the Add to segment and Remove from segment action nodes to the campaign sequence canvas with a segment picker 2026-08-29 23:45:06 -07:00
Matthew Meszaros d831aba034 feat: add segment_ids to the dashboard contact search model and route segment audit events and contact changes to the segments queries on the realtime spine 2026-08-29 23:45:05 -07:00
Matthew Meszaros 960f42b846 feat: add the dashboard segment model, API client and react-query hooks for segments, fields, preview, overrides, member modes, contact-side membership and campaign enrolment 2026-08-29 23:45:05 -07:00
Matthew Meszaros a02ff7c936 feat: address verification overhaul for #264: MillionVerifier as a pay-as-you-go integration plugin with org-sealed key and automatic fallback to the built-in check, built-in prober gains domain cache, Microsoft/Yahoo fingerprinting, MX fallback, disposable/role sub-statuses and a self-check breaker, imports and POST /contacts accept verification results in any known provider vocabulary with auto-detected columns, verdict provenance and expiry columns (migration 000110), campaigns park at paused_undeliverable with re-verify/send-anyway instead of finishing, POST/GET /contacts/verification bulk actions and overview, launch gate override via acknowledge_list_risk, animated verification marks, banner and settings card in the dashboard, and docs 2026-08-29 23:11:22 -07:00
Matthew Meszaros 2a831e9783 feat: let a linked self-hosted instance sign Google and Microsoft mailboxes in through Warmbly Cloud's own OAuth apps and send with cloud-brokered access tokens: the cloud runs the consent (pool_link_mailboxes.managed, brokered state in Redis, the existing /addresses/*/callback completes it and redirects to the instance's /cloud-oauth/done), keeps the refresh grant, mints short-lived tokens at /pool-link/instance/mailboxes/:id/token and refuses them for revoked links, removed, inactive or blocked mailboxes; the instance mirrors such mailboxes without a credential (cloud_link_mailboxes.managed), ships them to the worker as brokered so goog/msgraph init on a token source that pulls from /api/v1/internal/cloud-link/token/:id, lets the consumer ignore cloud warmup tokens for enrolled mailboxes, and can adopt mailboxes connected directly on the workspace; Add account shows the cloud path and the adoptable list, and the Warmbly Cloud guide documents the model 2026-08-29 09:50:52 -07:00
Matthew Meszaros b75fdcf86c Merge remote-tracking branch 'origin/main' into feature/self-hosted-warmup-pool-access
# Conflicts:
#	internal/scheduler/warmup_scheduler.go
2026-08-29 07:10:38 -07:00
Matthew Meszaros 37b60b59d3 feat: let a self-hosted instance warm its mailboxes in the hosted pool: device-code link approved at /connect, instance-token API that enrolls SMTP/IMAP mailboxes as warmup-only accounts (no history import, non-warmup mail dropped), free for 10 mailboxes and unlimited on the seeded $15 pool plan, tier fallback to proven healthy mailboxes when a pool runs thin, local warmup stands down for enrolled mailboxes, Settings > Warmbly Cloud step flow and linked-instances page, docs guide, marketing copy, and fix SetWarmupLifecycle re-reading the row with an org-scoped lookup so every warmup start/pause returned 404 2026-08-29 07:09:04 -07:00
Matthew Meszaros 5821fc2a1c feat: merge origin/main into the website tracking branch after PR #260 landed 000106_contact_source, renumber the website tracking migration to 000107, and fold the page_hit timeline type, its query source and its Website filter, meta line and expandable details into the reworked contact Activity tab and models from the contact-activity PR 2026-08-29 04:38:35 -07:00
Matthew Meszaros 896dc4fe3c feat: merge origin/main (campaign lead engagement filters, contact export org scoping) into the issue #255 branch, keeping both the engagement and the next-action sections of the campaigns guide 2026-08-29 03:40:13 -07:00
Matthew Meszaros a4739b63ec feat: add first-touch contact source attribution (migration 000106: contacts.source with a CHECK, source_detail, first_seen_at, existing rows stamped unknown) stamped at every creation site (dashboard manual/campaign, file import, Google Sheets sync, API key, AI assistant), write contact_created / campaign_added / campaign_removed / category_added / category_removed lifecycle events into contact_activities inside the same transactions as the links and merge them into the contact timeline, refactor FindNextRoutedPair's routing into a campaignRouter shared with a per-contact RouteContact and split CalculateNextCampaignTime into campaignSenders + placeCampaignSend so a read-only PreviewContactSend derives a contact's next step through the scheduler's own constraints, expose it as GET /contacts/:id/campaigns behind the contact service, render a per-campaign state panel, Campaigns/Lifecycle filter chips and expandable event rows in the Activity tab plus a Source section on Overview, cover it with TestLive* for the timeline events and the next-action preview (step wait, sending window, paused), and document it in the contacts, campaigns and API reference docs (issue #255) 2026-08-29 03:28:16 -07:00
Matthew Meszaros 27630eec0a feat: add website visitor tracking for issue #255 with migration 000106 (website_tracking_settings, website_visitors, website_page_hits, all registered in the orgtransfer spec), a consent-gated dependency-free tracking.js served by the Rust tracking service with a rate-limited, size-capped, prefetch-filtered POST /p ingest that forwards to a new backend internal page-hits endpoint for server-side user-agent and GeoIP enrichment, contact identification only through the click ticket the redirect appends to registered hosts, a per-workspace retention job, page_hit events with an expandable detail view in the contact Activity timeline, a Settings > Website tracking page for the snippet and consent, location and retention configuration, realtime PAGE_HIT fanout, and a website tracking guide plus endpoint, export and configuration docs 2026-08-29 03:25:50 -07:00
Matthew Meszaros d67cdb8562 feat: add recipient-level engagement to the campaign Leads view by adding an engagement search filter (opened, not_opened, clicked, not_clicked, replied, not_replied, bounced) that composes with lead_status as AND and rejects unknown values with stable 400 codes, counting only human opens so machine opens never read as engagement, adding contacted/opened/clicked/replied_any totals to lead_counts, rendering Opened/Clicked/Replied columns and clickable server-backed status and engagement chips in the Leads table with matching Filters sheet sections, offering lead_status/lead_opened/lead_clicked/lead_replied export columns from a campaign, covering every filter value with a live Postgres test, and documenting the columns, filters, error codes and export fields (issue #250) 2026-08-29 03:12:02 -07:00
Matthew Meszaros 5c832461b0 feat: wire the unreachable reserve send-lifecycle state to a real per-mailbox hold (POST /emails/:id/hold and /release behind manage_emails and WRITE_EMAILS, SetSendHold on the email service forcing the lifecycle past the rebalancer guard, a Hold from campaigns toggle on the mailbox drawer's Overview tab with the reserve notice rewritten to point at it, warmblyctl mailbox hold/release, and docs in the mailboxes guide, API reference and scope map), and drop the warming state nothing ever set from the model, the web type, the docs table and the DB check via migration 000104, which folds any legacy warming row back to active 2026-08-28 22:47:31 -07:00
Matthew Meszaros 9f23c663c4 feat: give a cold mailbox a rotation lifecycle so a tired one can rest (#237)
* feat: give a cold mailbox a rotation lifecycle so a tired one can rest and come back, instead of running at full volume until a hard band trips: send_lifecycle is warming, active, resting or reserve and decides whether cold sender resolution offers the mailbox at all, which is a different axis from risk_band deciding which worker and IP host it, so a resting mailbox is still a clean-band mailbox that keeps its warmup traffic and its reputation; the hourly rebalancer rests a mailbox at throttled and worse but never at watch, since watch is defined as the band that changes nothing a customer can feel and leaving cold rotation is very much something they feel, and a rested mailbox returns only after three clean days so one good hour cannot bounce it back to full volume; reserve is the owner's hold and is never overridden, the default is active so no existing mailbox changes on deploy, and the state never travels in a workspace archive because it is this instance's decision about sending it watched

* feat: stop a query error re-admitting rested mailboxes, make probation measure healthy time, and rotate the candidate window so no mailbox starves: sendLifecycles returned a nil map on failure and an unresolved state reads as active, so one bad query quietly put every resting and reserved mailbox back into cold rotation, and the gate is now applied only when the states were actually read, with the skip logged rather than silent; ReadyToResume measured total time resting, so a mailbox that sat unhealthy for three days resumed on its first healthy tick having served no clean time, and an unhealthy evaluation now restarts the streak; and ordering candidates by send_lifecycle_since put every never-moved mailbox equal-first, so on an install with more than one page of them the same page was re-examined forever, which a checked-at stamp and its index fix
2026-08-28 12:08:52 -07:00
Matthew Meszaros b019174910 feat: give an organization one fused abuse posture (#233)
* feat: give an organization one fused abuse posture, because every existing control watches a single subject and an actor slightly wrong on several axes sits under all of them: organizations gains risk_state, risk_score, risk_reason and an append-only risk_signals evidence blob, modelled on the warmup participant health machine that already works rather than a second vocabulary for the same idea; restricted cuts per-mailbox cold volume to a quarter and forces the free warmup pool so a risky tenant cannot spend the paid pool's shared reputation, suspended stops sending at the send gate, and watch deliberately changes nothing a customer can feel so evidence accumulates before anything is taken away; an operator's suspension outranks the derived band so a detector clearing cannot release a workspace a human suspended, transitions ride the audit spine to every teammate's dashboard, a banner says which limit is active and why rather than letting volume drop silently, and the posture never travels in a workspace archive since it is one platform's verdict reached from evidence the destination never saw

* feat: make the suspension actually stop sending, and emit the audit transitions the PR claimed: emailsend.SendEmail is only the manual and API path, so campaign and warmup sends went nowhere near the gate and a suspended workspace kept sending on its schedule, while the restricted multiplier floored every mailbox at one a day which quietly turned suspension into a trickle rather than a stop; the campaign scheduler now defers the whole campaign with a reason and the warmup task skips as org-suspended, since warmup is outbound mail from the same domains; separately the band change emitted no audit entry at all despite the entity type and the frontend spine entry both existing, so no banner moved for a teammate and there was no trail of who was restricted when, and only a real transition is logged so a detector re-recording the same finding cannot fill the feed; one of my own live tests also asserted how far out a slot lands, which depends on the hour the suite runs, and now asserts the property it was about

* chore: drop a test that belongs to the signup-risk change and reached this branch by mistake, where the type it exercises does not exist
2026-08-28 11:20:19 -07:00
Matthew Meszaros b9cce2fbdc feat: ease a graduating mailbox into cold volume (#231)
* feat: ease a graduating mailbox into cold volume instead of handing it the full cap the day it joins a campaign: cold sending read warmup HEALTH but never whether the mailbox had actually warmed enough, so one at its 40/day warmup ceiling could send 50 cold the next morning, which is the post-warmup spike providers penalise; effectiveCap gains a min() term that starts the mailbox at 5, 10 or 20 a day by how long it warmed and adds 5 per clean day toward its own cap, freezing on a spam placement through the same union-of-freezes the warmup ramp uses, anchored by a new cold_ramp_started_at stamped idempotently on the first cold send and reset on org import because it raises a ceiling the destination never watched being earned; mailboxes that never warmed are not gated, since capping senders who never opted into warmup is a different decision from smoothing the transition out of it

* feat: anchor the graduation ramp on a CONFIRMED send, and make the hold the drawer reports the hold the scheduler applies: stamping cold_ramp_started_at at dispatch started the clock on a send the worker then failed, so a mailbox climbed on days it had not proven anything, and the stamp moves to the worker's EMAIL_SENT; separately the drawer computed its hold over every placement while ColdCeiling only counts placements after the first cold send, so a placement predating the ramp read as paused while the scheduler kept climbing, and both now go through one ColdHeldUntil

* fix: restore the Callout closing tag my conflict resolution dropped, which types:check and lint both pass and only pnpm build catches
2026-08-28 10:47:03 -07:00
Matthew Meszaros ca5acb5aaf feat: score campaign copy as sent, not just as written (#229)
* feat: score campaign copy where it actually matters, on the message the recipient receives rather than only the template in the editor: warmlint.Score already ran at save time through /templates/score, but nothing checked the copy after merge fields, spintax, A/B selection and AI blocks resolved, which is exactly where a clean template becomes Hi comma or picks the one spammy spintax branch; the send path now scores the rendered message and writes one campaign-feed warning per step per day rather than one per recipient, RunPreflight gains the same check across every step, warmlint gains image-heavy, many-images and attachment heuristics, and the whole thing is advisory behind Settings > Sending > Content checks; the launch dialog also finally calls POST /campaigns/:id/preflight, whose report had no UI at all and so never reached the person about to send

* feat: fix three defects the review found in the content lint: a preflight run that could not read the campaign's steps left the worst score at 100 and reported the check as PASSED, which is the one outcome a check that did not run must never produce, so it now reports failed with the reason; the send-time path read organization settings while preflight read campaign-effective ones, so a campaign that turned the check off or moved its floor was still warned, and EffectiveSettings is now exposed and used by both; and the check-then-insert dedupe let concurrent recipients of the same low-scoring step each find nothing and each write, so it becomes one advisory-locked conditional insert proven by twelve racers producing exactly one feed entry
2026-08-28 10:29:39 -07:00
Matthew Meszaros 7a8406e35a feat: close the warmup ramp loop with an early-signal hold (#227)
* feat: close the warmup ramp loop so an early spam placement holds the mailbox instead of letting it climb into the wall: every band in evaluateMetrics needs a sample floor before it can trip (20 warmup sends in 7 days, 100 delivered in 30), which a mailbox in its first fortnight never reaches, so one landing in junk on day three kept adding an email a day until it had sent enough to be judged; a placement inside 48h now cuts the day about a quarter and holds the ramp where it stood, and the resume subtracts the frozen days rather than catching them up because climbing three steps in one morning is the spike the hold exists to prevent; the ramp arithmetic moves into internal/app/warmupramp so the mailbox drawer reports the target the scheduler will actually act on instead of its own private copy, and says which signal cut it and when it resumes

* feat: fix three defects the review found in the warmup ramp hold: the ramp is now elapsed days minus frozen days rather than a level held at the newest placement, because MAX(created_at) meant a second placement arriving mid-hold RAISED the held level and a mailbox landing in junk repeatedly ramped up; the early-signal cut and the health band are resolved in one shared warmupramp.Resolve so the scheduler's health gate and the dashboard's ungated copy can no longer disagree about the same mailbox; and the hold is reported for the whole 72h freeze instead of only the 48h cut window, so a mailbox between the two no longer shows a ramp that has stopped climbing with nothing to explain it
2026-08-28 10:06:19 -07:00
Matthew Meszaros b83a1a3ce5 feat: make send-time optimization actually schedule sends (#226)
* feat: make send-time optimization actually schedule sends, instead of being a documented setting with no caller: advanced.OptimizeSendTime had zero call sites anywhere in the codebase, so an org that enabled send_time_optimization through PATCH /outreach/settings changed nothing about when its campaign mail left, and the API reference said so in a callout; the campaign scheduler now resolves the recipient's timezone from the contact's timezone custom field, then the country-code suffix of its email domain, then the org fallback, and holds the slot until that clock reaches a preferred hour, raising hardFloor so the task handler reschedules rather than sending immediately, the snap is forward-only and never crosses the campaign end date, the default flips to off so no existing workspace silently re-times its sends, and Settings > Sending gives the block its first UI

* feat: stop the recipient-hour gate deferring a send forever, and fix the window parser that silently disabled every campaign schedule: recipientSlot now searches for a moment BOTH calendars accept and yields when they never meet, because raising a hard floor at an hour the sender cannot serve made every tick re-derive it, defer, wake in the sender's window and defer again; separately parseTimeOfDay accepted only the 15:04 layout while start_time, end_time, warmup_start_time and warmup_end_time are Postgres time columns pgx renders as 09:00:00.000000, so every read parsed to 0 and effectiveWindows read that as unconstrained, leaving both the campaign sending window and its day-of-week gate off for every campaign on the legacy fields and pinning warmup to its 08:00-20:00 fallbacks; Settings > Sending also gains the MANAGE_SETTINGS gate a direct visit needs
2026-08-28 09:46:39 -07:00
Matthew Meszaros 0a8c3314d0 feat: stop one undeliverable address from wedging a whole campaign, and stop the pre-send verifier from inventing the verdict that caused it: campaign routing in FindNextRoutedPair now excludes contacts whose address verification refused them (invalid, or risky with the campaign's send-to-risky toggle off) because the pre-send gates in campaign_task.go skip those without recording progress, so the finder re-served the same lead every tick and the campaign never reached the healthy leads behind it; the SMTP prober no longer reads every 5xx reply to RCPT TO as a dead mailbox, since Postfix defers HELO/sender/policy rejections to RCPT time and the old default greeting of localhost made them arrive as 504 5.5.2 Helo command rejected against the recipient, so a 5xx now becomes invalid only when the reply names the recipient and degrades to unknown otherwise; the HELO name falls back to the APP_URL host and the probe is skipped outright without a public FQDN; migration 000095 resets the verdicts the corrected classifier would no longer reach so poisoned contacts are re-verified; and skipped leads now report as undeliverable in the Leads view and in the campaign completion log instead of sitting at Queued forever 2026-08-26 06:03:20 -07:00
Matthew Meszaros efa914025c feat: stop an active campaign sitting at "Queued / Not started" with nothing sending: a campaign is one self-perpetuating task, so a tick that found nothing due parked its successor at the literal next-due moment (three days out for a "wait 3 days" step) and that parked task was also the next time anything re-read the campaign, so leads imported meanwhile stayed invisible until it fired and the reconciler never noticed because it only re-seeds chains with no pending task; deferral parks are now capped at config.CampaignMaxDeferMinutes via scheduler.DeferSlot at all three enqueue sites (a tick that actually sent still parks at its paced interval, so send spacing is untouched), the reconciler re-checks any active campaign parked beyond CampaignStaleParkHours and pulls its wakeup forward when the real next slot is CampaignReparkMarginMinutes sooner, attaching leads to a running campaign wakes it immediately through one CampaignWaker seam in the contact service that covers add/update/bulk-edit/import/Sheets-sync, even distribution now paces across the whole sender pool via poolRemainingOn instead of the one mailbox the tick picked (a three-mailbox campaign was sending at one mailbox's rate), the flat +/-20 minute jitter that was wider than the interval it perturbed is scaled to half the distance to the slot so it stops landing slots in the past where notBefore collapsed them onto the min-gap, and on the dashboard a full-day window renders "12am-midnight" instead of "12am-12pm", the campaign lead strip uses the server's campaign-wide lead_counts instead of counting the 50 loaded rows, and channel state moves out of a ref into React state so a live campaign's panel stops reading "Disconnected" forever 2026-08-25 07:39:18 -07:00
Matthew Meszaros cbf4190f42 feat: make a custom tracking domain verifiable instead of permanently "Pending DNS": the CNAME target is now this install's TRACKING_DOMAIN rather than a hardcoded t.warmbly.com that resolves nowhere, matching is exact on the label boundary (or on shared addresses, so a provider-flattened CNAME stops reading as no record at all) instead of strings.Contains, and every outcome carries the reason plus what DNS actually returned, including when the tracking host the customer is told to point at has no record of its own; a pasted URL is normalized to its host and a malformed one is rejected up front instead of saved and left pending forever; only a VERIFIED mailbox domain is used at send time with the shared host as the fallback and a campaign-feed entry saying why; pixels and click tickets are built from the configured host, and with none configured mail ships untracked rather than carrying links to another deployment's tracking service; adds GET /emails/:id/track and POST /emails/:id/track/verify plus an hourly re-resolution sweep so a record that propagates later starts being used and one that breaks stops routing links; and scopes the tracking-domain write by organization like the read, which also fixes GET /emails/:id passing a user id to an org-scoped query and 404ing for every caller 2026-08-24 09:02:29 -07:00
Matthew Meszaros 07f671150b feat: show the new failed lead state in the campaign Leads view (pill with the worker's reason on hover, strip chip and counts) and toast user-addressed mailbox and send errors from the realtime ERROR event, which previously only refreshed the dashboard so a reply that could not be sent was never surfaced 2026-08-23 10:18:50 -07:00
Matthew Meszaros 3739a36b67 feat: enforce the persisted SPF/DKIM/DMARC state as a real cold-send and warmup gate behind a 72h grace clock and an operator toggle, after first fixing the DMARC organizational-domain fallback in dnsauth so a dedicated sending subdomain covered by its parent's record stops reading as unauthenticated, adding auth_state to the four mailbox loaders that never selected it (which would have made the gate dead code), stamping auth_failing_since on entry to failing so a resolver hiccup can never stop a campaign, notifying the org on that transition, and reporting an all-gated pool as ErrDomainAuthFailing instead of a message about sending windows (#160) 2026-08-22 09:37:26 -07:00
Matthew Meszaros 94cf21d95e Fix truncated, unformatted and mis-encoded email content in the unibox (#137)
* feat: add internal/pkg/mailhtml, a mail-oriented HTML sanitizer and text flattener, because rendering a received message body means rendering the sender's markup: Sanitize builds on bluemonday's UGC policy but keeps what real email is made of (table layout attributes, inline CSS through the property-allowlisted style sanitizer, legacy font/center, data: and https: images) while dropping script, iframe, object and the text content of style/head blocks so a marketing email's stylesheet cannot render as body copy, forcing target=_blank plus nofollow/noreferrer on links and allowing only http, https, mailto and tel; ToText flattens the same input for previews, turning block boundaries into newlines and decoding entities back to the characters they stand for so an already-escaped body does not surface as literal &; LooksLikeHTML reports whether a stored body is actually markup, which is how a body recorded as HTML by an older sync but containing no tag at all can be recognised as the plain text it really is

* feat: add internal/pkg/mailhdr for RFC 5322 header values, since headers are ASCII on the wire and every transport was writing raw UTF-8 into Subject and display names: Subject and AddressList RFC 2047-encode non-ASCII (a no-op on plain ASCII, and a bare address stays bare rather than being wrapped in angle brackets), DecodeWords reverses encoded-words with a charset hook wired to go-message so legacy encodings Go does not handle natively still decode, and Bare/BareList strip a display name down to the routable address for SMTP envelope commands where 'Ana <a@b.com>' in RCPT TO is a syntax error, promoting go-message from an indirect to a direct dependency

* feat: encode outbound Subject and address headers on all three transports, so a subject or sender name containing an accent, a currency sign or an emoji reaches the recipient as the characters the user typed instead of mojibake: SMTP and Graph were writing the raw string into Subject (only the Gmail transport encoded it) and Graph built its From by fmt.Sprintf rather than mail.Address, so a non-ASCII display name went out unencoded there too, and all three joined To/Cc/Bcc entries verbatim so an encoded display name never appeared even when the caller supplied one; additionally the SMTP envelope now takes bare addresses through mailhdr.BareList, because an API caller may pass 'Name <addr>' (the compose handler has a bareAddress helper precisely because that arrives) and passing that to RCPT TO gets the recipient rejected by the server

* feat: rewrite the IMAP body reader, which was the reason received mail from SMTP/IMAP mailboxes came back corrupted: it built one FetchItemBodySection with a hardcoded Part []int{1} and a comment saying it would adjust when recursing, which it never did, so on a multipart/alternative the text/plain bytes were fetched twice and the second copy was stored as the HTML body (plain text rendered as markup loses every line break, shows & as an entity and swallows anything inside angle brackets), and decodeIfNeeded never reversed Content-Transfer-Encoding at all, leaving quoted-printable bodies full of =E2=80=99 runs and = soft breaks and base64 bodies unreadable, while its charset detection parsed params off a media-type string that never carried any and its mail.ReadMessage call could silently eat leading body lines as headers; the reader now walks the body structure for real part paths, fetches every text leaf in a single FETCH with a server-side Partial size cap, decodes quoted-printable and base64 (tolerating a tail cut mid-quantum by the cap) then converts the part's charset to UTF-8 with go-message, skips attachment-disposition parts so a .txt attachment cannot stand in for the body, takes one part per type inside a multipart/alternative but treats sibling inline parts in mixed/related as additive, and is bounded at five text parts per message; the stored body cap also goes from 200 KB to 512 KB because 200 KB cuts real HTML newsletters mid-document

* feat: decode Gmail's raw headers and entity-escaped snippets, because the Gmail API hands header values back exactly as they arrived on the wire, so a message from a sender whose subject or display name was RFC 2047-encoded showed in the dashboard as =?utf-8?q?caf=C3=A9?= rather than as the text it stands for, and the API's own snippet field is HTML-escaped, so a preview containing an apostrophe surfaced as &#39; in the conversation list and, until the thread reader stopped rendering snippets as message bodies, inside the message itself; getSingleHeader now runs values through mailhdr.DecodeWords (a no-op unless the value actually contains an encoded-word, so Message-ID and the warmup token header are untouched), the comma-split fallback in getAddressList does the same for display names net/mail could not parse, and the snippet is unescaped once on the way in

* feat: fix the conversation-list snippet, which collapsed whitespace before splitting on newlines so the quoted-line and signature filters below it could never match a thing, stripped HTML with bluemonday's strict policy and then showed the escaped output verbatim so an ampersand in an HTML-only message read as &amp; and a marketing email's stylesheet text rendered as body copy, and cut at 100 bytes with text[:100] so a multi-byte character or emoji at the boundary became a replacement glyph; it now flattens HTML through mailhtml.ToText (entities decoded, style and script content dropped) including when a sender puts markup in their text/plain part, filters quoted history and everything past the RFC 3676 signature delimiter while the text still has lines, collapses whitespace afterwards, and truncates on a rune boundary at 200 characters

* feat: make GET /unibox/:id serve a display-safe body and stop it failing outright, sanitizing body_html through mailhtml before it leaves the API so every consumer gets markup that cannot execute rather than each call site having to defend itself, degrading a body blob that cannot be read to the message's preview text with a new body_truncated flag instead of returning 500 (which made a message with a missing blob unopenable, and hit every seed, sandbox and dev-history fixture row since only the '<seed-' prefix was recognised while the sandbox uses '<sbx-' and dev history '<dev-'), and treating a stored HTML body that contains no tag at all as the plain text it really is, because mail synced before the IMAP reader addressed parts individually recorded the plain part under both bodies and serving that as HTML is exactly what collapsed a ten-line message onto one line

* feat: escape composer text before turning it into the HTML part of an outgoing email, replacing body_html: trimmedBody.replace(/\n/g, '<br />') in both the compose window and the reply composer with a shared plainToHtml that escapes the five markup characters first, so an email containing 'Terms & conditions' no longer ships a broken entity and one containing anything in angle brackets ('<see attached>', 'a < b', a pasted tag) no longer has the rest of the paragraph swallowed by the recipient's mail client as an unclosed tag, while runs of spaces survive as non-breaking spaces and bare URLs become links without eating the sentence punctuation after them; the same unescaped plain-to-HTML pattern in the campaign step editor's applyTemplate now goes through promptToHtml, which escapes as it paragraph-wraps

* feat: render the real message body in the unibox thread reader instead of the list preview, which is the whole of the reported bug: ThreadView mapped each thread row to a UniboxEmail whose body was '<p>' + escapeHtml(m.snippet) + '</p>' and MessageBubble rendered that as the message, but a snippet is a preview capped at 100 characters with every run of whitespace collapsed to one space, so a ten-line email displayed as roughly two lines on a single continuous line, and Gmail's already-escaped snippet was escaped a second time so an apostrophe read as &#39;; each expanded message now loads its own body from GET /unibox/:id (the newest message and anything unread open on mount, older messages collapse to their preview line so a long thread does not fetch every body at once) and renders it in a sandboxed iframe carrying no allow-scripts, which keeps a sender's stylesheet from restyling the dashboard and means nothing in the message can run even though the API already sanitized it, sizing itself from the inner document as images load, with the preview kept as the fallback when a body cannot be fetched and a notice when only a preview is stored

* feat: document how a message body is read and returned, adding a 'Reading a message' section to the unibox guide covering the expand-on-open behaviour, that formatting and special characters are preserved as sent, that the conversation list preview is a summary and not the message, and that HTML mail renders in an isolated frame with links opening in a new tab, plus a paragraph in the API endpoint reference stating that GET /unibox and GET /unibox/thread return previews carrying snippet while GET /unibox/:id returns body_plain and a sanitized body_html, and what body_truncated means

* feat: add email_accounts.save_to_sent, the per-mailbox switch for filing a copy of outbound mail in the Sent folder, defaulting on because plain SMTP submission leaves nothing behind in the sender's account while Gmail and Outlook file their own copy through their APIs, making it a per-mailbox choice rather than a global one since a submission server that files the copy itself (Gmail's SMTP, Fastmail, Zoho) would otherwise end up with two of everything, which is exactly why every desktop mail client ships the same switch, and wiring the column through the Email model, the mailbox read paths and UpdateEmail so it is readable and writable from the dashboard and the API

* feat: teach the IMAP client to APPEND a sent message and the SMTP client to hand back the exact bytes it submitted, the two transport pieces the Sent-folder copy needs: AppendToSent resolves the folder from the RFC 6154 \\Sent special-use attribute first (requesting it only when the server advertises SPECIAL-USE) and falls back to matching the known names against both the full mailbox name and its leaf, since servers namespace as INBOX.Sent and localize the label, caches the result for the life of the connection, files the message flagged \\Seen and dated when it was sent, and returns a sentinel rather than an error when the account has no Sent folder at all; APPEND addresses its mailbox by argument and never touches the selected mailbox, so unlike the warmup MOVE/STORE actions it is safe to run while the sync loop is mid-fetch on the same connection

* feat: file a copy of every SMTP send in the mailbox's Sent folder, closing the gap where a message sent from Warmbly through an SMTP/IMAP mailbox existed only in the recipient's inbox: nothing appeared in the customer's own mail client, and nothing appeared in the unibox either, whose thread reader can only show messages the sync found in a folder, so a user who sent from the dashboard and then went looking for what they sent found no record of it at all; the worker now APPENDs the exact bytes the SMTP client submitted after a successful send, best effort so a failed append never turns a delivered message into a failed task, skipping warmup traffic because filing dozens of machine-generated messages a day would bury the customer's real sent mail, and skipping Gmail and Graph mailboxes entirely since their APIs file their own copy; the per-mailbox setting rides along on the add-email worker payload as a pointer so an older control plane that does not send the field is read as unset and takes the default rather than as an explicit no

* feat: expose the Sent folder copy as a mailbox setting in the dashboard, adding a 'Keep a copy of sent mail' toggle to the Settings tab of the mailbox drawer that only renders for SMTP/IMAP mailboxes (Gmail and Outlook file their own copy, so the control would be a lie there), tracked by the drawer's save bar alongside the other editable fields, and worded so the one case where it should be turned off is obvious: a provider that already saves its own copy, where leaving it on means seeing every sent message twice

* feat: document the Sent folder copy in the mailboxes guide and the API reference, explaining why the toggle exists at all (SMTP submission leaves nothing in the sender's own account, so without it a sent message shows in neither the customer's mail client nor the unibox thread), when to turn it off (a provider such as Gmail, Fastmail or Zoho that already files its own copy of anything submitted over SMTP, where leaving it on doubles every message), that OAuth Gmail and Outlook mailboxes never show the control because their APIs file the copy themselves, that warmup traffic is deliberately excluded, and that PATCH /emails/:id takes save_to_sent

* feat: add unibox_emails.body_text and its search index, because unibox search ran against search_tsv, a generated column built from subject and snippet, and a snippet is a truncated one-line preview, so searching for a phrase that appears in the third paragraph of an email returned nothing at all and read as broken search rather than as search that only covers the first line; message bodies stay in object storage where they belong, and what lands in Postgres is a bounded 16 KB plain-text rendering carried on the new-email worker event, indexed with a GIN expression index rather than a second stored generated column since adding one of those rewrites the whole table while this builds against a column that is empty on every existing row

* feat: index what a message actually says, adding mailhtml.SearchText (HTML flattened, entities decoded, whitespace collapsed, quoted history deliberately kept because a phrase someone quoted back at you should still find the conversation, truncated on a rune boundary) and computing it on all three sync paths so IMAP, Gmail and Graph mail all arrive with searchable text, writing it on insert, and widening the unibox search filter to match either the existing subject-and-preview vector or the body expression, written exactly as the new index declares it so the index is actually used

* feat: backfill the searchable text of messages that were synced before bodies were indexed, so search covers the archive a customer already has instead of only mail that arrives from now on, which would have made the feature useless on day one for exactly the people who need it; the sweep pages through unibox_emails by id, reads each body from object storage under the mailbox owner's key, renders it with the same helper the sync path uses and writes it back, at 100 rows per 30 seconds because nothing waits on it, and returns for good once a pass finds nothing left to visit, with rows whose stored body really is empty simply revisited after the next restart rather than needing a tried-and-failed marker in the schema

* feat: document that unibox search now covers message text and not just subjects and previews, in the search paragraph of the unibox guide where the old wording only promised that search stays inside the current scope

* feat: add generation.RenderThread, the shared way to put a conversation in front of a model, because every AI surface was grounding on preview snippets and a draft written from the first hundred characters of each email answers the greeting rather than the question; it strips quoted history and signatures (the earlier messages are already in the prompt on their own, so quoting them again spends the budget twice, though a reply written underneath the quote is kept rather than thrown away when there is nothing meaningful above the attribution line), spends a bounded character budget newest-message-first since the message being replied to matters most, degrades older messages to their preview line instead of dropping them once the budget runs low, and renders oldest-first so the transcript reads in order

* feat: add grounding reads to the unibox service and repository, returning message text (the stored body, falling back to the preview for mail synced before bodies were indexed) for a thread or for all correspondence with one address, kept deliberately separate from the preview queries and given their own result type so a 16 KB body can never leak into a list response by accident, capped at twenty messages whatever a caller asks for, and paired with a RenderGrounding helper so every AI surface formats a conversation the same way instead of each one rolling its own transcript loop

* feat: ground every AI writing surface in what the messages actually say, switching the unibox reply draft, the compose draft's correspondence history, the inbox agent's thread history and the assistant's read-thread tool from preview snippets to real message text through the new grounding reads, which is what makes a drafted reply answer the question that was asked rather than the first sentence of the email; the inbox agent's triviality gate also reads the reply's full text now, since a preview line cannot tell a one-word ack apart from a long message that happens to open with one, and the assistant tool returns a bounded body per message with quoted history stripped instead of a snippet field

* feat: say in the docs that AI drafting reads the messages and not their previews, in both the unibox reply-draft section (adding that quoted history is stripped and the newest messages get the most room, so a draft answers what was asked rather than the opening sentence) and the inbox agent's grounding section, where 'the full thread so far' was true of the message list but not of how much of each message the model actually saw

* feat: renumber the two new migrations to 000087 and 000088 after rebasing onto main, which landed its own 000085 (org data transfer) and 000086 (email sync state) in the meantime, so the sequence has no duplicate versions

* feat: add the two new API fields to the OpenAPI spec that landed on main while this branch was open, documenting save_to_sent on the Mailbox and MailboxUpdate schemas and body_truncated on UniboxEmail, and saying on body_html that what the API returns is already sanitized so a client can render it directly
2026-08-21 18:45:09 +02:00