269 Commits

Include renames

Author	SHA1	Message	Date
Matthew Meszaros	1a7fd85f4b	feat: drop the throwaway migration harness that a blanket git add swept into the placement commit, since it only existed to apply migrations against a scratch database while verifying the branch	2026-09-09 04:56:07 -07:00
Matthew Meszaros	435dbb522f	feat: replace the worker tier/type/risk-pool/egress categories with a scored placement model and make the fleet pull-based, so a machine joins with one command, workers and consumers share one node registry with usage and liveness, nodes self-update to the version the control plane resolves, and the Hetzner provisioning, worker profiles and SSH orchestrator are removed	2026-09-09 04:54:01 -07:00
Matthew Meszaros	9ce1e060dd	feat: fix the campaign step body editor for issue #380 by registering TipTap's UndoRedo and HardBreak so Ctrl+Z and Shift+Enter work at all, normalising pasted HTML from Gmail/Outlook/Word so their blank-line scaffolding stops rendering a second gap on top of our paragraph margins, adding a workspace email image library (migration 000139, public email-images/ objects sharing the attachment storage quota, keyset-paginated /email-images endpoints) with toolbar upload, URL and library insertion, drag-drop, clipboard paste and a size/alignment/alt bubble, and adding an HTML source view that names the tags the schema would drop before switching back	2026-09-08 21:00:09 -07:00
Matthew Meszaros	29b479321a	feat: merge main and renumber the campaign entry-delay migration to 000138, since main released 000137_contacts_org_created_index while this branch was waiting to merge	2026-09-08 09:59:43 -07:00
Matthew Meszaros	c8042e2942	Merge remote-tracking branch 'origin/main' into feat/campaign-entry-delay	2026-09-08 09:58:38 -07:00
Matthew Meszaros	a62652de3a	feat: merge main and renumber the contacts index migration to 000137, since main released its own 000136 for lead sync segments while this branch was in review and two branches that each take the next number are green alone but leave golang-migrate refusing to build its source driver once both land	2026-09-08 08:34:52 -07:00
Matthew Meszaros	37ed967919	Merge remote-tracking branch 'origin/main' into fix/issue-382-user-experience	2026-09-08 08:32:31 -07:00
Matthew Meszaros	073bb3c6bc	feat: merge main and renumber the campaign entry-delay migration to 000137, since main released 000136_lead_sync_segments while this branch was open and a duplicate version makes golang-migrate refuse to build its source driver, which is what failed the Migrations job and the check-migrations prerequisite inside make lint on Go CI	2026-09-08 07:33:21 -07:00
Matthew Meszaros	4e52e9476b	Merge remote-tracking branch 'origin/main' into feat/campaign-entry-delay	2026-09-08 07:31:31 -07:00
Matthew Meszaros	ed50ad9f2f	feat: add a campaign entry delay so a contact's first email can wait a set time after they enter the campaign, with campaigns.entry_delay_minutes and a campaign_leads.added_at anchor (migration 000136), the delay applied in the router's per-lead due check and floored into the placer through ContactSequencePair.NotBefore, a distinct entry_delay constraint in the contact next-action preview, and the control surfaced on the Schedule tab, a new Trigger card at the top of the Steps canvas, the campaign wizard's Schedule step, the launch dialog, the update_campaign AI tool and the iOS schedule page, plus guides, API reference and live scheduler and repository tests	2026-09-08 05:06:04 -07:00
Matthew Meszaros	55894a0e9d	feat: fix the campaign Leads tab and contact export stopping partway through a large list by rebuilding the contacts keyset cursor: the boundary subquery named the outer row's alias so Postgres read it as a correlated self-reference and the whole comparison collapsed to c.id >= <cursor uuid>, serving each page the newest rows of a randomly shrinking id range until it ran out at a third of the leads, so the opaque token now carries the ordering it was taken under plus the boundary row's own sort value and id, rejects a token replayed under another sort, and pairs with a direction-following id tiebreak, an ORDER BY that names its NULL placement, a new (organization_id, created_at DESC, id DESC) index, EXISTS campaign and category filters and a lateral campaign_count the query only joins when a filter or sort asks for it, taking a page of a 50k-contact organization from 120ms to 3ms, and the dashboard now keeps the rows it already loaded when a later page fails and says how far through the list Load more is	2026-09-08 05:01:15 -07:00
Matthew Meszaros	6284eb27ec	feat: give a saved Google Sheets sync source segment targets (migration 000136 adds lead_sync_sources.segment_ids) that pin every synced row into those segments on each run, validated against the organization when the source is written and dropped from the run when a segment is deleted later, and filter GET /lead-sync/sources by ?segment_id= through a jsonb containment test so a segment can list the sources feeding it	2026-09-08 04:59:18 -07:00
Matthew Meszaros	c26300ae5e	feat: backend half of the admin panel upgrade: delete the unrouted provisioning, releases, plan, discount and enterprise-inquiry admin handlers with their service and repository methods, retire the six admin permission bits nothing gated as reserved placeholders so live bit positions and existing masks are unchanged and IsSuperAdmin checks the live set, add forty admin endpoints for mailbox sync governor state with clear-throttle and restart-backfill, in-flight send reservations, cross-workspace dead letters with replay, task failures, webhook delivery health with reclaim, fleet capacity, the control loops decision log, dedicated bindings with release and the routed convert-to-dedicated, operator-driven workspace export and import, per-organization API keys and webhooks, warmup invalid-token abuse and action history, and signups by acquisition channel, and add a scheduled_job_runs table (migration 000135) with a jobrun package that every backend and consumer loop now records through and a run-now request the owning process picks up within fifteen seconds	2026-09-07 21:40:38 -07:00
Matthew Meszaros	769a05aa90	feat: identify an IMAP folder by its name rather than by its UIDVALIDITY, which RFC 3501 never promised was unique across folders, so a mailbox on a server that stamps that number with the folder's creation time no longer loses the entire sync of every folder in a tree created in the same second, with the folder row keyed on (email_id, mailbox), each stored message stamped with its folder's name alongside the UIDVALIDITY generation its uid belongs to, a rename followed as a move that carries the mail and the cursor instead of orphaning both, and a changed UIDVALIDITY treated as what it is, the cursor going void	2026-09-07 08:21:17 -07:00
Matthew Meszaros	04dc83da38	Merge the Sentry review fixes into the PostHog branch, keeping the new JavaScript escaping for the PostHog runtime-config values too	2026-09-07 04:48:30 -07:00
Matthew Meszaros	f043bfc761	feat: address the CodeRabbit review on the Sentry PR by escaping every runtime-config value the web and admin entrypoints write into a JavaScript string literal so a quote or newline in a DSN cannot inject or discard the whole config, reporting only the count and types of a failed query's parameters instead of their values now that the db scope actually applies, cloning the fallback hub in errs.Hub so scope cannot leak between requests, flushing the tracking service before every fatal exit since process::exit skips the guard's drop, treating a blank WARMBLY_RELEASE or APP_ENV as unset in realtime, stamping the environment and a service tag into the forms page, and correcting the docs claim that an unset DSN means the SDK is never initialised	2026-09-07 04:47:51 -07:00
Matthew Meszaros	6f230a3e17	feat: renumber the organization_acquisition migration from 000132 to 000133 after main landed its own 000132 for unibox mailbox uid_next, since two branches that each took the next number are green alone and collide once both merge	2026-09-07 04:27:02 -07:00
Matthew Meszaros	d00960ad59	Merge feat/sentry-everywhere (with main) into the PostHog branch	2026-09-07 04:26:17 -07:00
Matthew Meszaros	be16f77f5d	Merge main into the Sentry branch	2026-09-07 04:25:41 -07:00
Matthew Meszaros	8e9f67f46b	feat: add cookieless PostHog analytics for the hosted marketing site and dashboard with server-side signup and subscription events, a first-party acquisition record written once at signup on a new organization_acquisition table registered in the org-transfer spec, an acquisition column and channel filter in the admin org list, and never a single request from a self-host because every key is unset by default	2026-09-07 04:18:33 -07:00
Matthew Meszaros	51dedc90ee	feat: put every runtime behind one optional error-reporting story: a single internal/observability/errs wrapper that is now the only package importing sentry-go, InitSentry for cmd/forms, release and environment tags on every service from the existing build stamp, optional Sentry in the admin panel and the public forms app, the sentry crate in the Rust tracking service, release tagging in realtime, CI source-map upload that only runs when a Sentry token is configured, and docs covering the DSN for each service	2026-09-07 03:51:06 -07:00
Matthew Meszaros	0df1a57589	feat: renumber the unibox_mailboxes uid_next migration from 000131 to 000132 after the campaign continuous lead sources migration took 000131 on main, so golang-migrate does not refuse to build its source driver once both are merged	2026-09-07 03:15:06 -07:00
Matthew Meszaros	1a87720d5a	merge	2026-09-07 03:14:42 -07:00
Matthew Meszaros	f2c35cf872	feat: sync every kind of IMAP mailbox (issues #345-#349): connect servers without CONDSTORE by following UIDNEXT per folder and mirroring read state with a periodic flag scan, so Outlook.com, Microsoft 365 over IMAP and Yahoo work instead of failing at load; fall back to a STATUS per folder when the server has no LIST-STATUS, which silently made an account look empty; bound the wait between IMAP responses so a peer that vanishes without a FIN cannot park a command forever; keep the inbox and special folders when a mailbox has more folders than the cap (raised to 100) and relay the overflow as a warning instead of failing the mailbox silently; recognize localized folder names and the server's own hierarchy delimiter so a Sent folder called Gesendete Elemente is not filed as inbox; and back off on a widening interval while a mail server is unreachable so one outage is one warning rather than one a minute	2026-09-07 03:12:22 -07:00
Matthew Meszaros	5d60fe9a21	feat: fix campaign restart dead end (issue #340): resolve preflight tracking senders through the same explicit/tags/all pool the scheduler uses so a connected mailbox is never reported missing, fetch the launch dialog's step count from the steps endpoint instead of a field the campaign API never returns, turn on Keep running for new leads when a form or an automation feeds a campaign (migration 000131 backfills existing ones) and when a member starts a campaign whose every lead has finished so it goes active and waits for leads instead of answering 400, return waiting_for_leads from the start endpoint for the dialog's success screen, and document the no_leads and no_remaining_leads codes and the new behaviour in the campaigns, forms, automations and API docs	2026-09-07 02:26:50 -07:00
Matthew Meszaros	7b11489c85	feat: keep a campaign running for new leads (issue #336): add a continuous campaign setting (migration 000130, on by default once a segment is linked) under which a campaign that runs out of leads stays active and idle with idle_since set instead of finishing, is startable with no leads, is re-checked by the reconciler every pass and shows as waiting for leads in the dashboard list, header and live activity panel with a CAMPAIGN_IDLE realtime event; make WakeCampaigns restart a finished campaign through the full launch checks so a lead added by the contacts API, a bulk update, an import or the add-to-campaign automation action reopens it exactly like a linked segment does, and write a refused restart to the campaign activity log once an hour per reason instead of only the backend log; expose continuous on the campaign API and preferences page and document it in the campaigns, segments, forms, automations, API reference and realtime docs	2026-09-06 02:50:16 -07:00
Matthew Meszaros	04a2ed59c9	feat: address the CodeRabbit review on the lead-intake PR by giving the webhook sink its own copy of every map payload so the automation goroutine never writes into the map still being marshalled for endpoint delivery, propagating contact lookup errors in the upsert action's skip mode and in the native actions adapter so a failed lookup is never mistaken for a missing contact, adding the contacts source CHECK as NOT VALID in migration 000129 to skip the table scan under the migration lock, defaulting a new create-or-update-contact node's email template to the address key the chosen trigger actually carries, correcting the n8n guide on which lead-form sources have a trigger node, naming API batches over 100 contacts among the silent bulk arrivals in the contacts guide, and updating the marketing automations page from nine to twelve triggers	2026-09-06 00:47:33 -07:00
Matthew Meszaros	a3989f9d9c	feat: turn automations into a lead-intake path so a Facebook, Instagram, LinkedIn or TikTok lead form pushed by Zapier, Make, n8n or any webhook becomes a tagged, campaign-enrolled contact without leaving Warmbly: add the warmbly.upsert_contact and warmbly.add_to_campaign built-in actions with templated field mapping, custom fields, tags, campaign and an if-exists policy, fire a rich contact.created event from the one contact write path (silent for file imports, sheet syncs and API batches over 100) and expose contact.created and form.submitted as automation triggers with condition fields, variables and sample data, carry an automation depth through events raised by an action so a flow that creates a contact cannot re-trigger itself past five hops, stamp automation-created contacts with the new automation source (migration 000129), share the campaign picker between the sheet sync wizard and the automation builder, document lead intake in the automations, Zapier, Make, contacts, forms, integrations, expressions and webhook pages plus a new n8n guide, mirror the new triggers and actions on the marketing automations page, and drop the 34 MB cli binary that was committed by mistake	2026-09-06 00:36:05 -07:00
Matthew Meszaros	028689fd2e	feat: device-code sign-in for the CLI, with a browser approval page at /cli that mints a scoped API key, self-revocation at DELETE /api-keys/self so a read-only credential can always end itself, and app_url plus websocket_url on /auth/config so a client can find the dashboard and the realtime gateway on a self-hosted layout	2026-09-04 20:14:27 -07:00
Matthew Meszaros	37d328c86a	feat: add a partial index on campaign_contact_progress.dispatch_task_id so the daily send counters can look a task's reservation up without scanning progress	2026-09-03 20:18:43 -07:00
Matthew Meszaros	701dc404a2	Merge remote-tracking branch 'origin/main' into fix/click-without-open-tracking	2026-09-03 05:22:05 -07:00
Matthew Meszaros	e97477a718	feat: renumber the engagement origin and pending index migrations to 000125 and 000126 because main released 000124 for unsubscribe opt-out while this branch was open	2026-09-03 05:22:05 -07:00
Matthew Meszaros	4c7229c7d6	feat: make a held-back click announcement retryable until its effects ran: the claim leases the click row for one attempt via announce_claimed_at and the flag clears only after the effects complete, the sweep retries expired leases as well as clicks a restart cut off, and the partial index for the sweep moves to its own concurrent migration 000125 so the live click table is never write-blocked	2026-09-03 04:34:30 -07:00
Matthew Meszaros	a3264f0d39	feat: address the review on the rebuilt engagement branch: a deferred human click's effects are now durable and once-only (announce_pending on the click row written before the event is marked processed, a claim in finishHumanClick and a minute-by-minute sweep that finishes what a restart cut off, a burst relabel clears the flag), the tracking service keys the source-address token with TRACKING_IP_HASH_KEY so it cannot be enumerated back into an IPv4 address, the migration no longer adds a blocking index to the existing click table, and timeline click and open rows carry task_id	2026-09-03 04:13:39 -07:00
Matthew Meszaros	31dabea0a4	feat: rebuild the click-without-open fix on top of the per-link click attribution from #298: a person's click now also counts as an open and a burst that withdraws the click withdraws the open it implied unless a real open is on record, routing readers ignore machine opens as the docs promised, every open gets its own log row and every open and click records the mail client or proxy, browser, device, OS, country, region and city (migration 000124: origin columns on email_link_clicks plus an email_opens table), the tracking service publishes only the address's network in a nullable client_ip field which the consumer resolves with GeoLite and drops, the contact Activity tab shows each open and the origin of opens and clicks, the campaign overview gains a who-engaged-from-where breakdown exposed as engagement in campaign analytics, live open and click events carry occurred_at, client and location, the leads table explains why an open is not always counted, both logs are pruned daily after a year, email_opens joins the export registry, the consumer reads GEODB_PATH optionally, and the guides and API references are updated (fixes #294)	2026-09-03 03:49:44 -07:00
Matthew Meszaros	579b0ac04c	feat: merge main into the unsubscribe branch again, carrying the UTM campaign columns alongside unsubscribe_mode in every campaign scanner, keeping unsubscribe links out of the moved link tracker and its UTM tagging, and renumbering the opt-out migration to 000124 because main released 000123 for link clicks	2026-09-03 03:43:47 -07:00
Matthew Meszaros	5fcb0fe882	feat: fold existing suppression rows to lowercase in migration 000123 and compare stored values as equalities so the existing unique key is the single case-insensitive identity and no extra index build is needed, and keep the plain-text alternative when a chosen A/B variant carries HTML only	2026-09-03 03:22:37 -07:00
Matthew Meszaros	78c4021053	Merge remote-tracking branch 'origin/main' into feat/link-click-attribution ... # Conflicts: # docs/content/docs/api/reference/campaigns.mdx # docs/content/docs/guides/campaigns.mdx # internal/repository/pg_campaign.go # internal/repository/pg_campaign_lifecycle.go # web/src/components/app/campaigns/NewCampaignDialog.tsx	2026-09-03 03:14:19 -07:00
Matthew Meszaros	3614d65d96	feat: renumber the link clicks migration to 000123 because main released 000122 for the campaign kind column first	2026-09-03 03:13:30 -07:00
Matthew Meszaros	64f2d1637b	feat: address the review on the unsubscribe PR: register the suppression family in warmblyctl, require the confirm field on the browser unsubscribe POST and cap its body, render a chosen A/B variant through the template engine so its merge fields and unsubscribe link resolve, fold curly apostrophes before opt-out phrase matching, write pasted suppression lists in one transaction, clamp copy by runes instead of bytes, add the constraints NOT VALID plus a lower(email) index in migration 000123, scope the unsubscribe link type-ahead to email bodies, and document DELETE /suppressions/:id	2026-09-03 02:20:47 -07:00
Matthew Meszaros	9485503e03	feat: merge main into the unsubscribe branch, keeping the new campaigns.kind column alongside unsubscribe_mode in every campaign scanner and renumbering the opt-out migration to 000123 because main took 000122 for campaign kind	2026-09-03 01:56:38 -07:00
Matthew Meszaros	5eb92c601c	feat: give every campaign email a working opt-out: a reply-to-opt-out line by default or an unsubscribe link (workspace setting under Settings > Sending with a per-campaign override and a {{.UnsubscribeLink}} variable), signed per-recipient unsubscribe links served on the API origin so the List-Unsubscribe header no longer points at a dead warmbly.com page, a confirm page on GET with RFC 8058 one-click on POST and a resubscribe button, reply opt-out detection through the whole-word compliance lexicon with quoted history stripped, a first-class suppression list (Contacts tab, GET/POST/DELETE /suppressions with address and domain entries, audited removal, contact drawer action), the contact Subscribed flag enforced in campaign routing, migration 000122 with a shared recipient_suppressed() predicate, and docs for all of it	2026-09-03 01:52:40 -07:00
Matthew Meszaros	24b7e26fbb	feat: add the one-time email campaign preset from #288: campaigns.kind column with sequence and one_time values and a kind field on POST /campaigns, GET /campaigns?kind= and an one_time count on /campaigns-overview, a POST /campaigns-estimate endpoint projecting deduplicated segment recipients against the mailbox pool's per-day capacity and finish date, a wizard type picker whose one-time flow runs Basics, Email, Audience, Sending and Send with send-now or scheduled start and the estimate panel and then creates, links segments and starts the campaign, a guard refusing a second email step on a one-time campaign, draft/scheduled/sending/sent wording with a One-time badge and Type filter in the campaigns list, kind on the AI create_campaign_draft tool, honouring text_only in the send and test-email paths so plain-text campaigns ship no HTML part or tracking, and the campaigns, segments, API reference and endpoint docs	2026-09-03 01:42:00 -07:00
Matthew Meszaros	99be92f159	feat: attribute every campaign click to the exact link with a per-link click log (email_link_clicks) behind the contact activity timeline, campaign live feed, recent activity and the email_clicked webhook, add per-campaign automatic UTM tagging (utm_tracking with source, medium and campaign overrides, utm_content from the link text) applied at send time to the stored ticket destination, and classify opens and clicks as machine when they arrive within ten seconds of dispatch or when several links of one email are followed within five seconds, so scanner clicks are logged but never count as engagement, fire automations or send webhooks	2026-09-03 01:05:34 -07:00
Matthew Meszaros	290f1f00aa	Merge remote-tracking branch 'origin/main' into feature/public-forms ... # Conflicts: # cmd/backend/main.go # docs/content/docs/guides/workspace-export-import.mdx	2026-09-01 09:29:53 -07:00
Matthew Meszaros	05d74dbe9f	feat: forms v2, a full redesign of the hosted form and its builder: eight one-click themes, card/wide/split layouts with a brand cover panel, classic paged and Typeform-style focus modes split on a new page_break block, logo/cover/background uploads with size, fit and veil controls, an optional header bar that can span the page or sit with the form, a real HSV colour picker and font preview, per-contact personalized links that prefill and attribute without email verification, a render-token gate so the form JSON cannot be scraped without loading the page, funnel analytics with per-page drop-off and identified visitors, a leads-style forms list and responses table, and an organization custom forms domain verified by CNAME with hourly re-checks	2026-09-01 09:27:28 -07:00
Matthew Meszaros	4385f6ec02	feat: merge main and renumber the segment migrations after PR #283 took 000114 and 000115: campaign_segments becomes 000116 and campaign_lead_removals becomes 000117, since unibox_folders and mailbox_security_mode are already recorded in deployments' schema_migrations and ours are not, restoring a clean 000001-000117 sequence that check-migrations and golang-migrate both accept	2026-09-01 04:06:13 -07:00
Matthew Meszaros	128e314103	Merge remote-tracking branch 'origin/main' into fix/issue-277	2026-09-01 04:03:37 -07:00
Matthew Meszaros	d6ddf1f170	feat: fix implicit-TLS SMTP on 465 and IMAP STARTTLS on 143 behind a stored per-mailbox security mode that accepts any port, stop worker ID churn orphaning mailbox assignments via flock-claimed persistent worker ids, give the unibox a standard mail-folder sidebar (inbox/sent/drafts/archive/spam/trash) backed by a provider-derived folder column, and expose the AI tool registry over REST for non-MCP function-calling agents (#283)	2026-09-01 03:53:19 -07:00
Matthew Meszaros	5e87b1bbbf	Merge remote-tracking branch 'origin/main' into feature/public-forms	2026-09-01 01:17:54 -07:00