276 Commits

Include renames

Author	SHA1	Message	Date
Matthew Meszaros	510ee692ba	feat: address the review on the split-deployment branch by moving the two broker routes onto their own NODE_BROKER_TOKEN so the internet-facing tracking and forms services no longer hold a credential that can open any organization's data key, refusing to presign any key outside the prefixes a node reaches, fixing IAM policies that named an alias ARN KMS never resolves in a Resource element, bounding both brokered HTTP clients because the sync loop's context never expires, no longer reporting a 403 from the object store as a missing body, and redacting the DSN and URL credentials the dry-run listing printed in clear	2026-09-10 14:19:53 +02:00
Matthew Meszaros	47ba13083e	feat: make a split deployment work end to end by fixing the three defects that made an off-host node impossible to configure (nodeEnvKeys shipped S3_BUCKET and KMS_KEY_ID, which nothing reads, so an AWS-backed node silently used the default bucket and key alias; a joined consumer never received PRIMARY_DB and died at boot; and node.env was rewritten on every join with no file an operator could add to), then removing the need for cloud credentials on a node at all with brokered KMS and blob providers that renderNodeEnv hands out automatically, plus deploy/split-cloud, scripts/aws-bootstrap.sh, two fleet instance checks and the docs	2026-09-10 13:58:59 +02:00
Matthew Meszaros	bb2b2b1743	feat: make the one-unresolved-error-per-code rule a partial unique index with a conflict path, collapsing the duplicate rows a repeating mail error already left behind	2026-09-09 09:46:39 -07:00
Matthew Meszaros	88ea60cd4d	feat: renumber the custom-domain TLS lookup indexes to 000144 after main landed 000143 for campaign lead senders, because two migrations sharing a version stop golang-migrate before either runs	2026-09-09 09:07:26 -07:00
Matthew Meszaros	cab27fac62	Merge remote-tracking branch 'origin/main' into fix/main-ci-failure-and-issue-400	2026-09-09 09:05:45 -07:00
Matthew Meszaros	18a8c7b009	feat: give a self-hosted instance on-demand TLS for customer tracking and forms domains, gating Caddy's ask on a /tls/authorize endpoint that answers only for domains this instance has verified, so a workspace CNAME stops serving every tracked link and opt-out link with no certificate	2026-09-09 08:56:08 -07:00
Matthew Meszaros	68b5d8f358	feat: bind a campaign lead to the mailbox that sends its first email so every follow-up leaves from the same address, holding a lead back while its mailbox is merely out of budget or outside its hours and moving it to another mailbox only when that one can no longer send for the campaign at all	2026-09-09 08:51:00 -07:00
Matthew Meszaros	1a7fd85f4b	feat: drop the throwaway migration harness that a blanket git add swept into the placement commit, since it only existed to apply migrations against a scratch database while verifying the branch	2026-09-09 04:56:07 -07:00
Matthew Meszaros	435dbb522f	feat: replace the worker tier/type/risk-pool/egress categories with a scored placement model and make the fleet pull-based, so a machine joins with one command, workers and consumers share one node registry with usage and liveness, nodes self-update to the version the control plane resolves, and the Hetzner provisioning, worker profiles and SSH orchestrator are removed	2026-09-09 04:54:01 -07:00
Matthew Meszaros	9ce1e060dd	feat: fix the campaign step body editor for issue #380 by registering TipTap's UndoRedo and HardBreak so Ctrl+Z and Shift+Enter work at all, normalising pasted HTML from Gmail/Outlook/Word so their blank-line scaffolding stops rendering a second gap on top of our paragraph margins, adding a workspace email image library (migration 000139, public email-images/ objects sharing the attachment storage quota, keyset-paginated /email-images endpoints) with toolbar upload, URL and library insertion, drag-drop, clipboard paste and a size/alignment/alt bubble, and adding an HTML source view that names the tags the schema would drop before switching back	2026-09-08 21:00:09 -07:00
Matthew Meszaros	29b479321a	feat: merge main and renumber the campaign entry-delay migration to 000138, since main released 000137_contacts_org_created_index while this branch was waiting to merge	2026-09-08 09:59:43 -07:00
Matthew Meszaros	c8042e2942	Merge remote-tracking branch 'origin/main' into feat/campaign-entry-delay	2026-09-08 09:58:38 -07:00
Matthew Meszaros	a62652de3a	feat: merge main and renumber the contacts index migration to 000137, since main released its own 000136 for lead sync segments while this branch was in review and two branches that each take the next number are green alone but leave golang-migrate refusing to build its source driver once both land	2026-09-08 08:34:52 -07:00
Matthew Meszaros	37ed967919	Merge remote-tracking branch 'origin/main' into fix/issue-382-user-experience	2026-09-08 08:32:31 -07:00
Matthew Meszaros	073bb3c6bc	feat: merge main and renumber the campaign entry-delay migration to 000137, since main released 000136_lead_sync_segments while this branch was open and a duplicate version makes golang-migrate refuse to build its source driver, which is what failed the Migrations job and the check-migrations prerequisite inside make lint on Go CI	2026-09-08 07:33:21 -07:00
Matthew Meszaros	4e52e9476b	Merge remote-tracking branch 'origin/main' into feat/campaign-entry-delay	2026-09-08 07:31:31 -07:00
Matthew Meszaros	ed50ad9f2f	feat: add a campaign entry delay so a contact's first email can wait a set time after they enter the campaign, with campaigns.entry_delay_minutes and a campaign_leads.added_at anchor (migration 000136), the delay applied in the router's per-lead due check and floored into the placer through ContactSequencePair.NotBefore, a distinct entry_delay constraint in the contact next-action preview, and the control surfaced on the Schedule tab, a new Trigger card at the top of the Steps canvas, the campaign wizard's Schedule step, the launch dialog, the update_campaign AI tool and the iOS schedule page, plus guides, API reference and live scheduler and repository tests	2026-09-08 05:06:04 -07:00
Matthew Meszaros	55894a0e9d	feat: fix the campaign Leads tab and contact export stopping partway through a large list by rebuilding the contacts keyset cursor: the boundary subquery named the outer row's alias so Postgres read it as a correlated self-reference and the whole comparison collapsed to c.id >= <cursor uuid>, serving each page the newest rows of a randomly shrinking id range until it ran out at a third of the leads, so the opaque token now carries the ordering it was taken under plus the boundary row's own sort value and id, rejects a token replayed under another sort, and pairs with a direction-following id tiebreak, an ORDER BY that names its NULL placement, a new (organization_id, created_at DESC, id DESC) index, EXISTS campaign and category filters and a lateral campaign_count the query only joins when a filter or sort asks for it, taking a page of a 50k-contact organization from 120ms to 3ms, and the dashboard now keeps the rows it already loaded when a later page fails and says how far through the list Load more is	2026-09-08 05:01:15 -07:00
Matthew Meszaros	6284eb27ec	feat: give a saved Google Sheets sync source segment targets (migration 000136 adds lead_sync_sources.segment_ids) that pin every synced row into those segments on each run, validated against the organization when the source is written and dropped from the run when a segment is deleted later, and filter GET /lead-sync/sources by ?segment_id= through a jsonb containment test so a segment can list the sources feeding it	2026-09-08 04:59:18 -07:00
Matthew Meszaros	c26300ae5e	feat: backend half of the admin panel upgrade: delete the unrouted provisioning, releases, plan, discount and enterprise-inquiry admin handlers with their service and repository methods, retire the six admin permission bits nothing gated as reserved placeholders so live bit positions and existing masks are unchanged and IsSuperAdmin checks the live set, add forty admin endpoints for mailbox sync governor state with clear-throttle and restart-backfill, in-flight send reservations, cross-workspace dead letters with replay, task failures, webhook delivery health with reclaim, fleet capacity, the control loops decision log, dedicated bindings with release and the routed convert-to-dedicated, operator-driven workspace export and import, per-organization API keys and webhooks, warmup invalid-token abuse and action history, and signups by acquisition channel, and add a scheduled_job_runs table (migration 000135) with a jobrun package that every backend and consumer loop now records through and a run-now request the owning process picks up within fifteen seconds	2026-09-07 21:40:38 -07:00
Matthew Meszaros	769a05aa90	feat: identify an IMAP folder by its name rather than by its UIDVALIDITY, which RFC 3501 never promised was unique across folders, so a mailbox on a server that stamps that number with the folder's creation time no longer loses the entire sync of every folder in a tree created in the same second, with the folder row keyed on (email_id, mailbox), each stored message stamped with its folder's name alongside the UIDVALIDITY generation its uid belongs to, a rename followed as a move that carries the mail and the cursor instead of orphaning both, and a changed UIDVALIDITY treated as what it is, the cursor going void	2026-09-07 08:21:17 -07:00
Matthew Meszaros	04dc83da38	Merge the Sentry review fixes into the PostHog branch, keeping the new JavaScript escaping for the PostHog runtime-config values too	2026-09-07 04:48:30 -07:00
Matthew Meszaros	f043bfc761	feat: address the CodeRabbit review on the Sentry PR by escaping every runtime-config value the web and admin entrypoints write into a JavaScript string literal so a quote or newline in a DSN cannot inject or discard the whole config, reporting only the count and types of a failed query's parameters instead of their values now that the db scope actually applies, cloning the fallback hub in errs.Hub so scope cannot leak between requests, flushing the tracking service before every fatal exit since process::exit skips the guard's drop, treating a blank WARMBLY_RELEASE or APP_ENV as unset in realtime, stamping the environment and a service tag into the forms page, and correcting the docs claim that an unset DSN means the SDK is never initialised	2026-09-07 04:47:51 -07:00
Matthew Meszaros	6f230a3e17	feat: renumber the organization_acquisition migration from 000132 to 000133 after main landed its own 000132 for unibox mailbox uid_next, since two branches that each took the next number are green alone and collide once both merge	2026-09-07 04:27:02 -07:00
Matthew Meszaros	d00960ad59	Merge feat/sentry-everywhere (with main) into the PostHog branch	2026-09-07 04:26:17 -07:00
Matthew Meszaros	be16f77f5d	Merge main into the Sentry branch	2026-09-07 04:25:41 -07:00
Matthew Meszaros	8e9f67f46b	feat: add cookieless PostHog analytics for the hosted marketing site and dashboard with server-side signup and subscription events, a first-party acquisition record written once at signup on a new organization_acquisition table registered in the org-transfer spec, an acquisition column and channel filter in the admin org list, and never a single request from a self-host because every key is unset by default	2026-09-07 04:18:33 -07:00
Matthew Meszaros	51dedc90ee	feat: put every runtime behind one optional error-reporting story: a single internal/observability/errs wrapper that is now the only package importing sentry-go, InitSentry for cmd/forms, release and environment tags on every service from the existing build stamp, optional Sentry in the admin panel and the public forms app, the sentry crate in the Rust tracking service, release tagging in realtime, CI source-map upload that only runs when a Sentry token is configured, and docs covering the DSN for each service	2026-09-07 03:51:06 -07:00
Matthew Meszaros	0df1a57589	feat: renumber the unibox_mailboxes uid_next migration from 000131 to 000132 after the campaign continuous lead sources migration took 000131 on main, so golang-migrate does not refuse to build its source driver once both are merged	2026-09-07 03:15:06 -07:00
Matthew Meszaros	1a87720d5a	merge	2026-09-07 03:14:42 -07:00
Matthew Meszaros	f2c35cf872	feat: sync every kind of IMAP mailbox (issues #345-#349): connect servers without CONDSTORE by following UIDNEXT per folder and mirroring read state with a periodic flag scan, so Outlook.com, Microsoft 365 over IMAP and Yahoo work instead of failing at load; fall back to a STATUS per folder when the server has no LIST-STATUS, which silently made an account look empty; bound the wait between IMAP responses so a peer that vanishes without a FIN cannot park a command forever; keep the inbox and special folders when a mailbox has more folders than the cap (raised to 100) and relay the overflow as a warning instead of failing the mailbox silently; recognize localized folder names and the server's own hierarchy delimiter so a Sent folder called Gesendete Elemente is not filed as inbox; and back off on a widening interval while a mail server is unreachable so one outage is one warning rather than one a minute	2026-09-07 03:12:22 -07:00
Matthew Meszaros	5d60fe9a21	feat: fix campaign restart dead end (issue #340): resolve preflight tracking senders through the same explicit/tags/all pool the scheduler uses so a connected mailbox is never reported missing, fetch the launch dialog's step count from the steps endpoint instead of a field the campaign API never returns, turn on Keep running for new leads when a form or an automation feeds a campaign (migration 000131 backfills existing ones) and when a member starts a campaign whose every lead has finished so it goes active and waits for leads instead of answering 400, return waiting_for_leads from the start endpoint for the dialog's success screen, and document the no_leads and no_remaining_leads codes and the new behaviour in the campaigns, forms, automations and API docs	2026-09-07 02:26:50 -07:00
Matthew Meszaros	7b11489c85	feat: keep a campaign running for new leads (issue #336): add a continuous campaign setting (migration 000130, on by default once a segment is linked) under which a campaign that runs out of leads stays active and idle with idle_since set instead of finishing, is startable with no leads, is re-checked by the reconciler every pass and shows as waiting for leads in the dashboard list, header and live activity panel with a CAMPAIGN_IDLE realtime event; make WakeCampaigns restart a finished campaign through the full launch checks so a lead added by the contacts API, a bulk update, an import or the add-to-campaign automation action reopens it exactly like a linked segment does, and write a refused restart to the campaign activity log once an hour per reason instead of only the backend log; expose continuous on the campaign API and preferences page and document it in the campaigns, segments, forms, automations, API reference and realtime docs	2026-09-06 02:50:16 -07:00
Matthew Meszaros	04a2ed59c9	feat: address the CodeRabbit review on the lead-intake PR by giving the webhook sink its own copy of every map payload so the automation goroutine never writes into the map still being marshalled for endpoint delivery, propagating contact lookup errors in the upsert action's skip mode and in the native actions adapter so a failed lookup is never mistaken for a missing contact, adding the contacts source CHECK as NOT VALID in migration 000129 to skip the table scan under the migration lock, defaulting a new create-or-update-contact node's email template to the address key the chosen trigger actually carries, correcting the n8n guide on which lead-form sources have a trigger node, naming API batches over 100 contacts among the silent bulk arrivals in the contacts guide, and updating the marketing automations page from nine to twelve triggers	2026-09-06 00:47:33 -07:00
Matthew Meszaros	a3989f9d9c	feat: turn automations into a lead-intake path so a Facebook, Instagram, LinkedIn or TikTok lead form pushed by Zapier, Make, n8n or any webhook becomes a tagged, campaign-enrolled contact without leaving Warmbly: add the warmbly.upsert_contact and warmbly.add_to_campaign built-in actions with templated field mapping, custom fields, tags, campaign and an if-exists policy, fire a rich contact.created event from the one contact write path (silent for file imports, sheet syncs and API batches over 100) and expose contact.created and form.submitted as automation triggers with condition fields, variables and sample data, carry an automation depth through events raised by an action so a flow that creates a contact cannot re-trigger itself past five hops, stamp automation-created contacts with the new automation source (migration 000129), share the campaign picker between the sheet sync wizard and the automation builder, document lead intake in the automations, Zapier, Make, contacts, forms, integrations, expressions and webhook pages plus a new n8n guide, mirror the new triggers and actions on the marketing automations page, and drop the 34 MB cli binary that was committed by mistake	2026-09-06 00:36:05 -07:00
Matthew Meszaros	028689fd2e	feat: device-code sign-in for the CLI, with a browser approval page at /cli that mints a scoped API key, self-revocation at DELETE /api-keys/self so a read-only credential can always end itself, and app_url plus websocket_url on /auth/config so a client can find the dashboard and the realtime gateway on a self-hosted layout	2026-09-04 20:14:27 -07:00
Matthew Meszaros	37d328c86a	feat: add a partial index on campaign_contact_progress.dispatch_task_id so the daily send counters can look a task's reservation up without scanning progress	2026-09-03 20:18:43 -07:00
Matthew Meszaros	701dc404a2	Merge remote-tracking branch 'origin/main' into fix/click-without-open-tracking	2026-09-03 05:22:05 -07:00
Matthew Meszaros	e97477a718	feat: renumber the engagement origin and pending index migrations to 000125 and 000126 because main released 000124 for unsubscribe opt-out while this branch was open	2026-09-03 05:22:05 -07:00
Matthew Meszaros	4c7229c7d6	feat: make a held-back click announcement retryable until its effects ran: the claim leases the click row for one attempt via announce_claimed_at and the flag clears only after the effects complete, the sweep retries expired leases as well as clicks a restart cut off, and the partial index for the sweep moves to its own concurrent migration 000125 so the live click table is never write-blocked	2026-09-03 04:34:30 -07:00
Matthew Meszaros	a3264f0d39	feat: address the review on the rebuilt engagement branch: a deferred human click's effects are now durable and once-only (announce_pending on the click row written before the event is marked processed, a claim in finishHumanClick and a minute-by-minute sweep that finishes what a restart cut off, a burst relabel clears the flag), the tracking service keys the source-address token with TRACKING_IP_HASH_KEY so it cannot be enumerated back into an IPv4 address, the migration no longer adds a blocking index to the existing click table, and timeline click and open rows carry task_id	2026-09-03 04:13:39 -07:00
Matthew Meszaros	31dabea0a4	feat: rebuild the click-without-open fix on top of the per-link click attribution from #298: a person's click now also counts as an open and a burst that withdraws the click withdraws the open it implied unless a real open is on record, routing readers ignore machine opens as the docs promised, every open gets its own log row and every open and click records the mail client or proxy, browser, device, OS, country, region and city (migration 000124: origin columns on email_link_clicks plus an email_opens table), the tracking service publishes only the address's network in a nullable client_ip field which the consumer resolves with GeoLite and drops, the contact Activity tab shows each open and the origin of opens and clicks, the campaign overview gains a who-engaged-from-where breakdown exposed as engagement in campaign analytics, live open and click events carry occurred_at, client and location, the leads table explains why an open is not always counted, both logs are pruned daily after a year, email_opens joins the export registry, the consumer reads GEODB_PATH optionally, and the guides and API references are updated (fixes #294)	2026-09-03 03:49:44 -07:00
Matthew Meszaros	579b0ac04c	feat: merge main into the unsubscribe branch again, carrying the UTM campaign columns alongside unsubscribe_mode in every campaign scanner, keeping unsubscribe links out of the moved link tracker and its UTM tagging, and renumbering the opt-out migration to 000124 because main released 000123 for link clicks	2026-09-03 03:43:47 -07:00
Matthew Meszaros	5fcb0fe882	feat: fold existing suppression rows to lowercase in migration 000123 and compare stored values as equalities so the existing unique key is the single case-insensitive identity and no extra index build is needed, and keep the plain-text alternative when a chosen A/B variant carries HTML only	2026-09-03 03:22:37 -07:00
Matthew Meszaros	78c4021053	Merge remote-tracking branch 'origin/main' into feat/link-click-attribution ... # Conflicts: # docs/content/docs/api/reference/campaigns.mdx # docs/content/docs/guides/campaigns.mdx # internal/repository/pg_campaign.go # internal/repository/pg_campaign_lifecycle.go # web/src/components/app/campaigns/NewCampaignDialog.tsx	2026-09-03 03:14:19 -07:00
Matthew Meszaros	3614d65d96	feat: renumber the link clicks migration to 000123 because main released 000122 for the campaign kind column first	2026-09-03 03:13:30 -07:00
Matthew Meszaros	64f2d1637b	feat: address the review on the unsubscribe PR: register the suppression family in warmblyctl, require the confirm field on the browser unsubscribe POST and cap its body, render a chosen A/B variant through the template engine so its merge fields and unsubscribe link resolve, fold curly apostrophes before opt-out phrase matching, write pasted suppression lists in one transaction, clamp copy by runes instead of bytes, add the constraints NOT VALID plus a lower(email) index in migration 000123, scope the unsubscribe link type-ahead to email bodies, and document DELETE /suppressions/:id	2026-09-03 02:20:47 -07:00
Matthew Meszaros	9485503e03	feat: merge main into the unsubscribe branch, keeping the new campaigns.kind column alongside unsubscribe_mode in every campaign scanner and renumbering the opt-out migration to 000123 because main took 000122 for campaign kind	2026-09-03 01:56:38 -07:00
Matthew Meszaros	5eb92c601c	feat: give every campaign email a working opt-out: a reply-to-opt-out line by default or an unsubscribe link (workspace setting under Settings > Sending with a per-campaign override and a {{.UnsubscribeLink}} variable), signed per-recipient unsubscribe links served on the API origin so the List-Unsubscribe header no longer points at a dead warmbly.com page, a confirm page on GET with RFC 8058 one-click on POST and a resubscribe button, reply opt-out detection through the whole-word compliance lexicon with quoted history stripped, a first-class suppression list (Contacts tab, GET/POST/DELETE /suppressions with address and domain entries, audited removal, contact drawer action), the contact Subscribed flag enforced in campaign routing, migration 000122 with a shared recipient_suppressed() predicate, and docs for all of it	2026-09-03 01:52:40 -07:00
Matthew Meszaros	24b7e26fbb	feat: add the one-time email campaign preset from #288: campaigns.kind column with sequence and one_time values and a kind field on POST /campaigns, GET /campaigns?kind= and an one_time count on /campaigns-overview, a POST /campaigns-estimate endpoint projecting deduplicated segment recipients against the mailbox pool's per-day capacity and finish date, a wizard type picker whose one-time flow runs Basics, Email, Audience, Sending and Send with send-now or scheduled start and the estimate panel and then creates, links segments and starts the campaign, a guard refusing a second email step on a one-time campaign, draft/scheduled/sending/sent wording with a One-time badge and Type filter in the campaigns list, kind on the AI create_campaign_draft tool, honouring text_only in the send and test-email paths so plain-text campaigns ship no HTML part or tracking, and the campaigns, segments, API reference and endpoint docs	2026-09-03 01:42:00 -07:00