Commit Graph
595 Commits
Author SHA1 Message Date
Matthew Meszaros 704fecdcc0 feat: settle on WARMBLY_NODE_REGION as the one region variable for every role, because the join script wrote it while the worker still read WORKER_REGION, so a joined worker reported no region and the placement geography term stayed dead for exactly the machines that had been given one 2026-09-09 05:04:55 -07:00
Matthew Meszaros de60bc3784 feat: let an isolated-egress reservation move a mailbox without clearing the score gain, because the reserved worker scores lower than the incumbent by construction (the incumbent holds the stickiness bonus) so the rotation loop refused the move on every tick and the organization never converged onto the worker it pays for 2026-09-09 05:02:53 -07:00
Matthew Meszaros 88992c396c feat: hand the node its env base64 encoded at join instead of as a raw JSON string, because the POSIX-sh join script has no JSON parser and its sed extraction appended a stray ","heartbeat_seconds line to node.env, and refuse a heartbeat that re-registers an existing node id under the other role, which would have left a worker's mailboxes assigned to a machine that had stopped doing worker work 2026-09-09 05:01:13 -07:00
Matthew Meszaros 435dbb522f feat: replace the worker tier/type/risk-pool/egress categories with a scored placement model and make the fleet pull-based, so a machine joins with one command, workers and consumers share one node registry with usage and liveness, nodes self-update to the version the control plane resolves, and the Hetzner provisioning, worker profiles and SSH orchestrator are removed 2026-09-09 04:54:01 -07:00
Matthew Meszaros e783c3e3a0 feat: merge main into the email editor branch 2026-09-08 21:27:28 -07:00
Matthew Meszaros 7ad49c649a Merge branch 'main' into fix/thread-on-in-reply-to 2026-09-08 21:22:31 -07:00
Matthew Meszaros 4f40ab7ec8 feat: merge main into the in-reply-to threading branch 2026-09-08 21:06:29 -07:00
Matthew Meszaros 95a4986d9e feat: merge main into the verification reopen branch 2026-09-08 21:05:37 -07:00
Matthew Meszaros 9ce1e060dd feat: fix the campaign step body editor for issue #380 by registering TipTap's UndoRedo and HardBreak so Ctrl+Z and Shift+Enter work at all, normalising pasted HTML from Gmail/Outlook/Word so their blank-line scaffolding stops rendering a second gap on top of our paragraph margins, adding a workspace email image library (migration 000139, public email-images/ objects sharing the attachment storage quota, keyset-paginated /email-images endpoints) with toolbar upload, URL and library insertion, drag-drop, clipboard paste and a size/alignment/alt bubble, and adding an HTML source view that names the tags the schema would drop before switching back 2026-09-08 21:00:09 -07:00
Matthew Meszaros 0e7525866a feat: merge main into the select-all branch and rebuild the contact selection on top of its new keyset pagination, dropping this branch's own cursor repair now that Search carries the sort key and boundary value in a SortCursor, re-extracting the shared WHERE builder from that version, and pointing SearchIDs at the contactSorts spec and the conditional campaign-count lateral so a resolved selection orders and joins exactly the way the list does 2026-09-08 20:26:34 -07:00
Matthew Meszaros 5a1a726fdc Merge branch 'main' into fix/reopen-verification-on-connect 2026-09-08 20:10:31 -07:00
Matthew Meszaros c8042e2942 Merge remote-tracking branch 'origin/main' into feat/campaign-entry-delay 2026-09-08 09:58:38 -07:00
joao-crm 83ba4b38b4 fix: re-check the addresses the built-in probe left unknown as soon as a workspace connects a paid verifier, and withdraw the connection's degraded state once that verifier answers again, since a verdict reached before the connection existed otherwise waited out the 30-day shelf life while the credits sat unused and the card kept reporting whatever the provider said on the very first pass 2026-09-08 15:48:12 +00:00
Matthew Meszaros 37ed967919 Merge remote-tracking branch 'origin/main' into fix/issue-382-user-experience 2026-09-08 08:32:31 -07:00
Matthew Meszaros 4e52e9476b Merge remote-tracking branch 'origin/main' into feat/campaign-entry-delay 2026-09-08 07:31:31 -07:00
Matthew Meszaros f8c0899be4 feat: address the CodeRabbit review by promoting a rounded wait into the next unit so a contact drawer never reads 60 minutes or 24 hours, describing the entry delay's anchor as the lead's own enrolment moment with the campaign's creation date only as the legacy fallback in both the campaigns and segments guides, replacing the launch dialog's 'sending begins immediately' line when a delay is set, following an outside value change with the picker's preset or custom mode, and chaining the trigger card's saves so two commits inside one round trip cannot land out of order 2026-09-08 07:30:24 -07:00
Matthew Meszaros 0f83329936 feat: answer the CodeRabbit re-review by guaranteeing the promise the failed-pin warning makes: an import-level note now goes to the front of the error list and is never dropped by the per-row cap, so a file full of bad addresses cannot bury or evict the one note explaining why the rows that did import are not in the segment they were imported into, with a single note however many segments were targeted and a live test that forces the write to fail behind a capped error list 2026-09-08 06:54:36 -07:00
Matthew Meszaros f46728733e feat: address the CodeRabbit review on the select-all PR by tying the header checkbox to the rows on screen and making it put unticked rows back before it clears a select-all, bounding the exclusion list before it sizes a map, skipping half-filled custom-field pills when a filter is saved as a segment so the segment matches the search, and naming the fifty-thousand selection cap on the campaigns and segments guides 2026-09-08 05:39:16 -07:00
Matthew Meszaros a6862477b3 feat: add a 'select all matching' selection to every contact list so a bulk action can cover the whole filtered set instead of only the loaded pages: a ContactSelection body (all + filters + exclude) on bulk update, bulk delete, verification, batch research, segment members and CRM push resolved server-side through a new SearchIDs that shares the search's WHERE builder, a Gmail-style banner and shared selection module in the contacts table, campaign Leads and segment members views, an uncapped select-all in the From contacts picker, and fixes for the keyset cursor comparing a row against itself (contact pagination repeated and skipped rows past page one) and the dashboard sending custom-field filters under the wrong wire name 2026-09-08 05:19:59 -07:00
Matthew Meszaros 99c4ec9e21 feat: answer the CodeRabbit review on #386 by making ContactImportResult.SegmentsPinned a *bool so omitempty stops collapsing a failed membership write into the same absent value as an import that asked for none, warning in the result step when a pin did not land instead of leaving it to a note under a green Import complete, holding the pinned-contacts truncation notice until the overrides listing has actually arrived so it cannot read the newest 0 of 5,000 while it loads, and pointing that notice at the contact drawer, since a pinned-out contact can never be reached from the member list it used to name 2026-09-08 05:19:03 -07:00
Matthew Meszaros ed50ad9f2f feat: add a campaign entry delay so a contact's first email can wait a set time after they enter the campaign, with campaigns.entry_delay_minutes and a campaign_leads.added_at anchor (migration 000136), the delay applied in the router's per-lead due check and floored into the placer through ContactSequencePair.NotBefore, a distinct entry_delay constraint in the contact next-action preview, and the control surfaced on the Schedule tab, a new Trigger card at the top of the Steps canvas, the campaign wizard's Schedule step, the launch dialog, the update_campaign AI tool and the iOS schedule page, plus guides, API reference and live scheduler and repository tests 2026-09-08 05:06:04 -07:00
Matthew Meszaros 55894a0e9d feat: fix the campaign Leads tab and contact export stopping partway through a large list by rebuilding the contacts keyset cursor: the boundary subquery named the outer row's alias so Postgres read it as a correlated self-reference and the whole comparison collapsed to c.id >= <cursor uuid>, serving each page the newest rows of a randomly shrinking id range until it ran out at a third of the leads, so the opaque token now carries the ordering it was taken under plus the boundary row's own sort value and id, rejects a token replayed under another sort, and pairs with a direction-following id tiebreak, an ORDER BY that names its NULL placement, a new (organization_id, created_at DESC, id DESC) index, EXISTS campaign and category filters and a lateral campaign_count the query only joins when a filter or sort asks for it, taking a page of a 50k-contact organization from 120ms to 3ms, and the dashboard now keeps the rows it already loaded when a later page fails and says how far through the list Load more is 2026-09-08 05:01:15 -07:00
Matthew Meszaros 7bac16aab0 feat: cover the segment import path with live tests: an import pins its inserted and skipped-as-duplicate rows into the target segment and the segment's own contact query returns them, an unknown segment is refused before any row is written while a recurring source skips one that is gone, and a sheet-sync source round-trips its segment targets through the new column and the ?segment_id= filter 2026-09-08 04:59:31 -07:00
Matthew Meszaros 6284eb27ec feat: give a saved Google Sheets sync source segment targets (migration 000136 adds lead_sync_sources.segment_ids) that pin every synced row into those segments on each run, validated against the organization when the source is written and dropped from the run when a segment is deleted later, and filter GET /lead-sync/sources by ?segment_id= through a jsonb containment test so a segment can list the sources feeding it 2026-09-08 04:59:18 -07:00
Matthew Meszaros f5b97f3875 feat: resolve a contact import's segment targets through one shared path that a recurring source can ask to be lenient about, expose it as ValidateSegmentTargets so a saved sheet-sync source is refused when it is written rather than on its next run, and report segments_pinned on the import result so a caller never claims a membership write that did not land 2026-09-08 04:59:10 -07:00
Ralf KleinandClaude Opus 5 3cef4e8268 fix: thread an IMAP message on In-Reply-To only, never on the sender's Reply-To
imapStore fell back to Reply-To as the thread parent whenever a message
carried no In-Reply-To. Reply-To is an address header -- send replies to
this mailbox -- not a message identifier, so the thread key became a
person's address and every message that sender ever sent collapsed into
one strand.

Measured on a production instance with 38 IMAP mailboxes: 484 of 1431
stored messages sat in 62 threads. One reporter's 76 unrelated DMARC
aggregate reports arrived as a single 76-message conversation, and one
mailbox's sequence tests, compose tests and live outreach merged into
another. The remaining 947 messages, which did carry In-Reply-To, were
threaded correctly.

Sequence safety was never affected: stop_on_reply matches In-Reply-To
against the task rather than the thread. The damage is to the unibox,
where replies are triaged.

A message that answers nothing now has no parent and roots its thread on
its own Message-ID, which is what the existing else branch already does.
The test fails if the fallback is reintroduced.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-09-08 12:20:52 +02:00
Matthew Meszaros 9729f608c0 feat: address the CodeRabbit review on the admin backend PR by carrying the keyset boundary inside the sync cursor token instead of re-reading the cursor row's mutable updated_at, clearing only a throttle that is still active so an expired one is not reported as cleared, counting a medium-only UTM attribution in the acquisition with_channel total the way the channel list already does, releasing the exact dedicated assignment the handler read by id so a binding created meanwhile is never released, and returning the underlying failure from the warmup engagement poller, heartbeat sync, danger zone and org transfer housekeeping loops so scheduled_job_runs records an error instead of ok 2026-09-07 21:56:23 -07:00
Matthew Meszaros c26300ae5e feat: backend half of the admin panel upgrade: delete the unrouted provisioning, releases, plan, discount and enterprise-inquiry admin handlers with their service and repository methods, retire the six admin permission bits nothing gated as reserved placeholders so live bit positions and existing masks are unchanged and IsSuperAdmin checks the live set, add forty admin endpoints for mailbox sync governor state with clear-throttle and restart-backfill, in-flight send reservations, cross-workspace dead letters with replay, task failures, webhook delivery health with reclaim, fleet capacity, the control loops decision log, dedicated bindings with release and the routed convert-to-dedicated, operator-driven workspace export and import, per-organization API keys and webhooks, warmup invalid-token abuse and action history, and signups by acquisition channel, and add a scheduled_job_runs table (migration 000135) with a jobrun package that every backend and consumer loop now records through and a run-now request the owning process picks up within fifteen seconds 2026-09-07 21:40:38 -07:00
joao-crm e116912f4e fix: log the reply-intent automation error in HandleNewEmail instead of discarding it with a bare underscore, since a silent failure there is indistinguishable from a reply that linked correctly and leaves nothing anywhere to explain why replied_at stayed NULL while stop_on_reply kept mailing someone who had answered 2026-09-08 00:31:45 +00:00
Matthew Meszaros 25484f70ab Merge remote-tracking branch 'origin/main' into feat/issue-357-loopback-mail-relays 2026-09-07 09:05:11 -07:00
Matthew Meszaros 8b757985f3 feat: address the review on the folder identity change by retiring a deleted folder's backfill floor with it, since a name is reusable and an inherited floor silently skips the next folder's history, by claiming a rename only when a UIDVALIDITY has exactly one missing folder and one new one, because two missing folders and one arrival cannot say which was renamed and guessing moves the wrong folder's mail, by deduplicating names before the folder cap rather than after so a name listed twice cannot spend a real folder's slot, by deleting on a legacy UIDVALIDITY-only event only when that number still names exactly one folder, and by moving the folder row and its mail in one transaction so a refused rename cannot leave the messages in a folder nothing renamed 2026-09-07 08:49:23 -07:00
Matthew Meszaros 769a05aa90 feat: identify an IMAP folder by its name rather than by its UIDVALIDITY, which RFC 3501 never promised was unique across folders, so a mailbox on a server that stamps that number with the folder's creation time no longer loses the entire sync of every folder in a tree created in the same second, with the folder row keyed on (email_id, mailbox), each stored message stamped with its folder's name alongside the UIDVALIDITY generation its uid belongs to, a rename followed as a move that carries the mail and the cursor instead of orphaning both, and a changed UIDVALIDITY treated as what it is, the cursor going void 2026-09-07 08:21:17 -07:00
Matthew Meszaros 766bd3ae6a feat: let a self-hosted instance connect a mail server on its own machine by adding a third mailbox security mode, "none", accepted only for a loopback literal and only where the worker shares a host with the relay, so Proton Bridge on 127.0.0.1:1143/1025 and a local Dovecot or Mailpit can be connected at all, with the rule enforced in onboarding and reauth validation, again by the worker against the peer it actually dialled rather than the name it was given, and hidden from the connect form on the hosted product where the worker is never the customer's machine 2026-09-07 08:07:40 -07:00
Matthew Meszaros a5802b53cf feat: address the review on the unsubscribe-link change by closing the tag scanner over quoted attributes in both the send path and the editor, so a > inside an attribute can no longer split a tag and rewrite the href that follows it into a dead link, and by scanning only the copy a plain-text step actually ships, resolving the campaign's inherited opt-out mode before warning, checking the subject too, and reporting an unreadable step list as a failed check rather than an empty scan 2026-09-07 06:34:45 -07:00
Matthew Meszaros b25897fe67 feat: render a hand-placed {{.UnsubscribeLink}} as a real anchor labelled with the workspace's unsubscribe link text instead of shipping the bare signed API URL in the body, let the composer turn a text selection into that link so the wording stays the author's, and warn in the step editor and at preflight when a plain-text-only campaign carries a body opt-out link that has nowhere to hide its address 2026-09-07 06:34:45 -07:00
Matthew Meszaros 26c88d2426 feat: address the CodeRabbit review on the PostHog PR by dropping a referrer that is not a hostname instead of storing its query string or fragment, redacting email-shaped values out of every acquisition field before they reach the database or an analytics property, counting an invited signup which returned before the count was taken, hanging subscription_started off the persisted trial-to-paid transition so a redelivered webhook cannot report a second start, capturing the validated provider in both OAuth mailbox paths including Warmbly Cloud, making the two acquisition toggles mutually exclusive and naming them after what they actually select, and replacing the unsupported CNIL consent-exemption claim with what the guidance says and an explicit note that qualifying is a deployment-specific assessment this document does not make 2026-09-07 05:45:11 -07:00
Matthew Meszaros b204737a05 Merge the Sentry branch (with main) into the PostHog branch 2026-09-07 05:01:38 -07:00
Matthew Meszaros 1239f1d191 Merge main into the Sentry branch, routing the new smtp reporting through the errs wrapper instead of the SDK import main added 2026-09-07 05:00:43 -07:00
Matthew Meszaros 8f604921ab Merge branch 'main' into fix/issue-343-behaviour 2026-09-07 04:49:48 -07:00
Matthew Meszaros 4582896f54 merge 2026-09-07 04:43:45 -07:00
Matthew Meszaros f1b6345337 feat: address the CodeRabbit review on the mailbox error lifecycle PR by bounding the transient-error resolution to failures raised before the sync pass ran, because JetStream is configured with MaxDeliver and no MaxAckPending so a redelivered older success could otherwise clear a newer outage and leave the mailbox looking healthy until the next distinct one, naming the archive folder in the drawer's folder note alongside the others the cap always keeps, and giving the duplicate-folder note a singular verb 2026-09-07 04:36:50 -07:00
Matthew Meszaros 87a7e3e1ed feat: make a hosted form creatable again by binding an unset embed allowlist and field list as empty arrays instead of NULL in the forms repository writes, which is what made every New form fail with a not-null violation on forms.allowed_domains since the feature shipped (issue #343), and while proving the flow end to end keep the port on the shared forms host so share links and embeds resolve on a ported install, build the embed snippet on the form's own origin so an organization on a verified custom forms domain does not embed from the shared one, and apply the same empty-array fix to webhook endpoints created without event_types, which answered the raw Postgres error instead of the documented subscribe-to-everything 2026-09-07 04:35:56 -07:00
Matthew Meszaros d00960ad59 Merge feat/sentry-everywhere (with main) into the PostHog branch 2026-09-07 04:26:17 -07:00
Matthew Meszaros be16f77f5d Merge main into the Sentry branch 2026-09-07 04:25:41 -07:00
Matthew Meszaros 1af5ac1ea6 feat: let a fixed mailbox problem stop showing as a permanent error (issues #362, #363): resolve a mailbox's connection errors on the first sync pass that reaches the server again, because nothing but a credential reconnect ever resolved an error row, so a five-minute outage left a red needs-attention on the mailbox for good and held its health at warning; carry what the folder listing had to skip as sync state shown in the drawer's Sync card instead of raising an error row nobody can withdraw, so the note disappears once the mailbox is back under the folder limit or the duplicate folder id is renamed; and keep only a fingerprint per message in the IMAP flag scan rather than the whole flag set and Message-ID, which held 74 MB per mailbox at the window and folder limits against 14 MB for the digests, on a worker whose base capacity is 16 mailboxes 2026-09-07 04:24:09 -07:00
Matthew Meszaros 211650af8d feat: send through every kind of SMTP server (issues #359-#361): negotiate the sign-in method from what the server advertises, preferring CRAM-MD5 then LOGIN then PLAIN, because sending AUTH PLAIN blind was refused by every server that offers only LOGIN, which is Microsoft 365 relays and most appliance relays, and that refusal was reported to the mailbox's owner as a wrong password and deactivated the account; put a deadline on the whole SMTP conversation so a peer that stops answering without closing the connection can no longer park a send goroutine forever, which only the dial was protected against; classify a refusal by its reply code so a permanent 5xx on the sender, the recipient or the message is reported as the rejection it is and not retried four times as though the server were offline, while a 4xx still retries; announce the sender's own domain in EHLO rather than net/smtp's localhost, which relays read as a spam signal; and share one AUTH LOGIN implementation with the notification mailer instead of keeping two copies of the code that handles credentials 2026-09-07 04:19:42 -07:00
Matthew Meszaros 8e9f67f46b feat: add cookieless PostHog analytics for the hosted marketing site and dashboard with server-side signup and subscription events, a first-party acquisition record written once at signup on a new organization_acquisition table registered in the org-transfer spec, an acquisition column and channel filter in the admin org list, and never a single request from a self-host because every key is unset by default 2026-09-07 04:18:33 -07:00
Matthew Meszaros b32a28778c Merge pull request #355 from warmbly/feat/agent-tools-forms-segments-suppressions
feat: expose forms, segments and the suppression list to agents
2026-09-07 04:06:57 -07:00
Matthew Meszaros 51dedc90ee feat: put every runtime behind one optional error-reporting story: a single internal/observability/errs wrapper that is now the only package importing sentry-go, InitSentry for cmd/forms, release and environment tags on every service from the existing build stamp, optional Sentry in the admin panel and the public forms app, the sentry crate in the Rust tracking service, release tagging in realtime, CI source-map upload that only runs when a Sentry token is configured, and docs covering the DSN for each service 2026-09-07 03:51:06 -07:00
Matthew Meszaros 875cea477f feat: give a duplicate IMAP folder id its own error code and guidance instead of reusing the folder-limit one, because the two problems have different fixes: getting under the folder cap versus renaming the folder the mail server gave a duplicate UIDVALIDITY, and the drawer was telling a user to delete folders when neither the cap nor deleting anything was involved 2026-09-07 03:42:57 -07:00