Commit Graph
372 Commits
Author SHA1 Message Date
Matthew Meszaros ae1a324801 Merge pull request #562 from rocker1166/feat/direct-mail-analytics
feat: add accurate direct-mail analytics and opt-in engagement tracking
2026-09-17 11:47:50 +00:00
Matthew Meszaros d1aa3d4361 feat: classify ambiguous Outlook and Apple image-proxy opens by delivery timing and stop inventing recipient device metadata 2026-09-17 04:27:34 -07:00
SUMAN JANA 2134c7a143 feat(analytics): report on mail written by hand, with opt-in open and click tracking per mailbox 2026-09-17 10:26:25 +00:00
Matthew Meszaros 0f5ca71155 feat: correct mailbox sending metrics and workspace analytics across dashboard surfaces 2026-09-16 21:44:42 -07:00
Matthew Meszaros 2255e2145d feat: reject outbound mailbox copies and mismatched campaign threads before they can mark contacts replied for stop-on-reply (issue #549) 2026-09-16 08:24:19 -07:00
Matthew Meszaros 31c1f101c2 feat: give EmailSentEvent and WarmupEmailSentEvent their own derived Avro schemas so the email-events and warmup-events analytics streams stop failing at serialize on every send and finally register a subject (#546) 2026-09-16 04:20:15 -07:00
Matthew Meszaros 140c7de436 feat: add the admin panel's Promo codes page and route the six /admin/discounts endpoints that existed as handlers but were never wired, so a launch offer is built in the operator UI instead of an INSERT against production, with caps that an explicit null can actually clear on PATCH 2026-09-16 04:04:09 -07:00
Matthew Meszaros 746dd40469 feat: strengthen the Avro round-trip tests after a cutover they failed to catch (#536)
* feat: compare the decoded event body's fields and not only its type, fill arrays so every uuid carries a real value instead of the zero one a codec could drop unnoticed, and decode once in a process that has never encoded, because production is four processes and one of them only ever reads what another wrote

* feat: register the union body types at package load instead of on first schema build, which is what a process that only ever decodes never reached, so every worker command arrived as a map keyed by its branch name, went through the JSON fallback, and became a struct with every field zero and no error anywhere
2026-09-15 20:25:21 -07:00
Matthew Meszaros f106c8541d feat: make the bus envelopes Avro-encodable (#535)
* feat: make both bus envelopes Avro-encodable by deriving each one's schema from a declared registry of body types, with a union branch per body and our own struct walk that skips unexported fields and honours avro:"-" before descending, so the schema describes exactly what encoding/json already puts on the wire, and narrow the two sync cursors on the wire DTOs to int64 because Avro has no unsigned 64-bit type

* feat: stop the instance health check, the config registry and the docs all claiming Avro cannot serialize a worker envelope, which stopped being true once the envelopes carried a declared union, and check the one thing that is still a real misconfiguration instead: avro selected with no SCHEMA_REGISTRY_URL to resolve against

* feat: emit a reference the second time a record appears in an envelope schema instead of defining it again, because Avro names a record once and a document that defines warmbly.events.Token three times is rejected outright, and keep the Schema Registry round-trip as a skip-by-default test since only a registry judges the document rather than the objects it was built from

* feat: carry uint64 as Avro fixed(8) rather than long, which lets the sync cursors keep their unsigned type instead of being narrowed, name every event field after its json tag so the schema and the JSON wire agree, and populate every field in the round-trip test because zero values are why a uint64 mapped to long passed in the first place

* feat: frame Avro in Confluent's wire format and encode through hamba's default API instead of going through avrov2, whose private avro.API holds a type resolver avro.Register cannot reach, so a union body failed there with unable to resolve type while encoding cleanly against the same schema, and keep the registry round-trip as a skip-by-default test
2026-09-15 10:50:30 -07:00
Matthew Meszaros e67b13e57e feat: stop reporting a mailbox's DKIM as missing when its selector was simply never probed, by deriving candidate selectors from the sending domain's own SPF and MX records on top of a wider default set, reporting a miss as the tri-state dkim_status undetermined rather than a red Missing row in the drawer, dropping DKIM from the Advisor's missing-records finding entirely, refusing a revoked p= key, holding the summary back from accusing anything when DNS never answered, and fixing the CLI auth-check table whose columns read mailbox fields the endpoint does not return (#528) 2026-09-15 02:27:38 -07:00
Matthew Meszaros 95885fa714 fix: save a contact's edited email address (#511) (#521)
* fix: save a contact's edited email address by giving models.UpdateContact the Email field the dashboard was already sending, normalizing it to a bare lowercased address, refusing one another contact in the workspace holds with a 409 contact_email_taken instead of letting the unique index 500, and dropping the verification verdict plus the delivery evidence that belonged to the old mailbox (issue #511)

* fix: reset a contact's verification evidence behind a watermark the delivery-credit job honours, clear the cached esp_provider the old domain produced, write a case-only edit in place instead of answering 200 and changing nothing, and normalize the address on create and import too so the two paths cannot disagree about what an address is

* fix: refuse a verification observation whose campaign step provably left before the contact's address was edited, so a hard bounce or an open for the old mailbox arriving afterwards cannot mark the corrected address invalid or valid, read the credit watermark against dispatched_at rather than the sent_at a worker result stamps later, and answer a racing address collision with the documented 409 instead of the unique index's 500

* fix: build the deliverability evidence step from the resolved campaign task so both halves name one real row, drop the email format hint from the OpenAPI schema now that the endpoint also accepts a display-name address, and drain the shared delivery-credit backlog in the live test instead of assuming one pass reaches this contact
2026-09-14 23:05:16 -07:00
Matthew Meszaros 50711a8e66 fix: a campaign's linked segments are its audience, so detaching one withdraws its leads (#510) (#523)
* fix: make a campaign's linked segments the audience rather than an accumulator, so detaching one withdraws the leads it enrolled instead of leaving the old list mixed in with the new, tracked by a new campaign_leads.source that keeps a hand-picked lead, an overlapping segment's member and anyone the campaign has already emailed out of the withdrawal, and reported back as withdrawn/contacted counts the dialog confirms and explains (issue #510)

* fix: serialize the linked-segment sweep against a link replacement by taking the same campaign lock, so a sweep that read the old set cannot re-enrol the audience the replacement just withdrew, and word the dialog's confirm and toast so the campaign, not the segment, is what has already emailed a lead

* fix: stop the one-shot segment enrol from re-stamping leads that are already in the campaign, since the Leads tab's Add back runs through it and pinning a whole linked audience as hand-picked because one held-out member was restored is the accumulation this change exists to end

* fix: lock the leads a detachment is about to withdraw before deciding, because a send is reserved by stamping campaign_contact_progress and only then locking the lead row, so a reservation committing mid-pass was invisible to the delete's snapshot and could withdraw a lead whose first email had already gone; and report the already-emailed count on every toast branch, since a detach where the whole audience had been emailed changed no count and said nothing after confirming a removal

* fix: add the campaign_leads.source check constraint NOT VALID, which still enforces every insert and update while skipping a full scan of the largest table in the product under ACCESS EXCLUSIVE to learn that a one-statement-old column holds its own default everywhere
2026-09-14 22:12:15 -07:00
Matthew Meszaros 619eb2729a fix: read a mailbox's Gmail send-as addresses from the worker holding it rather than from the backend, which was decrypting a mailbox credential in the control plane and showing Google a second client address for a mailbox whose mail moves through a worker, by round-tripping a new MAILBOX_IDENTITY command answered on the process channel like a credential validation, leaving the OAuth handshake as the one place the control plane still calls the provider (#522) 2026-09-14 21:28:07 -07:00
Matthew Meszaros 1ca3bd19b3 feat: carry a unibox read or unread change out to the mailbox itself through a new MESSAGE_SEEN worker command, so a conversation read in Warmbly stops showing bold in Gmail, Outlook and IMAP, relaying the state the row holds rather than the one the request asked for, only for messages that actually changed, dispatched detached from the request and never retried (#515) 2026-09-14 21:20:42 -07:00
Matthew Meszaros ffd27bc46d fix: stop every out-of-office notice and bounce opening a high-priority CRM follow-up by classifying machine replies from their headers and gating the task on a per-intent setting, and give the Tasks page multi-select with select-all-matching, bulk status, priority and delete over new PATCH and DELETE /crm/tasks endpoints (issue #471) 2026-09-14 12:20:47 -07:00
Matthew Meszaros 8d790ede6c feat: send from any address Google has verified a Gmail mailbox to send as and import the signature its owner already wrote in Gmail, reading both through gmail.settings.basic at connect and on demand via GET/POST /emails/:id/identity, validating the choice against the provider's own list in the service and again inside the UPDATE, clearing it when the provider stops verifying it, and never applying it to warmup (#514) 2026-09-14 10:13:36 -07:00
Matthew Meszaros 13e9ce8e10 feat: hold a lead whose mailbox answers out of office until they are back, resuming at the return date it names, plus a manual per-contact pause in one campaign that unsubscribing and the suppression list were the only stand-ins for 2026-09-14 09:26:06 -07:00
Matthew Meszaros d74d5e6836 fix: retire the warmup spam score, a counter that grew with volume rather than misbehaviour and that no band could act on (#508)
* fix: retire the warmup spam score, a ratchet that grew with volume rather than misbehaviour and that no band ever read, dropping the column from the pool row and the reputation ledger and explaining a pool finding with the band's own reason instead (#491)

* test: pin the advisor snapshot's pool columns against the scan, since the band's reason now reaches the finding through that select alone (#491)

* fix: hold a warmup sentence's score and reason with the sentence itself, keep the retired spam_score key on the published analytics payload as a deprecated zero, seed the sandbox with severity-shaped scores, and record the raw spam report when the warmup service is absent (#491)
2026-09-14 07:44:34 -07:00
Matthew Meszaros 2f6c027e37 fix: take the warmup health sweep from twelve round-trips per mailbox to two, list participants stalest first, stop at the deadline, and return the standing from the write (#502)
* perf: cut the warmup health sweep from twelve round-trips per mailbox to seven by reading the participant row once, counting placements and complaints in one scan, complaints and bounces in one scan, and taking the spam score from the row already in hand (#492)

* perf: take the warmup health sweep to two round-trips per mailbox (one metrics statement, the write returns the row), list participants stalest first and stop at the deadline, and drop the dead spam-score and count surface (#492)
2026-09-14 03:14:15 -07:00
Matthew Meszaros 2bbd72e758 fix: make cross-tier warmup borrowing real and one-directional: a thin premium tier borrows proven free mailboxes through one repository rule, gates each drawn partner in its own pool, and only reply-backs cross tiers (#496)
* fix: gate a warmup partner borrowed from the other tier against the pool it is in rather than the sender's, since the thin-tier fallback had rejected every borrowed candidate and a thin tier failed instead of borrowing

* fix: make cross-tier warmup borrowing one-directional and gate borrowed partners in their own pool, so a thin premium tier can actually borrow proven free mailboxes (#495)

* fix: pin the borrow floor at the exact boundary so a premium tier at the floor including its sender still borrows (#495)

* fix: put the warmup borrowing rule in one repository method (direction, floor, proven age, workspace standing) that the selector and scheduler both read, pin every drawn partner's gate to its own pool, fall through buckets when a stale row fails the gate, and allow only reply-backs across tiers (#495)

* fix: end the warmup partner draw by candidate exhaustion instead of a fixed attempt cap, and fail closed when a free mailbox's workspace standing cannot be read before it answers into a paid inbox (#495)

* fix: end the warmup partner draw by candidate exhaustion instead of a fixed attempt cap, and fail closed when a free mailbox's workspace standing cannot be read before it answers into a paid inbox (#495)
2026-09-14 02:51:02 -07:00
Matthew Meszaros 40506c4f05 fix: seed the two warmup pools on every instance under fixed ids and make one pool per type structural, since the baseline squash dropped the insert and a fresh self-hosted instance never warmed; move memberships onto the canonical pools, scope the standing mirror trigger to the columns it mirrors so a pool move keeps a retention window, commit the runtime and every seeder to the ids through MoveToPool, assert the pools at boot and in a warmup_pools_missing health check, and drop the guide's claim of cross-tier borrowing the health gate rejects (#493) 2026-09-13 21:37:17 -07:00
Matthew Meszaros adfe4c17aa Self-hosted pool plan: a price the server resolves, and a checkout that reaches it (#494)
* feat: make the self-hosted pool plan buyable by resolving its Stripe price server-side behind a new /pool-link/offer and /pool-link/checkout pair, adding the plans.price_yearly column the yearly price id never had, and landing the instance's Unlimited button on a dialog that names the workspace and the billing period instead of a plans grid the non-public plan never appears in

* feat: apply the pool dialog's yearly default once per opening rather than on every offer result, so a background refetch cannot move the billing period out from under someone who already chose monthly
2026-09-13 21:22:12 -07:00
Matthew Meszaros 6b6efca865 fix: campaign follow-ups opened a new conversation instead of replying in the contact's thread, so carry In-Reply-To/References and the Gmail threadId from the previous send, give every step a reply-in-thread switch, and let a threading step inherit the conversation's subject (issue #472) (#489) 2026-09-13 20:51:41 -07:00
Matthew Meszaros 1dc4aedc3c fix: retire the warmup invalid-token band with its table, metric query, service and repository methods and admin tab, since nothing has fed it since #481 and no attributable forged-token signal exists; key the live pool fixtures on the canonical pool ids so the warmup, repository routing and tasks routing suites run on a fresh database, and correct every doc, site and advisor line that still described the retired signal or a spam-score threshold nothing implements (#490) 2026-09-13 08:09:01 -07:00
Matthew Meszaros 43dcbde06c feat: tester accounts, creatable from the admin panel (#483)
* feat: excuse one named account from the emailed login code, so a vendor reviewer who cannot read this instance's mail can sign in without turning codes off for everyone, with the reason recorded beside it and every run of warmblyctl status naming the accounts that hold one

* feat: create and manage tester accounts from the admin panel, so letting a reviewer in is a form rather than a shell, with the password shown once and every live exemption listed on one page because forgetting one is the way this goes wrong

* fix: give tester management its own permission bit rather than borrowing ban_users, create the account and its exemption in one transaction so no invisible orphan survives a failure, require an accountable operator on the CLI grant, stop a halted row scan reading as the whole exempt list, and show a failed query as an error instead of as no testers

* chore: re-run CI after the aggregator tripped on a cancelled job from the branch update, with every underlying job green

* chore: retrigger CI, the previous run sat queued indefinitely while other branches ran

* feat: roll back a half-created tester when its workspace step fails and backfill the manage-testers bit onto admins already holding every other permission, so the address is not left taken by an unusable account and the new routes are not 403 for the existing admin

* feat: make the 000151 manage-testers backfill one-way, because clearing bit 22 on the way down would also revoke it from an admin granted it explicitly afterwards and the up migration would not restore that
2026-09-13 03:07:10 -07:00
Matthew Meszaros 2edccf812c fix: carry the managed flag on the base subscription type so GET /subscription reports it, because that is the endpoint the dashboard decides entitlements from and the flag only existed on the limits response (#478) 2026-09-12 22:32:14 -07:00
Matthew Meszaros 7f74664c72 fix: a granted plan unlocks nothing, and Turnstile never renders (#474)
* fix: let a granted plan unlock the dashboard, since the client decided paid from the Stripe status a managed subscription never touches, and replace the Turnstile size Cloudflare removed so the widget renders and can issue a token at all

* feat: tell people on a preview deployment that it is a public beta, once in a dialog and thereafter as a header pill they can reopen, driven by a config value rather than a hostname so one image stays reusable, and bind both Turnstile modals through onLoad because the component is not forwardRef and execution=execute never fires without the widget instance

* fix: keep the beta pill outside the desktop-only header group so the notice stays reopenable on a phone, and say in the docs that the value is baked into config.js at container start rather than read per load
2026-09-12 21:45:05 -07:00
Matthew Meszaros 017f4cf60f feat: plans an operator can grant, visible in the admin panel (#467)
* feat: add operator-granted plans so a workspace can be paid without Stripe, surfaced in the admin panel as a badge, a filter and a card carrying who granted it and why, because the only alternative was writing a fake stripe subscription id into the database

* fix: hold a granted plan beside the paid one rather than over it so a Stripe workspace returns to the plan it pays for when the grant ends, route entitlement lookups through EffectivePlanID, separate a repository failure from an unknown plan, end a grant at local end of day, and drop an index that served no query
2026-09-12 09:45:31 -07:00
Matthew Meszaros d456bc48c6 feat: fix the Warmbly Cloud pool link across both roles (#262): take an enrolled mailbox out of this instance's own warmup pool so local partners stop writing to it and their unverifiable warmup stops landing in the owner's unibox, recognise the cloud's warmup mail whose verify header did not survive delivery through a new warmup-deliveries lookup that ignores consumed_at because instance and cloud read the same mailbox, move the managed-mailbox access token route behind NODE_BROKER_TOKEN so the internet-facing tracking and forms services can no longer mint a live provider token, scope pause and resume to the caller's workspace, keep an enrolled mailbox listed once it goes inactive, release the cloud copy when the local mirror row cannot be written, refuse the one-time handshake when CREDENTIALS_ENCRYPTION_KEY is missing, blank an expired code's plaintext instance token, and stop errx answering 200 for a status outside its table 2026-09-12 06:58:25 -07:00
Matthew Meszaros c4aece241b feat: scope the tag, category and folder registries and unibox conversation labels to the organization instead of the creating user, so a teammate sees and can edit the labels the owner made, splitting a label two workspaces shared into one copy each and guarding every label write against ids from another workspace (#457) 2026-09-12 03:37:31 -07:00
Matthew Meszaros dc9ce403de feat: stop one un-sendable lead parking a whole campaign and stop the contact drawer's next-action time walking forward on every refresh (issue #437): route up to config.CampaignPlacementCandidates due leads per pass instead of one, classify a placement refusal that belongs to a single lead (ESP-strict finding no mailbox for that recipient's provider, a bound lead inside its own mailbox's minimum gap or waiting for it to reopen, a recipient's send-time-optimized hours) as the new ErrLeadDeferred so the pass moves to the lead behind them and only defers the campaign when every candidate is refused, log the ESP-strict deferral once a day rather than once per refused lead per tick, and make PreviewContactSend a pure read that answers unchanged state identically on every call by running placement with the even-distribution, jitter, conflict-resolution, distribution-curve and sub-minute layers off, taking a behaviour profile's gap at its floor instead of drawing it, picking the mailbox deterministically instead of re-rolling rotation, reporting the next sending day's first open minute instead of a jittered twenty-four-hours-from-now, and reporting a due step's time as the campaign chain's own stored wakeup 2026-09-12 02:58:48 -07:00
Matthew Meszaros 470654f5b0 feat: say machine_clicks counts the contacts whose only clicks on a step were automated rather than counting steps, document the zero-send rule on all four step rates in the OpenAPI schema, and stop get_campaign_stats dropping the machine open and click counts from both the campaign totals and each step 2026-09-11 09:12:22 -07:00
Matthew Meszaros 5087023e48 feat: give every campaign step its own open, click, reply and bounce rate in Step performance, computed against that step's own sends, with the automated share of its opens and clicks carried alongside them through GetSequenceStats, the campaign analytics API, the get_campaign_stats AI tool and the docs 2026-09-11 08:56:03 -07:00
Matthew Meszaros 9698052569 Merge branch 'main' into feat/cleanmylist-verification 2026-09-11 06:03:55 -07:00
Matthew Meszaros da4b89da0b feat: split a unibox message's provider placement into its own provider_folder column (migration 000146) so Archive and Delete in the thread header survive the next sync without the sync losing the ability to follow a real provider move, and narrow PATCH /unibox/folder to inbox/archive/trash behind the unibox feature gate with an audit entry so the move reaches every teammate's list live 2026-09-11 03:23:19 -07:00
SUMAN JANA 727ddb1482 feat: wire the unibox thread header's Mark as unread, Archive and Delete to a new PATCH /unibox/folder, add an Add as contact action for senders outside the CRM, and replace six private From-header parsers with one shared lib/helper/emailAddress that also understands the parenthesised form the IMAP sync stores, which left the reply composer's seeded To failing its own validator 2026-09-11 03:10:18 -07:00
Chris Edington 59122e6c8a feat: add CleanMyList contact verification with API key setup and built-in fallback 2026-09-10 20:06:29 +01:00
Matthew Meszaros 9e37ea73a8 feat: add a permanent delete for API keys, DELETE /api-keys/:id/permanent plus a Delete key button under a revoked key in the dashboard drawer and warmbly key purge, taking the key's usage logs with it and refusing any key that could still authenticate so revoking stays the step that records why a credential ended (issue #414) 2026-09-10 05:02:05 -07:00
Matthew Meszaros 68b5d8f358 feat: bind a campaign lead to the mailbox that sends its first email so every follow-up leaves from the same address, holding a lead back while its mailbox is merely out of budget or outside its hours and moving it to another mailbox only when that one can no longer send for the campaign at all 2026-09-09 08:51:00 -07:00
Matthew Meszaros 65fe82c472 feat: stop the node container being able to choose the image root runs, by giving it a dedicated uid-1000 subdirectory to write its update target into instead of making the whole state directory writable, validating the tag it asks for before the updater acts on it, and no longer recursively chowning a directory a bare-metal install also keeps its blobs in 2026-09-09 05:41:16 -07:00
Matthew Meszaros 58b7d81d33 Merge remote-tracking branch 'origin/main' into refactor/automatic-worker-mailbox-placement 2026-09-09 04:54:04 -07:00
Matthew Meszaros 435dbb522f feat: replace the worker tier/type/risk-pool/egress categories with a scored placement model and make the fleet pull-based, so a machine joins with one command, workers and consumers share one node registry with usage and liveness, nodes self-update to the version the control plane resolves, and the Hetzner provisioning, worker profiles and SSH orchestrator are removed 2026-09-09 04:54:01 -07:00
Matthew Meszaros db2a9ba439 feat: stamp the rotating campaign send's real sending mailbox onto tasks.email_account_id before dispatch so budgets, round-robin position, deliverability rates and the contact feed name the mailbox that actually sent, and stop the contact email list and campaign-state steps reporting automated prefetches as recipient opens (issue #392) 2026-09-09 03:55:28 -07:00
Matthew Meszaros e783c3e3a0 feat: merge main into the email editor branch 2026-09-08 21:27:28 -07:00
Matthew Meszaros 1e1bb3622e feat: address the CodeRabbit review on the email editor PR by generating email image object keys from a uuid and the sniffed extension instead of the uploader's filename so a name carrying '..' cannot produce a key the public route refuses to serve and a private filename never travels inside a recipient's URL, deleting the stored object before its row and refusing the request when storage will not take it so a failed delete leaves nothing loadable that the quota has stopped counting, accepting only https image addresses, resynchronising the HTML source view when a template replaces the body underneath it, and rendering the campaign preview through the inbox's sandboxed frame so markup written in the source view can neither run nor restyle the dashboard for a teammate 2026-09-08 21:17:36 -07:00
Matthew Meszaros 95a4986d9e feat: merge main into the verification reopen branch 2026-09-08 21:05:37 -07:00
Matthew Meszaros 9ce1e060dd feat: fix the campaign step body editor for issue #380 by registering TipTap's UndoRedo and HardBreak so Ctrl+Z and Shift+Enter work at all, normalising pasted HTML from Gmail/Outlook/Word so their blank-line scaffolding stops rendering a second gap on top of our paragraph margins, adding a workspace email image library (migration 000139, public email-images/ objects sharing the attachment storage quota, keyset-paginated /email-images endpoints) with toolbar upload, URL and library insertion, drag-drop, clipboard paste and a size/alignment/alt bubble, and adding an HTML source view that names the tags the schema would drop before switching back 2026-09-08 21:00:09 -07:00
Matthew Meszaros 0e7525866a feat: merge main into the select-all branch and rebuild the contact selection on top of its new keyset pagination, dropping this branch's own cursor repair now that Search carries the sort key and boundary value in a SortCursor, re-extracting the shared WHERE builder from that version, and pointing SearchIDs at the contactSorts spec and the conditional campaign-count lateral so a resolved selection orders and joins exactly the way the list does 2026-09-08 20:26:34 -07:00
Matthew Meszaros 5a1a726fdc Merge branch 'main' into fix/reopen-verification-on-connect 2026-09-08 20:10:31 -07:00
joao-crm 83ba4b38b4 fix: re-check the addresses the built-in probe left unknown as soon as a workspace connects a paid verifier, and withdraw the connection's degraded state once that verifier answers again, since a verdict reached before the connection existed otherwise waited out the 30-day shelf life while the credits sat unused and the card kept reporting whatever the provider said on the very first pass 2026-09-08 15:48:12 +00:00