Commit Graph
386 Commits
Author SHA1 Message Date
Matthew Meszaros e737506ba0 feat: recognise a reply typed by hand in a warmup thread by the message it answers (In-Reply-To against warmup sends, receipts and earlier recognised turns, locally and through the pool link), keep it out of the unibox, file it out of the customer's Gmail, Outlook or IMAP inbox with the same folder action, record each recognised turn in warmup_thread_messages so the turn after it is recognised too, and let the daily sweep repair replies that already leaked 2026-09-18 14:12:33 +02:00
Matthew Meszaros a0a19edeae feat: register a schema document whose fixed defaults are code-point strings and whose nested nulls are null so the registered envelope parses back, keep the warmup unibox row until the filing action is on the bus and the mailbox lookup is not a transient failure, recheck the heartbeat key before evacuating a worker, match the IMAP namespace prefix case-insensitively, and state the BACKWARD direction correctly 2026-09-18 11:29:48 +02:00
Matthew Meszaros bd1837833e feat: give every derived Avro field its zero as a default and register the marshalled schema so adding a field cannot refuse the whole envelope and stop every publish, file historical warmup leaks out of the customer's own mailbox rather than only the unibox, and make a worker earn a 10-minute absence before its mailboxes are evacuated so a version rollout costs no migrations 2026-09-18 10:34:36 +02:00
Matthew Meszaros e37c5053c2 feat: make warmup refunds atomic, count confirmed partner diversity in local and cloud mailbox views, and document cloud-safe mailbox deletion 2026-09-17 21:15:28 -07:00
Matthew Meszaros 177a0817c4 Merge remote-tracking branch 'origin/main' into fix/closiqode-reported-issues 2026-09-17 21:00:48 -07:00
Matthew Meszaros 68babc30f3 feat: give the mailbox delete its own cloud revocation that calls the pool before dropping the local row and refuses an unreadable link, so a nil answer is proof the credential is gone rather than proof the local row went, and drive the greylisting evidence guard through the real handler with stub repositories so removing it fails CI where the live test skips 2026-09-17 20:52:54 -07:00
Matthew Meszaros 68c3676717 feat: keep warmup out of the customer's own mailbox and off their deliverability record: Gmail foldering now removes INBOX and SENT instead of only labelling, sent copies and reply-backs are filed in both directions, filing is configurable per mailbox (folder/inbox/archive via warmup_placement + warmup_folder, migration 000177), IMAP relocates a moved message by Message-ID so read/important stop no-opping, and a warmup send's bounce notice no longer lands in the unibox or suppresses a pool partner 2026-09-17 20:46:03 -07:00
Matthew Meszaros 8ee1c50a1b feat: make a failed SMTP send name the step and the cause behind it instead of one bare SERVER_UNREACHABLE sentinel, give a refused warmup send its day back so sent_today can no longer climb past the target while the cap frees the slot, revoke a mailbox's Warmbly Cloud enrollment when it is deleted so the pool stops holding its password, and prefer warmup partners outside the sender's own workspace while showing the partner diversity a mailbox is actually getting (#574, #575) 2026-09-17 20:02:37 -07:00
Matthew Meszaros a900f1e04c feat: make worker moves atomic and preserve safe concentration and health state 2026-09-17 07:45:23 -07:00
Matthew Meszaros 2a1e55354d feat: merge latest main before requeueing worker capacity fixes
# Conflicts:
#	internal/app/stripe/service_test.go
2026-09-17 06:57:20 -07:00
Matthew Meszaros c1e45b194a feat: merge latest main before worker capacity queueing 2026-09-17 06:26:07 -07:00
Matthew Meszaros 8c7827c199 feat: make Stripe credit auto-top-ups idempotent across retries 2026-09-17 15:25:35 +02:00
Matthew Meszaros d6025ea4c0 feat: address worker capacity review findings with safe migrations and recovery reporting 2026-09-17 06:24:14 -07:00
Matthew Meszaros ae1a324801 Merge pull request #562 from rocker1166/feat/direct-mail-analytics
feat: add accurate direct-mail analytics and opt-in engagement tracking
2026-09-17 11:47:50 +00:00
Matthew Meszaros d1aa3d4361 feat: classify ambiguous Outlook and Apple image-proxy opens by delivery timing and stop inventing recipient device metadata 2026-09-17 04:27:34 -07:00
Matthew Meszaros bab9f86727 feat: correct worker capacity, mailbox distribution, observed IPv4, fleet pagination, and premium pool promotion 2026-09-17 04:15:05 -07:00
SUMAN JANA 2134c7a143 feat(analytics): report on mail written by hand, with opt-in open and click tracking per mailbox 2026-09-17 10:26:25 +00:00
Matthew Meszaros 0f5ca71155 feat: correct mailbox sending metrics and workspace analytics across dashboard surfaces 2026-09-16 21:44:42 -07:00
Matthew Meszaros 2255e2145d feat: reject outbound mailbox copies and mismatched campaign threads before they can mark contacts replied for stop-on-reply (issue #549) 2026-09-16 08:24:19 -07:00
Matthew Meszaros 31c1f101c2 feat: give EmailSentEvent and WarmupEmailSentEvent their own derived Avro schemas so the email-events and warmup-events analytics streams stop failing at serialize on every send and finally register a subject (#546) 2026-09-16 04:20:15 -07:00
Matthew Meszaros 140c7de436 feat: add the admin panel's Promo codes page and route the six /admin/discounts endpoints that existed as handlers but were never wired, so a launch offer is built in the operator UI instead of an INSERT against production, with caps that an explicit null can actually clear on PATCH 2026-09-16 04:04:09 -07:00
Matthew Meszaros 746dd40469 feat: strengthen the Avro round-trip tests after a cutover they failed to catch (#536)
* feat: compare the decoded event body's fields and not only its type, fill arrays so every uuid carries a real value instead of the zero one a codec could drop unnoticed, and decode once in a process that has never encoded, because production is four processes and one of them only ever reads what another wrote

* feat: register the union body types at package load instead of on first schema build, which is what a process that only ever decodes never reached, so every worker command arrived as a map keyed by its branch name, went through the JSON fallback, and became a struct with every field zero and no error anywhere
2026-09-15 20:25:21 -07:00
Matthew Meszaros f106c8541d feat: make the bus envelopes Avro-encodable (#535)
* feat: make both bus envelopes Avro-encodable by deriving each one's schema from a declared registry of body types, with a union branch per body and our own struct walk that skips unexported fields and honours avro:"-" before descending, so the schema describes exactly what encoding/json already puts on the wire, and narrow the two sync cursors on the wire DTOs to int64 because Avro has no unsigned 64-bit type

* feat: stop the instance health check, the config registry and the docs all claiming Avro cannot serialize a worker envelope, which stopped being true once the envelopes carried a declared union, and check the one thing that is still a real misconfiguration instead: avro selected with no SCHEMA_REGISTRY_URL to resolve against

* feat: emit a reference the second time a record appears in an envelope schema instead of defining it again, because Avro names a record once and a document that defines warmbly.events.Token three times is rejected outright, and keep the Schema Registry round-trip as a skip-by-default test since only a registry judges the document rather than the objects it was built from

* feat: carry uint64 as Avro fixed(8) rather than long, which lets the sync cursors keep their unsigned type instead of being narrowed, name every event field after its json tag so the schema and the JSON wire agree, and populate every field in the round-trip test because zero values are why a uint64 mapped to long passed in the first place

* feat: frame Avro in Confluent's wire format and encode through hamba's default API instead of going through avrov2, whose private avro.API holds a type resolver avro.Register cannot reach, so a union body failed there with unable to resolve type while encoding cleanly against the same schema, and keep the registry round-trip as a skip-by-default test
2026-09-15 10:50:30 -07:00
Matthew Meszaros e67b13e57e feat: stop reporting a mailbox's DKIM as missing when its selector was simply never probed, by deriving candidate selectors from the sending domain's own SPF and MX records on top of a wider default set, reporting a miss as the tri-state dkim_status undetermined rather than a red Missing row in the drawer, dropping DKIM from the Advisor's missing-records finding entirely, refusing a revoked p= key, holding the summary back from accusing anything when DNS never answered, and fixing the CLI auth-check table whose columns read mailbox fields the endpoint does not return (#528) 2026-09-15 02:27:38 -07:00
Matthew Meszaros 95885fa714 fix: save a contact's edited email address (#511) (#521)
* fix: save a contact's edited email address by giving models.UpdateContact the Email field the dashboard was already sending, normalizing it to a bare lowercased address, refusing one another contact in the workspace holds with a 409 contact_email_taken instead of letting the unique index 500, and dropping the verification verdict plus the delivery evidence that belonged to the old mailbox (issue #511)

* fix: reset a contact's verification evidence behind a watermark the delivery-credit job honours, clear the cached esp_provider the old domain produced, write a case-only edit in place instead of answering 200 and changing nothing, and normalize the address on create and import too so the two paths cannot disagree about what an address is

* fix: refuse a verification observation whose campaign step provably left before the contact's address was edited, so a hard bounce or an open for the old mailbox arriving afterwards cannot mark the corrected address invalid or valid, read the credit watermark against dispatched_at rather than the sent_at a worker result stamps later, and answer a racing address collision with the documented 409 instead of the unique index's 500

* fix: build the deliverability evidence step from the resolved campaign task so both halves name one real row, drop the email format hint from the OpenAPI schema now that the endpoint also accepts a display-name address, and drain the shared delivery-credit backlog in the live test instead of assuming one pass reaches this contact
2026-09-14 23:05:16 -07:00
Matthew Meszaros 50711a8e66 fix: a campaign's linked segments are its audience, so detaching one withdraws its leads (#510) (#523)
* fix: make a campaign's linked segments the audience rather than an accumulator, so detaching one withdraws the leads it enrolled instead of leaving the old list mixed in with the new, tracked by a new campaign_leads.source that keeps a hand-picked lead, an overlapping segment's member and anyone the campaign has already emailed out of the withdrawal, and reported back as withdrawn/contacted counts the dialog confirms and explains (issue #510)

* fix: serialize the linked-segment sweep against a link replacement by taking the same campaign lock, so a sweep that read the old set cannot re-enrol the audience the replacement just withdrew, and word the dialog's confirm and toast so the campaign, not the segment, is what has already emailed a lead

* fix: stop the one-shot segment enrol from re-stamping leads that are already in the campaign, since the Leads tab's Add back runs through it and pinning a whole linked audience as hand-picked because one held-out member was restored is the accumulation this change exists to end

* fix: lock the leads a detachment is about to withdraw before deciding, because a send is reserved by stamping campaign_contact_progress and only then locking the lead row, so a reservation committing mid-pass was invisible to the delete's snapshot and could withdraw a lead whose first email had already gone; and report the already-emailed count on every toast branch, since a detach where the whole audience had been emailed changed no count and said nothing after confirming a removal

* fix: add the campaign_leads.source check constraint NOT VALID, which still enforces every insert and update while skipping a full scan of the largest table in the product under ACCESS EXCLUSIVE to learn that a one-statement-old column holds its own default everywhere
2026-09-14 22:12:15 -07:00
Matthew Meszaros 619eb2729a fix: read a mailbox's Gmail send-as addresses from the worker holding it rather than from the backend, which was decrypting a mailbox credential in the control plane and showing Google a second client address for a mailbox whose mail moves through a worker, by round-tripping a new MAILBOX_IDENTITY command answered on the process channel like a credential validation, leaving the OAuth handshake as the one place the control plane still calls the provider (#522) 2026-09-14 21:28:07 -07:00
Matthew Meszaros 1ca3bd19b3 feat: carry a unibox read or unread change out to the mailbox itself through a new MESSAGE_SEEN worker command, so a conversation read in Warmbly stops showing bold in Gmail, Outlook and IMAP, relaying the state the row holds rather than the one the request asked for, only for messages that actually changed, dispatched detached from the request and never retried (#515) 2026-09-14 21:20:42 -07:00
Matthew Meszaros ffd27bc46d fix: stop every out-of-office notice and bounce opening a high-priority CRM follow-up by classifying machine replies from their headers and gating the task on a per-intent setting, and give the Tasks page multi-select with select-all-matching, bulk status, priority and delete over new PATCH and DELETE /crm/tasks endpoints (issue #471) 2026-09-14 12:20:47 -07:00
Matthew Meszaros 8d790ede6c feat: send from any address Google has verified a Gmail mailbox to send as and import the signature its owner already wrote in Gmail, reading both through gmail.settings.basic at connect and on demand via GET/POST /emails/:id/identity, validating the choice against the provider's own list in the service and again inside the UPDATE, clearing it when the provider stops verifying it, and never applying it to warmup (#514) 2026-09-14 10:13:36 -07:00
Matthew Meszaros 13e9ce8e10 feat: hold a lead whose mailbox answers out of office until they are back, resuming at the return date it names, plus a manual per-contact pause in one campaign that unsubscribing and the suppression list were the only stand-ins for 2026-09-14 09:26:06 -07:00
Matthew Meszaros d74d5e6836 fix: retire the warmup spam score, a counter that grew with volume rather than misbehaviour and that no band could act on (#508)
* fix: retire the warmup spam score, a ratchet that grew with volume rather than misbehaviour and that no band ever read, dropping the column from the pool row and the reputation ledger and explaining a pool finding with the band's own reason instead (#491)

* test: pin the advisor snapshot's pool columns against the scan, since the band's reason now reaches the finding through that select alone (#491)

* fix: hold a warmup sentence's score and reason with the sentence itself, keep the retired spam_score key on the published analytics payload as a deprecated zero, seed the sandbox with severity-shaped scores, and record the raw spam report when the warmup service is absent (#491)
2026-09-14 07:44:34 -07:00
Matthew Meszaros 2f6c027e37 fix: take the warmup health sweep from twelve round-trips per mailbox to two, list participants stalest first, stop at the deadline, and return the standing from the write (#502)
* perf: cut the warmup health sweep from twelve round-trips per mailbox to seven by reading the participant row once, counting placements and complaints in one scan, complaints and bounces in one scan, and taking the spam score from the row already in hand (#492)

* perf: take the warmup health sweep to two round-trips per mailbox (one metrics statement, the write returns the row), list participants stalest first and stop at the deadline, and drop the dead spam-score and count surface (#492)
2026-09-14 03:14:15 -07:00
Matthew Meszaros 2bbd72e758 fix: make cross-tier warmup borrowing real and one-directional: a thin premium tier borrows proven free mailboxes through one repository rule, gates each drawn partner in its own pool, and only reply-backs cross tiers (#496)
* fix: gate a warmup partner borrowed from the other tier against the pool it is in rather than the sender's, since the thin-tier fallback had rejected every borrowed candidate and a thin tier failed instead of borrowing

* fix: make cross-tier warmup borrowing one-directional and gate borrowed partners in their own pool, so a thin premium tier can actually borrow proven free mailboxes (#495)

* fix: pin the borrow floor at the exact boundary so a premium tier at the floor including its sender still borrows (#495)

* fix: put the warmup borrowing rule in one repository method (direction, floor, proven age, workspace standing) that the selector and scheduler both read, pin every drawn partner's gate to its own pool, fall through buckets when a stale row fails the gate, and allow only reply-backs across tiers (#495)

* fix: end the warmup partner draw by candidate exhaustion instead of a fixed attempt cap, and fail closed when a free mailbox's workspace standing cannot be read before it answers into a paid inbox (#495)

* fix: end the warmup partner draw by candidate exhaustion instead of a fixed attempt cap, and fail closed when a free mailbox's workspace standing cannot be read before it answers into a paid inbox (#495)
2026-09-14 02:51:02 -07:00
Matthew Meszaros 40506c4f05 fix: seed the two warmup pools on every instance under fixed ids and make one pool per type structural, since the baseline squash dropped the insert and a fresh self-hosted instance never warmed; move memberships onto the canonical pools, scope the standing mirror trigger to the columns it mirrors so a pool move keeps a retention window, commit the runtime and every seeder to the ids through MoveToPool, assert the pools at boot and in a warmup_pools_missing health check, and drop the guide's claim of cross-tier borrowing the health gate rejects (#493) 2026-09-13 21:37:17 -07:00
Matthew Meszaros adfe4c17aa Self-hosted pool plan: a price the server resolves, and a checkout that reaches it (#494)
* feat: make the self-hosted pool plan buyable by resolving its Stripe price server-side behind a new /pool-link/offer and /pool-link/checkout pair, adding the plans.price_yearly column the yearly price id never had, and landing the instance's Unlimited button on a dialog that names the workspace and the billing period instead of a plans grid the non-public plan never appears in

* feat: apply the pool dialog's yearly default once per opening rather than on every offer result, so a background refetch cannot move the billing period out from under someone who already chose monthly
2026-09-13 21:22:12 -07:00
Matthew Meszaros 6b6efca865 fix: campaign follow-ups opened a new conversation instead of replying in the contact's thread, so carry In-Reply-To/References and the Gmail threadId from the previous send, give every step a reply-in-thread switch, and let a threading step inherit the conversation's subject (issue #472) (#489) 2026-09-13 20:51:41 -07:00
Matthew Meszaros 1dc4aedc3c fix: retire the warmup invalid-token band with its table, metric query, service and repository methods and admin tab, since nothing has fed it since #481 and no attributable forged-token signal exists; key the live pool fixtures on the canonical pool ids so the warmup, repository routing and tasks routing suites run on a fresh database, and correct every doc, site and advisor line that still described the retired signal or a spam-score threshold nothing implements (#490) 2026-09-13 08:09:01 -07:00
Matthew Meszaros 43dcbde06c feat: tester accounts, creatable from the admin panel (#483)
* feat: excuse one named account from the emailed login code, so a vendor reviewer who cannot read this instance's mail can sign in without turning codes off for everyone, with the reason recorded beside it and every run of warmblyctl status naming the accounts that hold one

* feat: create and manage tester accounts from the admin panel, so letting a reviewer in is a form rather than a shell, with the password shown once and every live exemption listed on one page because forgetting one is the way this goes wrong

* fix: give tester management its own permission bit rather than borrowing ban_users, create the account and its exemption in one transaction so no invisible orphan survives a failure, require an accountable operator on the CLI grant, stop a halted row scan reading as the whole exempt list, and show a failed query as an error instead of as no testers

* chore: re-run CI after the aggregator tripped on a cancelled job from the branch update, with every underlying job green

* chore: retrigger CI, the previous run sat queued indefinitely while other branches ran

* feat: roll back a half-created tester when its workspace step fails and backfill the manage-testers bit onto admins already holding every other permission, so the address is not left taken by an unusable account and the new routes are not 403 for the existing admin

* feat: make the 000151 manage-testers backfill one-way, because clearing bit 22 on the way down would also revoke it from an admin granted it explicitly afterwards and the up migration would not restore that
2026-09-13 03:07:10 -07:00
Matthew Meszaros 2edccf812c fix: carry the managed flag on the base subscription type so GET /subscription reports it, because that is the endpoint the dashboard decides entitlements from and the flag only existed on the limits response (#478) 2026-09-12 22:32:14 -07:00
Matthew Meszaros 7f74664c72 fix: a granted plan unlocks nothing, and Turnstile never renders (#474)
* fix: let a granted plan unlock the dashboard, since the client decided paid from the Stripe status a managed subscription never touches, and replace the Turnstile size Cloudflare removed so the widget renders and can issue a token at all

* feat: tell people on a preview deployment that it is a public beta, once in a dialog and thereafter as a header pill they can reopen, driven by a config value rather than a hostname so one image stays reusable, and bind both Turnstile modals through onLoad because the component is not forwardRef and execution=execute never fires without the widget instance

* fix: keep the beta pill outside the desktop-only header group so the notice stays reopenable on a phone, and say in the docs that the value is baked into config.js at container start rather than read per load
2026-09-12 21:45:05 -07:00
Matthew Meszaros 017f4cf60f feat: plans an operator can grant, visible in the admin panel (#467)
* feat: add operator-granted plans so a workspace can be paid without Stripe, surfaced in the admin panel as a badge, a filter and a card carrying who granted it and why, because the only alternative was writing a fake stripe subscription id into the database

* fix: hold a granted plan beside the paid one rather than over it so a Stripe workspace returns to the plan it pays for when the grant ends, route entitlement lookups through EffectivePlanID, separate a repository failure from an unknown plan, end a grant at local end of day, and drop an index that served no query
2026-09-12 09:45:31 -07:00
Matthew Meszaros d456bc48c6 feat: fix the Warmbly Cloud pool link across both roles (#262): take an enrolled mailbox out of this instance's own warmup pool so local partners stop writing to it and their unverifiable warmup stops landing in the owner's unibox, recognise the cloud's warmup mail whose verify header did not survive delivery through a new warmup-deliveries lookup that ignores consumed_at because instance and cloud read the same mailbox, move the managed-mailbox access token route behind NODE_BROKER_TOKEN so the internet-facing tracking and forms services can no longer mint a live provider token, scope pause and resume to the caller's workspace, keep an enrolled mailbox listed once it goes inactive, release the cloud copy when the local mirror row cannot be written, refuse the one-time handshake when CREDENTIALS_ENCRYPTION_KEY is missing, blank an expired code's plaintext instance token, and stop errx answering 200 for a status outside its table 2026-09-12 06:58:25 -07:00
Matthew Meszaros c4aece241b feat: scope the tag, category and folder registries and unibox conversation labels to the organization instead of the creating user, so a teammate sees and can edit the labels the owner made, splitting a label two workspaces shared into one copy each and guarding every label write against ids from another workspace (#457) 2026-09-12 03:37:31 -07:00
Matthew Meszaros dc9ce403de feat: stop one un-sendable lead parking a whole campaign and stop the contact drawer's next-action time walking forward on every refresh (issue #437): route up to config.CampaignPlacementCandidates due leads per pass instead of one, classify a placement refusal that belongs to a single lead (ESP-strict finding no mailbox for that recipient's provider, a bound lead inside its own mailbox's minimum gap or waiting for it to reopen, a recipient's send-time-optimized hours) as the new ErrLeadDeferred so the pass moves to the lead behind them and only defers the campaign when every candidate is refused, log the ESP-strict deferral once a day rather than once per refused lead per tick, and make PreviewContactSend a pure read that answers unchanged state identically on every call by running placement with the even-distribution, jitter, conflict-resolution, distribution-curve and sub-minute layers off, taking a behaviour profile's gap at its floor instead of drawing it, picking the mailbox deterministically instead of re-rolling rotation, reporting the next sending day's first open minute instead of a jittered twenty-four-hours-from-now, and reporting a due step's time as the campaign chain's own stored wakeup 2026-09-12 02:58:48 -07:00
Matthew Meszaros 470654f5b0 feat: say machine_clicks counts the contacts whose only clicks on a step were automated rather than counting steps, document the zero-send rule on all four step rates in the OpenAPI schema, and stop get_campaign_stats dropping the machine open and click counts from both the campaign totals and each step 2026-09-11 09:12:22 -07:00
Matthew Meszaros 5087023e48 feat: give every campaign step its own open, click, reply and bounce rate in Step performance, computed against that step's own sends, with the automated share of its opens and clicks carried alongside them through GetSequenceStats, the campaign analytics API, the get_campaign_stats AI tool and the docs 2026-09-11 08:56:03 -07:00
Matthew Meszaros 9698052569 Merge branch 'main' into feat/cleanmylist-verification 2026-09-11 06:03:55 -07:00
Matthew Meszaros da4b89da0b feat: split a unibox message's provider placement into its own provider_folder column (migration 000146) so Archive and Delete in the thread header survive the next sync without the sync losing the ability to follow a real provider move, and narrow PATCH /unibox/folder to inbox/archive/trash behind the unibox feature gate with an audit entry so the move reaches every teammate's list live 2026-09-11 03:23:19 -07:00
SUMAN JANA 727ddb1482 feat: wire the unibox thread header's Mark as unread, Archive and Delete to a new PATCH /unibox/folder, add an Add as contact action for senders outside the CRM, and replace six private From-header parsers with one shared lib/helper/emailAddress that also understands the parenthesised form the IMAP sync stores, which left the reply composer's seeded To failing its own validator 2026-09-11 03:10:18 -07:00