Commit Graph
529 Commits
Author SHA1 Message Date
Matthew Meszaros d456bc48c6 feat: fix the Warmbly Cloud pool link across both roles (#262): take an enrolled mailbox out of this instance's own warmup pool so local partners stop writing to it and their unverifiable warmup stops landing in the owner's unibox, recognise the cloud's warmup mail whose verify header did not survive delivery through a new warmup-deliveries lookup that ignores consumed_at because instance and cloud read the same mailbox, move the managed-mailbox access token route behind NODE_BROKER_TOKEN so the internet-facing tracking and forms services can no longer mint a live provider token, scope pause and resume to the caller's workspace, keep an enrolled mailbox listed once it goes inactive, release the cloud copy when the local mirror row cannot be written, refuse the one-time handshake when CREDENTIALS_ENCRYPTION_KEY is missing, blank an expired code's plaintext instance token, and stop errx answering 200 for a status outside its table 2026-09-12 06:58:25 -07:00
Matthew Meszaros 06b8db5529 feat: report the one silent state a campaign had no words for, a mailbox pool that is part out of budget and part outside its own sending hours, which fell between the 'every mailbox is capped, sending resumes tomorrow' line and the deliberately unlogged closed-hours band and so left an active campaign sending nothing with an empty activity feed; give it its own line under the mailboxes_unavailable event the pool's other refusals already use, naming how many mailboxes are in each state and carrying the moment the pool comes back in metadata rather than promising a day, while leaving daily_cap_reached to mean exactly what it meant before, every usable mailbox spent and nothing coming back until tomorrow 2026-09-12 03:47:36 -07:00
Matthew Meszaros c4aece241b feat: scope the tag, category and folder registries and unibox conversation labels to the organization instead of the creating user, so a teammate sees and can edit the labels the owner made, splitting a label two workspaces shared into one copy each and guarding every label write against ids from another workspace (#457) 2026-09-12 03:37:31 -07:00
Matthew Meszaros 5869a2148f feat: stop a campaign sending outside its sending window, and past its end date, when a follow-up is overdue: the placer's schedule gates are asked about the moment a step became due, and nextScheduleSlot deliberately returns an instant that was already inside a window unchanged, so a step that came due at 2pm still read as due at 11pm and the task sent it there, while the end-date comparison found a candidate predating the end date; floor an overdue candidate at now so the window, the weekday mask and the end date are all asked about the send that is actually about to happen, with live tests for a closed window, a passed end date and the overdue step that must still go out promptly while the window is open 2026-09-12 03:29:33 -07:00
Matthew Meszaros 47defafa09 feat: fix the six self-host defects reported in issue #439 (#456)
* feat: fix the six defects reported in issue #439 by mapping the IMAP UNAVAILABLE, INUSE and NONEXISTENT response codes to retry-level errors instead of a critical reconnect prompt, synthesising a stable no-msgid key so one message with no Message-ID header can no longer 400 the internal map endpoint and wedge every later sync pass with its cursors held, adding mailhtml.FromText and HasContent so an API or agent-created step with a plain body stops shipping the composer's empty div placeholder as its text/html part (derived on create and plain-only update, exposed as body_html on update_campaign_step, dropped at send and preview time, and refused at campaign start with empty_step_body), honouring sender_strategy='explicit' in ResolveCampaignSenderPool and ValidateCampaignReady so an emptied explicit pool parks the campaign instead of widening it to every mailbox in the workspace, making the paused_no_accounts auto-pause loud with an error log line, an error-level activity-feed entry and an org-scoped CAMPAIGN_PAUSED realtime pulse, gating the admin sign-in's Turnstile widget on GET /v1/auth/config so a self-host with CAPTCHA_PROVIDER=none is not locked out, and parsing NATS_URL down to its host:port so a credentialed bus URL no longer reports NATS down

* feat: act on the self-review of the issue #439 fixes by dropping the campaign wizard's own escapeHtml body_html builder, which entity-escaped the quotes in a conditional and made the template fail to parse at send time, and letting the backend's FromText render that part instead so wizard-written steps also get their bare URLs linked for click tracking, correcting the docs and openapi description that claimed an explicit sender pool never falls back when it still unions its tags as migration 000013 designed, extracting the duplicated blank-HTML-part guard into dropBlankHTMLPart shared by the send path and the preview, and recording why the no-msgid key keeps the folder name despite a RENAME changing it

* feat: address the CodeRabbit review on the issue #439 fixes by holding the admin sign-in's Turnstile widget unmounted until /v1/auth/config resolves so an instance with no route to Cloudflare cannot raise a widget error on a screen nobody submitted, failing StartCampaign closed when the sequence read errors rather than skipping both the malformed-template and empty-body refusals, giving TCPCheck the default port its protocol assumes so a portless NATS_URL is no longer reported down, leaving a URL that carries a merge field unanchored because the send path renders bodies with text/template and a quoted contact value would break out of the href, and correcting the sequences guide and the Campaign and CampaignUpdate openapi descriptions that named the wrong tag field
2026-09-12 03:13:38 -07:00
Matthew Meszaros dc9ce403de feat: stop one un-sendable lead parking a whole campaign and stop the contact drawer's next-action time walking forward on every refresh (issue #437): route up to config.CampaignPlacementCandidates due leads per pass instead of one, classify a placement refusal that belongs to a single lead (ESP-strict finding no mailbox for that recipient's provider, a bound lead inside its own mailbox's minimum gap or waiting for it to reopen, a recipient's send-time-optimized hours) as the new ErrLeadDeferred so the pass moves to the lead behind them and only defers the campaign when every candidate is refused, log the ESP-strict deferral once a day rather than once per refused lead per tick, and make PreviewContactSend a pure read that answers unchanged state identically on every call by running placement with the even-distribution, jitter, conflict-resolution, distribution-curve and sub-minute layers off, taking a behaviour profile's gap at its floor instead of drawing it, picking the mailbox deterministically instead of re-rolling rotation, reporting the next sending day's first open minute instead of a jittered twenty-four-hours-from-now, and reporting a due step's time as the campaign chain's own stored wakeup 2026-09-12 02:58:48 -07:00
Matthew Meszaros b0050507e4 fix: tell a joining node a tag that exists, falling back to the published prod tag instead of a latest this project has never published, and drop the WORKER_IMAGE docs left behind by the removed push-based provisioning (#455) 2026-09-12 01:00:11 -07:00
Matthew Meszaros a4f503f620 fix: realtime cannot complete either of its TLS connections (#452)
* fix: let the realtime service actually establish its TLS connections, adding the wildcard hostname match fun Redix needs for every managed Redis and a CA bundle option for an RDS Postgres that the system trust store cannot verify

* feat: drop the redundant wget package from the realtime image since busybox already provides it, and fix a comment that read as literal interpolation

* fix: treat a blank DATABASE_SSL_CA_FILE as unset in the realtime config, because compose passes every optional variable through as empty and cacertfile with an empty path is no CA source at all rather than a fallback to the system store
2026-09-12 00:29:34 -07:00
Matthew Meszaros 8a2856fce0 Merge branch 'main' into feature/editor-image-links-and-buttons 2026-09-11 23:02:15 -07:00
Matthew Meszaros e8186e5f52 fix: the tracking service must not write Avro to a JSON consumer (#450)
* feat: make the Rust tracking publisher honour CODEC_PROVIDER on Kafka instead of always writing Avro, so a json consumer stops silently dropping every open and click, and refuse avro at boot when no Schema Registry is configured

* feat: build the tracking service's kafka feature in CI, because clippy on the default build never opens kafka.rs and that file now ships as the published tracking -kafka image

* feat: install libcurl and the rest of the librdkafka build dependencies for the tracking kafka clippy step, which fails at the first object without curl headers even with WITH_CURL=0
2026-09-11 23:00:15 -07:00
Matthew Meszaros ea8d15374e Merge branch 'main' into feature/editor-image-links-and-buttons 2026-09-11 22:56:16 -07:00
Matthew Meszaros 88b483d227 Merge branch 'main' into fix/issue-432 2026-09-11 22:47:54 -07:00
Matthew Meszaros 7d79dcc282 feat: append a -kafka image variant to every version the control plane hands a fleet node when the instance runs Kafka, so a joining worker pulls a build that can actually reach the bus instead of failing at boot (#449) 2026-09-11 22:46:02 -07:00
Matthew Meszaros f41eac289d Merge branch 'main' into fix/issue-432 2026-09-11 22:35:43 -07:00
Matthew Meszaros 20b13dcea7 Merge branch 'main' into feature/editor-image-links-and-buttons 2026-09-11 22:34:38 -07:00
Matthew Meszaros 280a3e64ac feat: publish Kafka-linked image variants (#448)
* feat: publish Kafka-linked images for backend, consumer, worker and tracking as -kafka tag variants built per-arch on native runners, because the cgo librdkafka link cannot cross-compile on the build-go path

* feat: carry BUILT_AT inside the Go Kafka targets' build-args instead of the shared build-native block, so tracking and realtime stop warning about an unconsumed arg, and emit the matrix with printf because echo expands the separating backslash-n in some shells
2026-09-11 22:31:41 -07:00
Matthew Meszaros a3aec0a9cb Merge branch 'main' into feature/editor-image-links-and-buttons 2026-09-11 22:27:11 -07:00
Matthew Meszaros e3092f2322 feat: address the review by releasing the topic lock before the broker call so one slow creation cannot stall every publish, subscribe and close for the full admin timeout, refusing to open an admin connection once the bus is closed so a publish racing past the closed check cannot resurrect a client nothing will shut, building rather than vetting the tagged Kafka backend in CI because vet does not link and a CGO backend fails at link time, and correcting the Confluent tier wording to say auto topic creation is configurable only on Dedicated 2026-09-12 06:54:34 +02:00
Matthew Meszaros 6504d9958a feat: create Kafka topics from the bus that uses them, because a worker's command topic is named after the node id issued at join time so the set is not knowable in advance, and the broker's auto-creation is off by default on Confluent Cloud and not configurable below Standard, which left a worker subscribed to a topic that did not exist receiving nothing and reporting no error, and compile the tagged Kafka build in CI so a backend nothing else builds cannot rot unnoticed 2026-09-12 06:02:49 +02:00
Matthew Meszaros 704e793c8a Merge branch 'main' into feature/editor-image-links-and-buttons 2026-09-11 20:57:20 -07:00
Matthew Meszaros 131e9ff093 feat: give the JetStream stream a size ceiling from NATS_MAX_BYTES, accepting a byte count or a size like 2GiB, because a managed account can require every stream to declare one and Synadia's Max Bytes Required rejects creation without it, turning that refusal into an error naming the variable to set, and fix the retry path which passed a zero max age and would have recreated the stream with no age limit after a failure 2026-09-12 05:40:43 +02:00
Matthew Meszaros f1f5249153 feat: authenticate to NATS with a user JWT and nkey seed so a managed bus like Synadia Cloud can replace a self-run one, taking the credential from a file path for containers and from a single-line base64 value for the fleet, because a node receives environment variables rather than files and the env file docker reads cannot express the multi-line credentials format, in both the Go event bus and the Rust tracking publisher 2026-09-12 05:29:09 +02:00
Matthew Meszaros 6501cb599c feat: fix the "Edit with AI" rewrite in the campaign body and the unibox composer for issue #432 by running /generation/edit on a new generation.BuildEditRules system prompt through AIProvider.Complete instead of the cold-outreach writer prompt that redefined the model's role, capped it at 80 words and imposed a five-part email skeleton on every instruction, raising the completion cap so a full-body rewrite is no longer truncated at 1024 tokens and counting the request limits in runes rather than bytes, carrying merge variables, AI blocks, conditionals, form links and link destinations through the round trip in web/src/components/app/ai/richTextPassage.ts instead of deleting every atom node via doc.textBetween, replacing the passage with paste semantics so a phrase rewritten inside a sentence stops splitting its paragraph into three, saying "No change" when the model hands the passage back untouched, and clamping the floating AI card to the surface it is editing so it no longer draws outside the step drawer over the flow canvas 2026-09-11 20:04:32 -07:00
Matthew Meszaros 3ee636e2e6 Merge branch 'main' into feature/editor-image-links-and-buttons 2026-09-11 20:01:18 -07:00
Matthew Meszaros 7ff9f4d2c6 feat: let a campaign body image carry a link and add a call-to-action button to the editor toolbar, both rendered as mail-safe markup (an anchor-wrapped img, a one-cell table whose cell holds the colour and padding for Outlook's Word engine), with the plain-text half now keeping every link destination 2026-09-11 20:01:16 -07:00
Matthew Meszaros 470654f5b0 feat: say machine_clicks counts the contacts whose only clicks on a step were automated rather than counting steps, document the zero-send rule on all four step rates in the OpenAPI schema, and stop get_campaign_stats dropping the machine open and click counts from both the campaign totals and each step 2026-09-11 09:12:22 -07:00
Matthew Meszaros 5087023e48 feat: give every campaign step its own open, click, reply and bounce rate in Step performance, computed against that step's own sends, with the automated share of its opens and clicks carried alongside them through GetSequenceStats, the campaign analytics API, the get_campaign_stats AI tool and the docs 2026-09-11 08:56:03 -07:00
Matthew Meszaros a5b0e2c2f3 Merge branch 'main' into feat/cleanmylist-verification 2026-09-11 06:12:10 -07:00
Matthew Meszaros 9698052569 Merge branch 'main' into feat/cleanmylist-verification 2026-09-11 06:03:55 -07:00
Matthew Meszaros 0e8a05b6df Merge remote-tracking branch 'origin/main' into fix/scanner-timing-window 2026-09-11 04:02:06 -07:00
Matthew Meszaros 9f61d070d4 feat: give the unibox thread header a way back out of a filing mistake, with Undo on the Archive and Delete toasts, Move to inbox while reading the Trash or Archive folder, and the actions disabled while one is in flight, plus stop the invite page offering Accept before it knows which account the browser is signed in as 2026-09-11 03:23:23 -07:00
Matthew Meszaros da4b89da0b feat: split a unibox message's provider placement into its own provider_folder column (migration 000146) so Archive and Delete in the thread header survive the next sync without the sync losing the ability to follow a real provider move, and narrow PATCH /unibox/folder to inbox/archive/trash behind the unibox feature gate with an audit entry so the move reaches every teammate's list live 2026-09-11 03:23:19 -07:00
Matthew Meszaros 80c3c79a31 feat: add live coverage that the machine windows survive the settings document's jsonb round trip and reach the classifier, a guard that every shipped scanner CIDR is written as its own network address since the loader truncates host bits silently, and correct the comments that claimed an edit lands on the very next event when the consumer reads through a thirty second cache in its own process 2026-09-11 03:11:35 -07:00
Matthew Meszaros c115b44d33 feat: correct the docs wording for the automated-engagement windows so the API reference and the campaigns guide name the dispatch-to-worker clock rather than the send, and describe the click window as independently configurable with thirty seconds as its default rather than as a fixed relationship to the open window 2026-09-11 03:02:58 -07:00
Matthew Meszaros 4d0f0fb6b6 feat: hold an exhausted verification account that publishes no balance for a cooldown instead of re-deriving its health from an account check that cannot see exhaustion, since CleanMyList answers GET /v1/jobs identically whether or not there is allowance left, so the minute-long lookup cache retired every observed 402 and put the whole next batch back on doomed paid calls while Settings reported the service as healthy, and refuse a second verification connection while one is connected rather than letting creation order silently move every check onto a different bill 2026-09-11 02:57:08 -07:00
Matthew Meszaros 184a3dc08e feat: make the automated-open and automated-click windows operator-editable under Instance settings and raise their defaults to 60s and 30s, because the ten-second window was anchored on dispatch to the worker rather than on delivery and routinely expired before the recipient-side gateway it was meant to catch had even seen the message, and add Barracuda's published Email Gateway Defense blocks to the scanner catalogue with Proofpoint, Mimecast and Cisco shipped commented out because browser isolation renders a clicked page from the vendor's own network 2026-09-11 02:53:34 -07:00
Matthew Meszaros 170c33780a feat: pass ui_host to PostHog in the dashboard, admin panel and marketing site from its own environment variable so a proxied api_host stops breaking toolbar and session-replay links, which the SDK builds against whatever it sends events to and which a reverse proxy does not serve, defaulting to us.posthog.com so an install that does not proxy is unaffected 2026-09-11 11:34:02 +02:00
Matthew Meszaros 1866c45c67 feat: serve a PostHog reverse proxy at /ingest on the backend so the dashboard, admin panel and marketing site can report analytics and errors through this instance instead of posthog.com, which content blockers drop for a large share of visitors, splitting asset traffic to the bundle host because sending it to the ingestion host 404s, withholding the caller's cookies and Authorization from a third party, and preserving the trailing slash that path cleaning removes and PostHog's capture endpoint needs 2026-09-11 06:02:21 +02:00
Matthew Meszaros 853afe458e Merge pull request #430 from warmbly/fix/bootstrap-and-tracking-domain-check
fix: two aws-bootstrap defects, and a check for tracking on your own brand
2026-09-10 20:29:46 -07:00
Matthew Meszaros eb21d978a8 feat: stop aws-bootstrap from treating a bucket that exists in another region as done, since head-bucket answers globally and a second region silently kept its blobs in the first, stop it printing the database master password to stdout where a terminal, a CI log or an agent transcript keeps it forever, and add an instance check that reports a tracking domain sharing a registered domain with the product's own URLs 2026-09-11 05:24:57 +02:00
Matthew Meszaros a962683511 feat: ship AWS's RDS truststore in the backend and consumer images and point the docs at it, because Amazon RDS chains to a root that is in no public trust store so the sslmode=verify-full those docs recommended failed every connection with x509 certificate signed by unknown authority, while deliberately not setting PGSSLROOTCERT by default since an RDS-only store would break a Postgres fronted by a public CA 2026-09-11 05:23:23 +02:00
Chris Edington 59122e6c8a feat: add CleanMyList contact verification with API key setup and built-in fallback 2026-09-10 20:06:29 +01:00
Matthew Meszaros a84ab48729 Merge branch 'main' into feature/posthog-error-tracking 2026-09-10 10:25:21 -07:00
Matthew Meszaros 2ff350ecf9 feat: apply WARMBLY_POSTHOG_ERROR_TRACKING to the public form pages too, by having cmd/forms stamp an empty browser key when it is false, since the page can only act on whether a key arrived and the flag otherwise silenced the dashboard and the admin panel while leaving form pages reporting 2026-09-10 19:14:34 +02:00
Matthew Meszaros ced741e352 feat: make PostHog the default error tracker across every runtime while keeping Sentry fully supported alongside or instead of it, by turning internal/observability/errs into a two-sink fan-out with a local-log fallback, adding $exception capture to the Go services, the Rust tracking service, the Elixir realtime service and the dashboard, admin and form apps, reporting gin panics with their route, request id, workspace and user, attaching that identity plus a route and failed-request trail to browser exceptions, and wiring POSTHOG_ERROR_TRACKING, the node join env, compose, source-map upload and the docs to match 2026-09-10 19:11:32 +02:00
Matthew Meszaros d51de7db3a feat: address the CodeRabbit review by quoting a fragment in the copy's own casing rather than the model's retyping of it, extracting the case-fold offset map into internal/pkg/casefold so the AI half gets the same Unicode safety the rules half has, giving a trigger term a span in each half it appears in instead of losing the second one to deduplication, scanning subject links before body anchors so the display cap cannot drop the subject's own, requiring WRITE_TEMPLATES on the credit-spending analyze route so a read-only key cannot spend the workspace balance, refusing to tell a customer their credits came back when the refund is what failed, and no longer letting a stale analysis retire the newer rules request that was about to replace it 2026-09-10 09:50:28 -07:00
Matthew Meszaros 5411b1033c feat: say in the analyze endpoint reference that a finding's category is absent when the model named something outside the documented set, matching the enum the response is now held to 2026-09-10 09:29:35 -07:00
Matthew Meszaros 9f2ddc218c feat: correct the content-check guide to describe the badge the editor actually renders, which names both halves when the wording straddles them rather than carrying a line number the rules pass never puts there 2026-09-10 09:28:00 -07:00
Matthew Meszaros f468f44e1d feat: let a content-check issue's field carry the location instead of repeating it in the message, so the launch dialog and campaign feed stop reading 'Body: 3 spam-trigger term(s) found in subject/body', and say 'Subject and body' for an issue whose fragments straddle both halves rather than dropping the location entirely, with the editor labelling each quoted word on such an issue 2026-09-10 09:28:00 -07:00
Matthew Meszaros ade18d1650 feat: leave an AI finding's field empty when the model labelled neither half and nothing in the finding could be anchored in the copy, instead of defaulting it to the body and rendering a badge that sends the writer to the wrong box on the one panel whose whole purpose is saying which box to open 2026-09-10 09:28:00 -07:00