Matthew Meszaros
280a3e64ac
feat: publish Kafka-linked image variants ( #448 )
...
* feat: publish Kafka-linked images for backend, consumer, worker and tracking as -kafka tag variants built per-arch on native runners, because the cgo librdkafka link cannot cross-compile on the build-go path
* feat: carry BUILT_AT inside the Go Kafka targets' build-args instead of the shared build-native block, so tracking and realtime stop warning about an unconsumed arg, and emit the matrix with printf because echo expands the separating backslash-n in some shells
2026-09-11 22:31:41 -07:00
Matthew Meszaros
e3092f2322
feat: address the review by releasing the topic lock before the broker call so one slow creation cannot stall every publish, subscribe and close for the full admin timeout, refusing to open an admin connection once the bus is closed so a publish racing past the closed check cannot resurrect a client nothing will shut, building rather than vetting the tagged Kafka backend in CI because vet does not link and a CGO backend fails at link time, and correcting the Confluent tier wording to say auto topic creation is configurable only on Dedicated
2026-09-12 06:54:34 +02:00
Matthew Meszaros
6504d9958a
feat: create Kafka topics from the bus that uses them, because a worker's command topic is named after the node id issued at join time so the set is not knowable in advance, and the broker's auto-creation is off by default on Confluent Cloud and not configurable below Standard, which left a worker subscribed to a topic that did not exist receiving nothing and reporting no error, and compile the tagged Kafka build in CI so a backend nothing else builds cannot rot unnoticed
2026-09-12 06:02:49 +02:00
Matthew Meszaros
131e9ff093
feat: give the JetStream stream a size ceiling from NATS_MAX_BYTES, accepting a byte count or a size like 2GiB, because a managed account can require every stream to declare one and Synadia's Max Bytes Required rejects creation without it, turning that refusal into an error naming the variable to set, and fix the retry path which passed a zero max age and would have recreated the stream with no age limit after a failure
2026-09-12 05:40:43 +02:00
Matthew Meszaros
f1f5249153
feat: authenticate to NATS with a user JWT and nkey seed so a managed bus like Synadia Cloud can replace a self-run one, taking the credential from a file path for containers and from a single-line base64 value for the fleet, because a node receives environment variables rather than files and the env file docker reads cannot express the multi-line credentials format, in both the Go event bus and the Rust tracking publisher
2026-09-12 05:29:09 +02:00
Matthew Meszaros
470654f5b0
feat: say machine_clicks counts the contacts whose only clicks on a step were automated rather than counting steps, document the zero-send rule on all four step rates in the OpenAPI schema, and stop get_campaign_stats dropping the machine open and click counts from both the campaign totals and each step
2026-09-11 09:12:22 -07:00
Matthew Meszaros
5087023e48
feat: give every campaign step its own open, click, reply and bounce rate in Step performance, computed against that step's own sends, with the automated share of its opens and clicks carried alongside them through GetSequenceStats, the campaign analytics API, the get_campaign_stats AI tool and the docs
2026-09-11 08:56:03 -07:00
Matthew Meszaros
a5b0e2c2f3
Merge branch 'main' into feat/cleanmylist-verification
2026-09-11 06:12:10 -07:00
Matthew Meszaros
9698052569
Merge branch 'main' into feat/cleanmylist-verification
2026-09-11 06:03:55 -07:00
Matthew Meszaros
0e8a05b6df
Merge remote-tracking branch 'origin/main' into fix/scanner-timing-window
2026-09-11 04:02:06 -07:00
Matthew Meszaros
9f61d070d4
feat: give the unibox thread header a way back out of a filing mistake, with Undo on the Archive and Delete toasts, Move to inbox while reading the Trash or Archive folder, and the actions disabled while one is in flight, plus stop the invite page offering Accept before it knows which account the browser is signed in as
2026-09-11 03:23:23 -07:00
Matthew Meszaros
da4b89da0b
feat: split a unibox message's provider placement into its own provider_folder column (migration 000146) so Archive and Delete in the thread header survive the next sync without the sync losing the ability to follow a real provider move, and narrow PATCH /unibox/folder to inbox/archive/trash behind the unibox feature gate with an audit entry so the move reaches every teammate's list live
2026-09-11 03:23:19 -07:00
Matthew Meszaros
80c3c79a31
feat: add live coverage that the machine windows survive the settings document's jsonb round trip and reach the classifier, a guard that every shipped scanner CIDR is written as its own network address since the loader truncates host bits silently, and correct the comments that claimed an edit lands on the very next event when the consumer reads through a thirty second cache in its own process
2026-09-11 03:11:35 -07:00
Matthew Meszaros
c115b44d33
feat: correct the docs wording for the automated-engagement windows so the API reference and the campaigns guide name the dispatch-to-worker clock rather than the send, and describe the click window as independently configurable with thirty seconds as its default rather than as a fixed relationship to the open window
2026-09-11 03:02:58 -07:00
Matthew Meszaros
4d0f0fb6b6
feat: hold an exhausted verification account that publishes no balance for a cooldown instead of re-deriving its health from an account check that cannot see exhaustion, since CleanMyList answers GET /v1/jobs identically whether or not there is allowance left, so the minute-long lookup cache retired every observed 402 and put the whole next batch back on doomed paid calls while Settings reported the service as healthy, and refuse a second verification connection while one is connected rather than letting creation order silently move every check onto a different bill
2026-09-11 02:57:08 -07:00
Matthew Meszaros
184a3dc08e
feat: make the automated-open and automated-click windows operator-editable under Instance settings and raise their defaults to 60s and 30s, because the ten-second window was anchored on dispatch to the worker rather than on delivery and routinely expired before the recipient-side gateway it was meant to catch had even seen the message, and add Barracuda's published Email Gateway Defense blocks to the scanner catalogue with Proofpoint, Mimecast and Cisco shipped commented out because browser isolation renders a clicked page from the vendor's own network
2026-09-11 02:53:34 -07:00
Matthew Meszaros
170c33780a
feat: pass ui_host to PostHog in the dashboard, admin panel and marketing site from its own environment variable so a proxied api_host stops breaking toolbar and session-replay links, which the SDK builds against whatever it sends events to and which a reverse proxy does not serve, defaulting to us.posthog.com so an install that does not proxy is unaffected
2026-09-11 11:34:02 +02:00
Matthew Meszaros
1866c45c67
feat: serve a PostHog reverse proxy at /ingest on the backend so the dashboard, admin panel and marketing site can report analytics and errors through this instance instead of posthog.com, which content blockers drop for a large share of visitors, splitting asset traffic to the bundle host because sending it to the ingestion host 404s, withholding the caller's cookies and Authorization from a third party, and preserving the trailing slash that path cleaning removes and PostHog's capture endpoint needs
2026-09-11 06:02:21 +02:00
Matthew Meszaros
853afe458e
Merge pull request #430 from warmbly/fix/bootstrap-and-tracking-domain-check
...
fix: two aws-bootstrap defects, and a check for tracking on your own brand
2026-09-10 20:29:46 -07:00
Matthew Meszaros
eb21d978a8
feat: stop aws-bootstrap from treating a bucket that exists in another region as done, since head-bucket answers globally and a second region silently kept its blobs in the first, stop it printing the database master password to stdout where a terminal, a CI log or an agent transcript keeps it forever, and add an instance check that reports a tracking domain sharing a registered domain with the product's own URLs
2026-09-11 05:24:57 +02:00
Matthew Meszaros
a962683511
feat: ship AWS's RDS truststore in the backend and consumer images and point the docs at it, because Amazon RDS chains to a root that is in no public trust store so the sslmode=verify-full those docs recommended failed every connection with x509 certificate signed by unknown authority, while deliberately not setting PGSSLROOTCERT by default since an RDS-only store would break a Postgres fronted by a public CA
2026-09-11 05:23:23 +02:00
Chris Edington
59122e6c8a
feat: add CleanMyList contact verification with API key setup and built-in fallback
2026-09-10 20:06:29 +01:00
Matthew Meszaros
a84ab48729
Merge branch 'main' into feature/posthog-error-tracking
2026-09-10 10:25:21 -07:00
Matthew Meszaros
2ff350ecf9
feat: apply WARMBLY_POSTHOG_ERROR_TRACKING to the public form pages too, by having cmd/forms stamp an empty browser key when it is false, since the page can only act on whether a key arrived and the flag otherwise silenced the dashboard and the admin panel while leaving form pages reporting
2026-09-10 19:14:34 +02:00
Matthew Meszaros
ced741e352
feat: make PostHog the default error tracker across every runtime while keeping Sentry fully supported alongside or instead of it, by turning internal/observability/errs into a two-sink fan-out with a local-log fallback, adding $exception capture to the Go services, the Rust tracking service, the Elixir realtime service and the dashboard, admin and form apps, reporting gin panics with their route, request id, workspace and user, attaching that identity plus a route and failed-request trail to browser exceptions, and wiring POSTHOG_ERROR_TRACKING, the node join env, compose, source-map upload and the docs to match
2026-09-10 19:11:32 +02:00
Matthew Meszaros
d51de7db3a
feat: address the CodeRabbit review by quoting a fragment in the copy's own casing rather than the model's retyping of it, extracting the case-fold offset map into internal/pkg/casefold so the AI half gets the same Unicode safety the rules half has, giving a trigger term a span in each half it appears in instead of losing the second one to deduplication, scanning subject links before body anchors so the display cap cannot drop the subject's own, requiring WRITE_TEMPLATES on the credit-spending analyze route so a read-only key cannot spend the workspace balance, refusing to tell a customer their credits came back when the refund is what failed, and no longer letting a stale analysis retire the newer rules request that was about to replace it
2026-09-10 09:50:28 -07:00
Matthew Meszaros
5411b1033c
feat: say in the analyze endpoint reference that a finding's category is absent when the model named something outside the documented set, matching the enum the response is now held to
2026-09-10 09:29:35 -07:00
Matthew Meszaros
9f2ddc218c
feat: correct the content-check guide to describe the badge the editor actually renders, which names both halves when the wording straddles them rather than carrying a line number the rules pass never puts there
2026-09-10 09:28:00 -07:00
Matthew Meszaros
f468f44e1d
feat: let a content-check issue's field carry the location instead of repeating it in the message, so the launch dialog and campaign feed stop reading 'Body: 3 spam-trigger term(s) found in subject/body', and say 'Subject and body' for an issue whose fragments straddle both halves rather than dropping the location entirely, with the editor labelling each quoted word on such an issue
2026-09-10 09:28:00 -07:00
Matthew Meszaros
ade18d1650
feat: leave an AI finding's field empty when the model labelled neither half and nothing in the finding could be anchored in the copy, instead of defaulting it to the body and rendering a badge that sends the writer to the wrong box on the one panel whose whole purpose is saying which box to open
2026-09-10 09:28:00 -07:00
Matthew Meszaros
331db196d8
feat: locate every content-check issue in the subject or the body with the exact fragments that caused it and a one-line fix, add POST /templates/analyze running the configured LLM over a campaign template for located spam findings quoted verbatim from the copy plus a rewritten subject and an overall score, verify every model quote against the draft so an invented sentence is dropped rather than shown, pin the analysis temperature so re-checking unchanged copy returns the same number, and give the editor panel a Re-check button that re-runs both passes and reports the movement since the last check
2026-09-10 09:28:00 -07:00
Matthew Meszaros
bbe9d9055a
feat: let web and admin be served from a static host by teaching each app's own entrypoint to render config.js wherever WARMBLY_CONFIG_OUT points, so one definition of the runtime key set serves both the container that renders it at start and a build:pages script that renders it into dist, ship a _redirects in each so a deep link stops 404ing without nginx try_files, and add scripts/check-pages-build.sh to make lint because a malformed config.js reads fine in a diff and leaves the app blank at runtime
2026-09-10 18:03:00 +02:00
Matthew Meszaros
a08ada6666
feat: address the review on the bus bundle by downloading the compose file and config the documented one-command install never fetched, resolving container names through compose so --install-dir stops inspecting containers that do not exist, renewing a certificate that is present but expired instead of starting a stack that refuses every connection, refusing to write an empty credential when openssl fails, and falling back to dig or host where getent does not consult DNS
2026-09-10 17:39:58 +02:00
Matthew Meszaros
9e689e6d91
feat: fix the three defects that stopped the split-deployment bus bundle from starting at all (NATS given store_dir twice on the command line and in nats.conf, which it refuses rather than reconciling; Redis unable to read its own private key because the image's entrypoint drops from root with gosu and discards the added group; and a healthcheck on localhost when the monitor binds IPv4 loopback, leaving the service unhealthy while serving fine), add bus/setup.sh so standing the box up is one idempotent command that verifies with real clients before printing the control plane's URLs, and add scripts/check-split-cloud.sh to make lint so a bundle that cannot start fails CI
2026-09-10 17:22:14 +02:00
Matthew Meszaros
2998f8a8c6
Merge remote-tracking branch 'origin/main' into feat/worker-capacity-soft-target
2026-09-10 06:26:47 -07:00
Matthew Meszaros
fa2b5330d7
feat: drop the auth-pressure placement term because worker_capacity_view aggregates auth_errors (per-mailbox credential failures) and not rate_limit_errors (the 454/421 per-IP throttles it claimed to measure), measure projected utilization against an age-free Capacity.Target so a freshly joined node can relieve a full fleet instead of scoring as 200% loaded after one mailbox, bound the isolated-egress override with an explicit OverTarget check now that Eligible no longer caps it, and cap rotation moves per destination since a tick scores every mailbox against one frozen materialized-view snapshot
2026-09-10 05:51:17 -07:00
Matthew Meszaros
7fa4fdfbc4
feat: make worker capacity a placement target rather than a hard gate, so Eligible refuses only on health and an over-target worker costs enough score to lose to anything with room instead of returning nil and dropping assignment into selectFallback, score projected utilization including the incoming mailbox's own weight, and penalise the 454/421 auth pressure the capacity view already collected and threw away
2026-09-10 05:38:17 -07:00
Matthew Meszaros
396bab0e06
Merge branch 'main' into feat/issue-414-delete-api-key
2026-09-10 05:36:06 -07:00
Matthew Meszaros
1e1231f3b1
feat: address the review of the API key delete, putting the permanent-delete route in the published OpenAPI contract with its 409, keeping one auto margin in the drawer footer so the Delete button lands on the right, and deciding the dashboard's status pill, its footer and the key-count strip on whether the key can still authenticate rather than on a status column that never says expired
2026-09-10 05:23:56 -07:00
Matthew Meszaros
64dfaa8c60
Merge remote-tracking branch 'origin/main' into feat/split-cloud-hosting
2026-09-10 14:20:22 +02:00
Matthew Meszaros
510ee692ba
feat: address the review on the split-deployment branch by moving the two broker routes onto their own NODE_BROKER_TOKEN so the internet-facing tracking and forms services no longer hold a credential that can open any organization's data key, refusing to presign any key outside the prefixes a node reaches, fixing IAM policies that named an alias ARN KMS never resolves in a Resource element, bounding both brokered HTTP clients because the sync loop's context never expires, no longer reporting a 403 from the object store as a missing body, and redacting the DSN and URL credentials the dry-run listing printed in clear
2026-09-10 14:19:53 +02:00
Matthew Meszaros
804ac149fd
feat: name the API key case in the docs' 409 section, so the error-codes page lists the state-based conflict alongside the duplicate-resource one
2026-09-10 05:13:38 -07:00
Matthew Meszaros
9e37ea73a8
feat: add a permanent delete for API keys, DELETE /api-keys/:id/permanent plus a Delete key button under a revoked key in the dashboard drawer and warmbly key purge, taking the key's usage logs with it and refusing any key that could still authenticate so revoking stays the step that records why a credential ended (issue #414 )
2026-09-10 05:02:05 -07:00
Matthew Meszaros
47ba13083e
feat: make a split deployment work end to end by fixing the three defects that made an off-host node impossible to configure (nodeEnvKeys shipped S3_BUCKET and KMS_KEY_ID, which nothing reads, so an AWS-backed node silently used the default bucket and key alias; a joined consumer never received PRIMARY_DB and died at boot; and node.env was rewritten on every join with no file an operator could add to), then removing the need for cloud credentials on a node at all with brokered KMS and blob providers that renderNodeEnv hands out automatically, plus deploy/split-cloud, scripts/aws-bootstrap.sh, two fleet instance checks and the docs
2026-09-10 13:58:59 +02:00
Matthew Meszaros
9d6f71d683
feat: match every word of a contact search against first name, last name, email, company and phone instead of taking the query whole, so "Test Demo" finds the contact whose name is split across two columns (issue #413 ), cap a search at six words, and say in the contacts guide what search matches
2026-09-10 04:53:50 -07:00
Matthew Meszaros
2599d6abb2
feat: stop handing the website-tracking identification ticket to a recognised scanner, which filed its walk of a tracked link as the recipient's own page view, read TRACKING_SCANNER_BUILTINS the way the backend's configuration registry does so off no longer means on, and ship the whole-cloud Microsoft and Google ASN entries commented out because a recipient browsing from Azure or Google Cloud sits inside them
2026-09-10 04:32:52 -07:00
Matthew Meszaros
08f1420c0b
Merge branch 'main' into fix/issue-410-scanner-networks
2026-09-10 04:32:36 -07:00
Matthew Meszaros
82baf6b884
feat: name the source-network rule alongside the timing rules in the campaigns guide's account of what makes a click automated, so the guides page and the analytics page describe the same three rules
2026-09-10 03:41:53 -07:00
Matthew Meszaros
6684dafe3a
feat: classify opens and clicks that arrive from a known mail-filtering network as automated, so Microsoft 365 Defender's delivery-time pixel fetches and Safe Links URL detonations stop counting as engagement (issue #410 ), with a shipped scanner catalogue, TRACKING_SCANNER_* overrides and optional ASN matching from a trusted edge header
2026-09-10 03:38:50 -07:00
Matthew Meszaros
6dcbc6d6a7
feat: stop sending prompt=consent on the Outlook/Microsoft 365 authorize request, which made Entra ID re-run the consent eligibility check per sign-in and refuse every non-admin with AADSTS90095 even under a tenant-wide admin grant (issue #409 ), and ask for prompt=select_account instead so the account picker survives while Google keeps access_type=offline plus forced consent for its refresh token
2026-09-10 03:24:16 -07:00
Matthew Meszaros
Chris Edington