Commit Graph
441 Commits
Author SHA1 Message Date
Matthew Meszaros 2ff350ecf9 feat: apply WARMBLY_POSTHOG_ERROR_TRACKING to the public form pages too, by having cmd/forms stamp an empty browser key when it is false, since the page can only act on whether a key arrived and the flag otherwise silenced the dashboard and the admin panel while leaving form pages reporting 2026-09-10 19:14:34 +02:00
Matthew Meszaros ced741e352 feat: make PostHog the default error tracker across every runtime while keeping Sentry fully supported alongside or instead of it, by turning internal/observability/errs into a two-sink fan-out with a local-log fallback, adding $exception capture to the Go services, the Rust tracking service, the Elixir realtime service and the dashboard, admin and form apps, reporting gin panics with their route, request id, workspace and user, attaching that identity plus a route and failed-request trail to browser exceptions, and wiring POSTHOG_ERROR_TRACKING, the node join env, compose, source-map upload and the docs to match 2026-09-10 19:11:32 +02:00
Matthew Meszaros bbe9d9055a feat: let web and admin be served from a static host by teaching each app's own entrypoint to render config.js wherever WARMBLY_CONFIG_OUT points, so one definition of the runtime key set serves both the container that renders it at start and a build:pages script that renders it into dist, ship a _redirects in each so a deep link stops 404ing without nginx try_files, and add scripts/check-pages-build.sh to make lint because a malformed config.js reads fine in a diff and leaves the app blank at runtime 2026-09-10 18:03:00 +02:00
Matthew Meszaros a08ada6666 feat: address the review on the bus bundle by downloading the compose file and config the documented one-command install never fetched, resolving container names through compose so --install-dir stops inspecting containers that do not exist, renewing a certificate that is present but expired instead of starting a stack that refuses every connection, refusing to write an empty credential when openssl fails, and falling back to dig or host where getent does not consult DNS 2026-09-10 17:39:58 +02:00
Matthew Meszaros 9e689e6d91 feat: fix the three defects that stopped the split-deployment bus bundle from starting at all (NATS given store_dir twice on the command line and in nats.conf, which it refuses rather than reconciling; Redis unable to read its own private key because the image's entrypoint drops from root with gosu and discards the added group; and a healthcheck on localhost when the monitor binds IPv4 loopback, leaving the service unhealthy while serving fine), add bus/setup.sh so standing the box up is one idempotent command that verifies with real clients before printing the control plane's URLs, and add scripts/check-split-cloud.sh to make lint so a bundle that cannot start fails CI 2026-09-10 17:22:14 +02:00
Matthew Meszaros 2998f8a8c6 Merge remote-tracking branch 'origin/main' into feat/worker-capacity-soft-target 2026-09-10 06:26:47 -07:00
Matthew Meszaros fa2b5330d7 feat: drop the auth-pressure placement term because worker_capacity_view aggregates auth_errors (per-mailbox credential failures) and not rate_limit_errors (the 454/421 per-IP throttles it claimed to measure), measure projected utilization against an age-free Capacity.Target so a freshly joined node can relieve a full fleet instead of scoring as 200% loaded after one mailbox, bound the isolated-egress override with an explicit OverTarget check now that Eligible no longer caps it, and cap rotation moves per destination since a tick scores every mailbox against one frozen materialized-view snapshot 2026-09-10 05:51:17 -07:00
Matthew Meszaros 7fa4fdfbc4 feat: make worker capacity a placement target rather than a hard gate, so Eligible refuses only on health and an over-target worker costs enough score to lose to anything with room instead of returning nil and dropping assignment into selectFallback, score projected utilization including the incoming mailbox's own weight, and penalise the 454/421 auth pressure the capacity view already collected and threw away 2026-09-10 05:38:17 -07:00
Matthew Meszaros 396bab0e06 Merge branch 'main' into feat/issue-414-delete-api-key 2026-09-10 05:36:06 -07:00
Matthew Meszaros 1e1231f3b1 feat: address the review of the API key delete, putting the permanent-delete route in the published OpenAPI contract with its 409, keeping one auto margin in the drawer footer so the Delete button lands on the right, and deciding the dashboard's status pill, its footer and the key-count strip on whether the key can still authenticate rather than on a status column that never says expired 2026-09-10 05:23:56 -07:00
Matthew Meszaros 64dfaa8c60 Merge remote-tracking branch 'origin/main' into feat/split-cloud-hosting 2026-09-10 14:20:22 +02:00
Matthew Meszaros 510ee692ba feat: address the review on the split-deployment branch by moving the two broker routes onto their own NODE_BROKER_TOKEN so the internet-facing tracking and forms services no longer hold a credential that can open any organization's data key, refusing to presign any key outside the prefixes a node reaches, fixing IAM policies that named an alias ARN KMS never resolves in a Resource element, bounding both brokered HTTP clients because the sync loop's context never expires, no longer reporting a 403 from the object store as a missing body, and redacting the DSN and URL credentials the dry-run listing printed in clear 2026-09-10 14:19:53 +02:00
Matthew Meszaros 804ac149fd feat: name the API key case in the docs' 409 section, so the error-codes page lists the state-based conflict alongside the duplicate-resource one 2026-09-10 05:13:38 -07:00
Matthew Meszaros 9e37ea73a8 feat: add a permanent delete for API keys, DELETE /api-keys/:id/permanent plus a Delete key button under a revoked key in the dashboard drawer and warmbly key purge, taking the key's usage logs with it and refusing any key that could still authenticate so revoking stays the step that records why a credential ended (issue #414) 2026-09-10 05:02:05 -07:00
Matthew Meszaros 47ba13083e feat: make a split deployment work end to end by fixing the three defects that made an off-host node impossible to configure (nodeEnvKeys shipped S3_BUCKET and KMS_KEY_ID, which nothing reads, so an AWS-backed node silently used the default bucket and key alias; a joined consumer never received PRIMARY_DB and died at boot; and node.env was rewritten on every join with no file an operator could add to), then removing the need for cloud credentials on a node at all with brokered KMS and blob providers that renderNodeEnv hands out automatically, plus deploy/split-cloud, scripts/aws-bootstrap.sh, two fleet instance checks and the docs 2026-09-10 13:58:59 +02:00
Matthew Meszaros 9d6f71d683 feat: match every word of a contact search against first name, last name, email, company and phone instead of taking the query whole, so "Test Demo" finds the contact whose name is split across two columns (issue #413), cap a search at six words, and say in the contacts guide what search matches 2026-09-10 04:53:50 -07:00
Matthew Meszaros 2599d6abb2 feat: stop handing the website-tracking identification ticket to a recognised scanner, which filed its walk of a tracked link as the recipient's own page view, read TRACKING_SCANNER_BUILTINS the way the backend's configuration registry does so off no longer means on, and ship the whole-cloud Microsoft and Google ASN entries commented out because a recipient browsing from Azure or Google Cloud sits inside them 2026-09-10 04:32:52 -07:00
Matthew Meszaros 08f1420c0b Merge branch 'main' into fix/issue-410-scanner-networks 2026-09-10 04:32:36 -07:00
Matthew Meszaros 82baf6b884 feat: name the source-network rule alongside the timing rules in the campaigns guide's account of what makes a click automated, so the guides page and the analytics page describe the same three rules 2026-09-10 03:41:53 -07:00
Matthew Meszaros 6684dafe3a feat: classify opens and clicks that arrive from a known mail-filtering network as automated, so Microsoft 365 Defender's delivery-time pixel fetches and Safe Links URL detonations stop counting as engagement (issue #410), with a shipped scanner catalogue, TRACKING_SCANNER_* overrides and optional ASN matching from a trusted edge header 2026-09-10 03:38:50 -07:00
Matthew Meszaros 6dcbc6d6a7 feat: stop sending prompt=consent on the Outlook/Microsoft 365 authorize request, which made Entra ID re-run the consent eligibility check per sign-in and refuse every non-admin with AADSTS90095 even under a tenant-wide admin grant (issue #409), and ask for prompt=select_account instead so the account picker survives while Google keeps access_type=offline plus forced consent for its refresh token 2026-09-10 03:24:16 -07:00
Matthew Meszaros 817233cc09 feat: record one unresolved row per mailbox error code instead of one per sync pass, and clear IMAP_UNKNOWN when a pass completes (issue #405) 2026-09-09 09:30:17 -07:00
Matthew Meszaros cab27fac62 Merge remote-tracking branch 'origin/main' into fix/main-ci-failure-and-issue-400 2026-09-09 09:05:45 -07:00
Matthew Meszaros 9356c748b9 Merge remote-tracking branch 'origin/main' into fix/main-ci-failure-and-issue-400
# Conflicts:
#	docs/content/docs/guides/mailboxes.mdx
#	site/public/install.sh.sha256
2026-09-09 08:59:37 -07:00
Matthew Meszaros 3591d64404 Merge remote-tracking branch 'origin/main' into fix/issue-401 2026-09-09 08:58:08 -07:00
Matthew Meszaros 18a8c7b009 feat: give a self-hosted instance on-demand TLS for customer tracking and forms domains, gating Caddy's ask on a /tls/authorize endpoint that answers only for domains this instance has verified, so a workspace CNAME stops serving every tracked link and opt-out link with no certificate 2026-09-09 08:56:08 -07:00
Matthew Meszaros 494996ea8a feat: stop the backend image copying scripts/install-worker.sh, which the pull-based fleet change deleted, and clear the rest of that removal's dangling references out of the backend systemd unit, deploy/README, architecture and troubleshooting docs 2026-09-09 08:56:05 -07:00
Matthew Meszaros 16261de4cb Merge remote-tracking branch 'origin/main' into fix/issue-401 2026-09-09 08:51:02 -07:00
Matthew Meszaros 68b5d8f358 feat: bind a campaign lead to the mailbox that sends its first email so every follow-up leaves from the same address, holding a lead back while its mailbox is merely out of budget or outside its hours and moving it to another mailbox only when that one can no longer send for the campaign at all 2026-09-09 08:51:00 -07:00
Matthew Meszaros 3153f9ff23 Merge remote-tracking branch 'origin/main' into fix/self-hosted-unsubscribe-domain 2026-09-09 08:44:13 -07:00
Matthew Meszaros b18bbcad7a feat: name API_HOST as the listen address the unsubscribe link falls back to when API_PUBLIC_URL is unset, rather than describing the resulting localhost address alone 2026-09-09 08:34:04 -07:00
Matthew Meszaros 035865e5c5 Merge remote-tracking branch 'origin/main' into feat/issue-393-html-css-email 2026-09-09 08:30:52 -07:00
Matthew Meszaros f5eeac7b2a feat: address the CodeRabbit review by carrying every stretch of stylesheet the CSS parser cannot read through as a verbatim item, since the sheet is rewritten from parsed items the moment any rule inlines and an unrelated match was deleting the rest, keeping a link's destination out of the content score now that the text renderer emits it so a CTA pointing at a free-trial page stops costing eight points, promising inlining in the editor only for a stylesheet that is actually eligible for it, skipping the client notes entirely for a plain-text campaign that ships no HTML part, switching a step into HTML mode when a template replaces its body with document markup rather than waiting for the next visual edit to gut it, accepting a pasted background shorthand only when it is a single colour so Word's "yellow none repeat scroll" stops becoming an invalid longhand, listing in SCHEMA_TAGS only the tags the mounted schema actually keeps so the warning fires for h1, font, center, thead and caption instead of staying silent while they are dropped, and correcting the guide's byte-for-byte claim and its unconditional plain-text claim 2026-09-09 08:30:49 -07:00
Matthew Meszaros 6ebf9cfdcf Merge remote-tracking branch 'origin/main' into fix/self-hosted-unsubscribe-domain 2026-09-09 08:24:04 -07:00
Matthew Meszaros a96a44d916 Merge remote-tracking branch 'origin/main' into refactor/automatic-worker-mailbox-placement 2026-09-09 07:19:03 -07:00
Matthew Meszaros cdbdad09ee feat: stop the unibox conversation list resetting to the top when an email is opened, by keying the app shell's route boundary and Suspense on route identity rather than the raw pathname (the open thread is a path segment, so every click was remounting the whole page), remembering each list's scroll offset across remounts and the mobile display:none pane, patching mark-as-read into the query cache instead of refetching every loaded page, and auto-loading the next page at the end of the list (issue #396) 2026-09-09 06:52:57 -07:00
Matthew Meszaros a8d0e80f68 feat: address the CodeRabbit review: render EMAIL_BRAND_NAME in every transactional template body and subject instead of a hardcoded product name, name FRONTEND_BASE_URL as the source when emailed links come from it, warn on an insecure effective dashboard URL even when APP_URL was inferred rather than configured, and correct the unsubscribe guide's claim that an unset API_PUBLIC_URL disables link minting 2026-09-09 06:52:37 -07:00
Matthew Meszaros 2c1b5d8204 Merge remote-tracking branch 'origin/main' into feat/issue-393-html-css-email 2026-09-09 06:35:44 -07:00
Matthew Meszaros 426ea45013 feat: full HTML and CSS support for campaign bodies and mailbox signatures, adding a send-time CSS inliner and a structure-aware plain-text renderer in internal/pkg/mailhtml, persisting HTML mode on a step through the previously unused sequences.body_code so a designed email is no longer gutted by the editor schema on reopen, widening the TipTap schema to hold tables, div containers, colours, fonts, sizes and alignment with toolbar controls for each, folding alignment into a single style producer because TipTap merges two style attributes by splitting on the first colon and truncated background-image url(https://...) to url("https"), replacing the regex plain-text stripper that put a whole stylesheet into the text/plain part and cost a content score eight points for a .free-trial-banner class, locating </body> past Outlook conditional comments so the signature and opt-out footer stop landing inside one, forcing the signature source view for markup a contentEditable cannot host safely, and reporting per-client compatibility findings from the preview endpoint 2026-09-09 06:34:47 -07:00
Matthew Meszaros 7d58b874b8 feat: keep every recipient-facing and self-host-facing address on the deployment's own domain: mint unsubscribe links on a workspace's verified tracking domain (served by the tracking service, proxied to the backend that owns the pages), attach RFC 8058 one-click only over https, resolve all branding through config.Brand() gated on SelfHosted() so a self-host's email footer, sign-in links, stats card, API example and public form badge name nobody else, drop the app.warmbly.com fallback from AppBaseURL, blank TRACKING_DOMAIN and FORMS_DOMAIN on core-only installs, and have install.sh offer to configure a fresh interactive install instead of silently defaulting to localhost 2026-09-09 06:34:43 -07:00
Matthew Meszaros 75a068cd07 feat: always create and mount BLOB_FS_ROOT for a node because its storage layer exits when the directory is not writable and an unmounted path is root-owned by docker, and say plainly in the script and the docs that filesystem blobs cannot serve a fleet at all since the worker has to read the body the backend wrote 2026-09-09 06:06:04 -07:00
Matthew Meszaros 704fecdcc0 feat: settle on WARMBLY_NODE_REGION as the one region variable for every role, because the join script wrote it while the worker still read WORKER_REGION, so a joined worker reported no region and the placement geography term stayed dead for exactly the machines that had been given one 2026-09-09 05:04:55 -07:00
Matthew Meszaros 58b7d81d33 Merge remote-tracking branch 'origin/main' into refactor/automatic-worker-mailbox-placement 2026-09-09 04:54:04 -07:00
Matthew Meszaros 435dbb522f feat: replace the worker tier/type/risk-pool/egress categories with a scored placement model and make the fleet pull-based, so a machine joins with one command, workers and consumers share one node registry with usage and liveness, nodes self-update to the version the control plane resolves, and the Hetzner provisioning, worker profiles and SSH orchestrator are removed 2026-09-09 04:54:01 -07:00
Matthew Meszaros db2a9ba439 feat: stamp the rotating campaign send's real sending mailbox onto tasks.email_account_id before dispatch so budgets, round-robin position, deliverability rates and the contact feed name the mailbox that actually sent, and stop the contact email list and campaign-state steps reporting automated prefetches as recipient opens (issue #392) 2026-09-09 03:55:28 -07:00
Matthew Meszaros 1e1bb3622e feat: address the CodeRabbit review on the email editor PR by generating email image object keys from a uuid and the sniffed extension instead of the uploader's filename so a name carrying '..' cannot produce a key the public route refuses to serve and a private filename never travels inside a recipient's URL, deleting the stored object before its row and refusing the request when storage will not take it so a failed delete leaves nothing loadable that the quota has stopped counting, accepting only https image addresses, resynchronising the HTML source view when a template replaces the body underneath it, and rendering the campaign preview through the inbox's sandboxed frame so markup written in the source view can neither run nor restyle the dashboard for a teammate 2026-09-08 21:17:36 -07:00
Matthew Meszaros 9ce1e060dd feat: fix the campaign step body editor for issue #380 by registering TipTap's UndoRedo and HardBreak so Ctrl+Z and Shift+Enter work at all, normalising pasted HTML from Gmail/Outlook/Word so their blank-line scaffolding stops rendering a second gap on top of our paragraph margins, adding a workspace email image library (migration 000139, public email-images/ objects sharing the attachment storage quota, keyset-paginated /email-images endpoints) with toolbar upload, URL and library insertion, drag-drop, clipboard paste and a size/alignment/alt bubble, and adding an HTML source view that names the tags the schema would drop before switching back 2026-09-08 21:00:09 -07:00
Matthew Meszaros 0e7525866a feat: merge main into the select-all branch and rebuild the contact selection on top of its new keyset pagination, dropping this branch's own cursor repair now that Search carries the sort key and boundary value in a SortCursor, re-extracting the shared WHERE builder from that version, and pointing SearchIDs at the contactSorts spec and the conditional campaign-count lateral so a resolved selection orders and joins exactly the way the list does 2026-09-08 20:26:34 -07:00
Matthew Meszaros c7a44ad909 feat: complete the CRM push 400 description in the OpenAPI spec with the shared resolver's own failures, a select-all carrying no filters and a filter matching more than fifty thousand contacts, so the published contract matches every way the endpoint can refuse a selection 2026-09-08 20:02:26 -07:00
Matthew Meszaros 1432e7c459 feat: publish the exclusion-list bound that ResolveSelection enforces by giving every OpenAPI exclude field maxItems 50000 and naming too_many_contacts in the bulk update, bulk delete and CRM push 400 descriptions, and note the cap on the selection table in the contacts API reference 2026-09-08 19:50:46 -07:00