Commit Graph
74 Commits
Author SHA1 Message Date
Devin AI d69ec96868 feat: complete invited signup in the dashboard and atomically provision accounts with workspace roles and resource grants 2026-10-10 04:11:45 +00:00
Devin AI 24b0130c45 feat: reconcile mailbox sync recovery with merged worker Kafka retry and broker-confirmed delivery safeguards 2026-10-09 18:18:26 +00:00
Devin AI 1ca77a431b feat: integrate merged monitoring updates with ordered worker command recovery 2026-10-09 16:57:57 +00:00
Devin AI 4758216dbf feat: integrate current monitoring and member access changes with durable mailbox synchronization 2026-10-09 16:56:53 +00:00
Matthew Meszaros 5ab4b5e544 feat: drain Kafka worker commands through bounded ordered mailbox lanes and accelerate safe startup recovery while backing off duplicate filing retries 2026-10-09 16:35:26 +00:00
Matthew Meszaros 9a09740c84 feat: add permission-shaped operational monitoring snapshots with aggregate recovery evidence and read-only broker diagnostics 2026-10-09 15:12:54 +00:00
Devin AI 445e014b5e feat: recover mailbox synchronization across Gmail Outlook and IMAP with encrypted arrival replay and safe worker lifecycle 2026-10-09 14:24:10 +00:00
Matthew MeszarosandDevin AI ec6129c0c8 feat: preserve admin confirmed-send and deferred warmup recovery regressions when updating counter repair against main
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-10-09 11:46:19 +00:00
Matthew MeszarosandDevin AI a3cdf7b2fa feat: keep deferred warmup commands free of terminal send results and recover existing never-started preparation conflicts without releasing real delivery holds
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-10-09 11:06:49 +00:00
Matthew MeszarosandDevin AI 6c1207a48c feat: populate admin overview send counters from provider-confirmed warmup receipts and applied outbound results with UTC reporting
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-10-09 08:24:21 +00:00
Matthew MeszarosandDevin AI f06617afe3 feat: verify existing warmup state and new mailbox tasks survive release upgrades and gate CI on live recovery regressions
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-10-09 06:05:20 +00:00
Matthew Meszaros b9cc87db1a feat: enable configurable 28–32 daily mailbox sending plans with admission rechecks, clear limit controls and consistent UTC placement budgets 2026-10-09 04:18:52 +00:00
Matthew Meszaros c6c79d8dcd feat: enforce configurable per-mailbox campaign admission caps with UTC budgets, accurate diagnostics and PostgreSQL regressions 2026-10-09 04:18:52 +00:00
Matthew MeszarosandDevin AI d8d537fb9a feat: guard Kafka polling, offset storage and publication against client shutdown and preserve production diagnostics by trimming routine internal lookup access logs
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-10-07 07:52:15 +00:00
Matthew MeszarosandDevin AI 43d3e72e69 feat: run the Pages route regression in Web CI after its TypeScript dependency is installed
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-10-07 07:23:57 +00:00
Matthew MeszarosandDevin AI dfed1e1593 feat: validate API UUIDs, recover expired Gmail history and guard consumer startup and stale frontend assets
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-10-07 07:19:15 +00:00
Matthew MeszarosandDevin AI a865db3386 feat: deploy configurable Cloudflare Pages project lists from exact releases with isolated dashboard, admin, site and docs builds
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-10-05 09:48:40 +00:00
Matthew MeszarosandDevin AI 94cd8b1692 feat: reuse Cloudflare Pages production configuration for exact-release dashboard builds and cover safe runtime imports in CI
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-10-05 09:14:37 +00:00
Matthew MeszarosandDevin AI e7b6877ba4 feat: deploy the dashboard to the existing Cloudflare Pages production project only after stable releases with opt-in configuration and cutover docs
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-10-05 09:00:34 +00:00
Matthew Meszaros 38f1cdb517 Merge remote-tracking branch 'origin/main' into devin/1791174213-turnstile-trusted-device 2026-10-05 05:58:16 +00:00
Matthew MeszarosandDevin AI 55ca24e33a ci: keep a Codecov uploader crash from failing Go CI
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-10-05 05:31:39 +00:00
Matthew MeszarosandDevin AI 5301e79d84 ci: don't fail Go CI when codecov upload crashes
codecov-action v4 can throw (e.g. TLS EPROTO fetching the CLI) before
fail_ci_if_error applies, which failed the merge-queue run for #832.

Refs WAR-10

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-10-05 04:22:42 +00:00
Matthew MeszarosandDevin AI 0ea856537e fix(security): drop jwt-go via first-party Apple auth, bump yoke-derive, grant checks:write to rust audit
Refs WAR-10

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-10-04 18:36:28 +00:00
Matthew Meszaros 59f6d72d3b feat: bump docs to next 16.3.8, site to patched devalue and http-cache-semantics, and realtime to mint 1.11.0, gate the security workflow on called Go vulnerabilities with a fixed version and on retired or upgradable Hex packages through scripts/govulncheck-gate.sh and scripts/hex-audit-gate.sh with a new Elixir audit job, and keep advisory ids and reachability notes out of .trivyignore, mix.exs and the site pnpm workspace 2026-10-04 02:57:17 -07:00
Matthew Meszaros efb9d0b433 feat: add the qa/ proof harness that records scripted Playwright flows as 1080p H.264 walkthroughs and stills against an isolated per-worktree stack (lite, full, sandbox) and publishes them to the PR with gh --attach, with before/after follow-up comments, a machine-wide recording lock, idle auto-stop, a qa-ci typecheck job and AGENTS.md rules for when to record 2026-10-03 09:54:59 -07:00
Matthew Meszaros ca66b25d8d feat: register the release's bus schemas from the booting backend and hold the fleet on its current release until the backend runs the new one and the schema registry accepts them, check every published schema against a recorded snapshot in CI (make schemas) so a change the registry would refuse fails before merge, pin a FORWARD or FULL registry subject to BACKWARD on refusal, bound advisor narration so the run still lands, store unlabelled 8-bit mail with its invalid UTF-8 replaced, treat a pending or credential-less passkey ceremony as a cancellation, and drop link-scanner rejections from site error tracking 2026-09-29 19:20:07 +02:00
Matthew Meszaros 6026168334 feat: stop blocking boot on the GeoIP download and swap the database in when it lands, retry a refused mirror with backoff honouring Retry-After, revalidate a database older than a week with If-Modified-Since instead of keeping the first copy forever, and add a twice-weekly job mirroring both MaxMind editions to a private bucket so the fleet stops spending a 30-a-day allowance per boot 2026-09-20 17:55:38 +02:00
Matthew Meszaros f615c3f4d5 feat: serialize release workflow runs on one concurrency group so two tags pushed close together queue instead of racing the shared buildx cache scope and failing the second build on a missing layer blob 2026-09-20 15:57:02 +02:00
Matthew Meszaros 3923b50a2b feat: raise the tracking builder to Rust 1.96 because the AWS SDK now declares rust-version 1.94.1, and pin the Rust CI job to whatever that Dockerfile says instead of stable so a dependency outgrowing the builder fails a pull request rather than a release 2026-09-19 13:49:43 +02:00
Matthew Meszaros e668a2a36b feat: complete the ADA CASA v2.1.1 AL1 control set across authentication, sessions, access control, cryptography, input validation and configuration, adding a breached-password denylist and per-account login throttling, enforced multi-factor authentication on the admin panel, step-up confirmation before an action that mints a lasting credential, purpose-scoped session tokens, single-use TOTP steps, tenant verification on every cross-referenced identifier, security headers on every surface, encrypted webhook signing secrets, per-organization idempotency, PKCE and a minimal two-scope Gmail consent on the mailbox OAuth flow, bounded spreadsheet and archive decoding, a patched Go toolchain with govulncheck in CI, and the evidence pack under compliance/casa 2026-09-19 08:18:35 +02:00
Matthew Meszaros d96e3182f8 feat: run required CI checks for merge queue groups so validated pull requests can merge through branch protection 2026-09-16 15:15:18 +02:00
Matthew Meszaros 7300b3b021 feat: full PostHog coverage: identify the signed-in user and workspace in the dashboard and admin panel with autocapture, heatmaps, dead and rage clicks, web vitals, network timing, console capture and session replay masking only password fields, send server-side signup, trial and subscription events under the user id with the organization as a group, keep the marketing site and form pages cookieless while capturing everything stateless plus a form funnel, upload the form app's source maps, and add WARMBLY_POSTHOG_SESSION_REPLAY 2026-09-13 20:58:22 -07:00
Matthew Meszaros e8186e5f52 fix: the tracking service must not write Avro to a JSON consumer (#450)
* feat: make the Rust tracking publisher honour CODEC_PROVIDER on Kafka instead of always writing Avro, so a json consumer stops silently dropping every open and click, and refuse avro at boot when no Schema Registry is configured

* feat: build the tracking service's kafka feature in CI, because clippy on the default build never opens kafka.rs and that file now ships as the published tracking -kafka image

* feat: install libcurl and the rest of the librdkafka build dependencies for the tracking kafka clippy step, which fails at the first object without curl headers even with WITH_CURL=0
2026-09-11 23:00:15 -07:00
Matthew Meszaros 280a3e64ac feat: publish Kafka-linked image variants (#448)
* feat: publish Kafka-linked images for backend, consumer, worker and tracking as -kafka tag variants built per-arch on native runners, because the cgo librdkafka link cannot cross-compile on the build-go path

* feat: carry BUILT_AT inside the Go Kafka targets' build-args instead of the shared build-native block, so tracking and realtime stop warning about an unconsumed arg, and emit the matrix with printf because echo expands the separating backslash-n in some shells
2026-09-11 22:31:41 -07:00
Matthew Meszaros e3092f2322 feat: address the review by releasing the topic lock before the broker call so one slow creation cannot stall every publish, subscribe and close for the full admin timeout, refusing to open an admin connection once the bus is closed so a publish racing past the closed check cannot resurrect a client nothing will shut, building rather than vetting the tagged Kafka backend in CI because vet does not link and a CGO backend fails at link time, and correcting the Confluent tier wording to say auto topic creation is configurable only on Dedicated 2026-09-12 06:54:34 +02:00
Matthew Meszaros 6504d9958a feat: create Kafka topics from the bus that uses them, because a worker's command topic is named after the node id issued at join time so the set is not knowable in advance, and the broker's auto-creation is off by default on Confluent Cloud and not configurable below Standard, which left a worker subscribed to a topic that did not exist receiving nothing and reporting no error, and compile the tagged Kafka build in CI so a backend nothing else builds cannot rot unnoticed 2026-09-12 06:02:49 +02:00
Matthew Meszaros ced741e352 feat: make PostHog the default error tracker across every runtime while keeping Sentry fully supported alongside or instead of it, by turning internal/observability/errs into a two-sink fan-out with a local-log fallback, adding $exception capture to the Go services, the Rust tracking service, the Elixir realtime service and the dashboard, admin and form apps, reporting gin panics with their route, request id, workspace and user, attaching that identity plus a route and failed-request trail to browser exceptions, and wiring POSTHOG_ERROR_TRACKING, the node join env, compose, source-map upload and the docs to match 2026-09-10 19:11:32 +02:00
Matthew Meszaros 92b811773e feat: fail the build when a Dockerfile COPY names a path that is not in the repository, because build-push.yml runs only on push to main and nothing on a pull request builds an image, so a COPY of a file the same change deleted is green everywhere until it breaks main 2026-09-09 09:03:54 -07:00
Matthew Meszaros 6947aca9b3 feat: make the join-script check assert on what the script renders via a new --print-unit mode, after the previous version compared a heredoc copied into the checker and stayed green when the systemd command-substitution bug was put back, and make docker_mounts pure so the unit can be rendered without creating directories 2026-09-09 06:37:56 -07:00
Matthew Meszaros c4bfbaf4a9 feat: address the review on the image publicity gate by passing build-push only the seven services that workflow actually publishes, since web, admin and cli have no :dev tag and would have warned falsely on every push to main, by taking the release tag through the step env instead of interpolating github.ref_name into the shell, and by no longer asserting the tag is fine when a pull is refused, because GHCR denies an unknown namespace exactly as it denies a private one, so a mistyped --registry now gets its own reading in the installer message, the troubleshooting table and the fork note, which also splits the personal and organization paths to the visibility setting 2026-09-07 09:02:23 -07:00
Matthew Meszaros 719a81866c feat: gate the release on every published image being pullable with no credentials, because GHCR creates each package private and does not inherit the repository's visibility, so the authenticated imagetools inspect in create-release passed for four releases while ghcr.io/warmbly/warmbly/* returned unauthorized to everyone outside the org and no curl | sh self-host install could pull a byte (#371); adds scripts/check-images-public.sh as the anonymous pull test, makes it the gate before create-release and the source of the digests in images.json, warns from build-push on main where a new service image first appears, and teaches install.sh to report a registry refusal as one instead of as a missing tag 2026-09-07 08:51:57 -07:00
Matthew Meszaros 51dedc90ee feat: put every runtime behind one optional error-reporting story: a single internal/observability/errs wrapper that is now the only package importing sentry-go, InitSentry for cmd/forms, release and environment tags on every service from the existing build stamp, optional Sentry in the admin panel and the public forms app, the sentry crate in the Rust tracking service, release tagging in realtime, CI source-map upload that only runs when a Sentry token is configured, and docs covering the DSN for each service 2026-09-07 03:51:06 -07:00
Matthew Meszaros b35e111705 fix: address the review on the CLI PR: give the sign-in handshake its own per-IP budget so a 200-poll login cannot lock the address out of the browser login, keep https for a remote host that names a port instead of sending a bearer token in cleartext, report truncation when a paginated walk stops at max-pages, accept a piped secret with no trailing newline, normalise WEBSOCKET_URL on its suffix so a bare /socket becomes a real endpoint, destroy a minted secret the moment its code expires, gate cli-installer-ci on the required status check, print a rejected flag instead of an unbound-variable error, and use a portable sha256 so the packaging runs on macOS 2026-09-04 21:03:58 -07:00
Matthew Meszaros 4e94d2b47b feat: install the CLI in one command from warmbly.com/cli.sh or cli.ps1, with checksum-verified archives, a Homebrew formula and Scoop manifest generated alongside them, a ghcr cli image, and a make cli-check that installs from a local mirror and proves a tampered checksum stops the install 2026-09-04 20:14:38 -07:00
Matthew Meszaros 77058a2cb4 feat: address the review on the installer branch by keeping the database password out of pg_dump's argv, excluding backup bundles from the blob root they are written into, tolerating blobs that change or vanish mid-archive, making the instance-settings bootstrap a single atomic insert, and validating the release tag before it is written into .env 2026-09-04 06:24:05 -07:00
Matthew Meszaros d68bbcd2ab feat: add a one-command self-host installer at warmbly.com/install.sh with an interactive data-control wizard, give docker-compose.yml image keys and per-store volume variables, add an image-mode updater, move engagement/form/audit retention into instance settings, and add warmblyctl backup/restore 2026-09-04 05:49:54 -07:00
Matthew Meszaros b2ea1f1961 feat: add self-hosted update awareness and one-click updates: every binary is stamped with its version and commit, the backend polls GitHub Releases and a new host-side updater (cmd/updater, compose profile or systemd unit) reports the checkout's commit distance, the admin panel's top bar shows a version pill that turns into an update indicator and opens a dialog with confirmation, live step progress and log, restart tracking and result, the dashboard header shows the same pill to every member of a self-hosted instance with the full update flow for platform admins, Setup and health gains update_available and updater_unreachable checks, warmblyctl status prints the version, make upgrade and scripts/upgrade-bare-metal.sh cover the by-hand paths, and docs gain an Updates page plus configuration, health, deployment and API reference updates 2026-09-03 05:04:30 -07:00
Matthew Meszaros 53b932292b fix: address review findings on forms v2: the list aggregates query passed a Go int into a text concatenation so pgx could not encode it and starts, identified and the trend sparkline were always empty behind a silent degrade; the forms tables sat above campaigns in the org transfer order despite carrying a campaign_id foreign key, which would fail a full import; an imported forms domain inherited the destination workspace's stale verified flag; a form whose creator was offboarded stopped creating contacts because created_by is ON DELETE SET NULL, now falling back to the workspace owner; the forms mirror check never ran when only the web-side copy changed; and the bare metal guide asked for five DNS names while certbot requested a sixth 2026-09-01 09:49:06 -07:00
Matthew Meszaros 60c9e316d9 feat: hosted lead-capture forms end to end: drag-and-drop builder with field settings, design panel, embed/share and submissions tabs in the dashboard, a public TanStack form app (forms/) served by the new standalone forms service (cmd/forms + internal/formserver) on FORMS_DOMAIN with per-form frame-ancestors CSP, honeypot/fill-time/Turnstile/per-IP submit protection and a same-origin JSON API proxying the backend internal API, form submissions creating contacts with categories and campaign enrollment plus realtime, audit, webhook and org-transfer coverage, migration 000114, seed forms, CI jobs, Dockerfile, systemd/nginx/compose manifests and docs 2026-09-01 01:17:51 -07:00
Matthew Meszaros 42ddb78e60 feat: make the realtime service survive a client that spams channel joins, and stop both clients from being that client: ws_join is now spent per phx_join in a shared RealtimeWeb.ChannelGuard that every channel runs BEFORE its Auth lookup (it was only ever spent on the socket handshake, so an established socket could issue unlimited joins and each one cost a Postgres query), handshakes move to their own ws_connect bucket so a reconnect storm cannot eat the budget a client needs to rejoin with, the retry_after_ms hint now points at the next window instead of shrinking as overage grew, every channel's join rejection carries the numeric code the API reference already promised (plus 4005 for a malformed topic), BulkChannel only pushes an event to the user who owns the operation instead of to anyone who guesses the id, and the web and admin clients treat a rate_limited join as transient (wait out the hint, keep the socket) while backing off per-channel rejoins on a decaying schedule rather than reconnecting on the 120ms floor forever; adds an ExUnit suite for realtime and a vitest suite for admin, both wired into CI 2026-08-26 21:14:34 -07:00