Commit Graph
127 Commits
Author SHA1 Message Date
Matthew Meszaros 9c5a6c3387 Merge pull request #1125 from warmbly/chore/review-kafka-costs
feat: add private AWS NATS operations and a compatible Kafka migration
2026-11-03 06:03:01 +00:00
Matthew Meszaros 0dfd746dc7 feat: verify all stored credential domains from a bounded consistent database snapshot 2026-11-03 06:50:19 +01:00
Matthew Meszaros 0d80e9e1d0 feat: identify each NATS notification policy statement for SNS validation 2026-11-03 06:48:45 +01:00
Matthew Meszaros a735bc50fe feat: tighten private NATS operations with durable quorum checks and scoped application permissions 2026-11-03 06:47:47 +01:00
Matthew Meszaros 19c1040549 feat: validate stored mailbox credentials with their organization keys in the AWS deployment gate 2026-11-03 06:36:50 +01:00
Matthew Meszaros b63a853723 feat: automate private AWS NATS backups and serial security patching with quorum checks and operational alarms 2026-11-02 19:17:00 +01:00
Matthew Meszaros 999597508c feat: provision private three-zone NATS JetStream and add gated Kafka migration readers with compatible codecs 2026-11-02 15:25:15 +01:00
Matthew Meszaros 9b0b319ee7 feat: enforce issued password format and add scoped private attachment downloads with restricted runtime TLS ciphers 2026-11-02 13:05:45 +01:00
Matthew Meszaros 6cde875da1 feat: gate the hosted TLS recovery evidence writer with scoped conditional uploads and infrastructure approval 2026-11-01 14:05:44 +01:00
Matthew Meszaros 45e4ba23e6 feat: stage runtime TLS routing and draining with live compatibility and cleanup gates 2026-10-31 19:25:44 +01:00
Matthew Meszaros c499230823 feat: add opt-in SERVE_TLS listeners to backend, forms, tracking and realtime, pin forms and tracking to the backend certificate with BACKEND_INTERNAL_CA, and stage the AWS runtime onto HTTPS target groups and a TLS internal hop through target_tls (off, serve, both, on) with mocked terraform tests 2026-10-31 11:28:28 +01:00
Matthew Meszaros 9e6fd3643e feat: include the required target tag in the worker fleet handover command 2026-10-28 14:19:22 +01:00
Matthew Meszaros 23cc5f9c6a feat: start staged fleet rollouts from release deployments and verify worker updater handover 2026-10-28 14:11:09 +01:00
Matthew Meszaros 21acf00257 feat: resume a key rotation stage that died part way from its saved intent with per-phase state tokens, deploy all to only the ECS services an install runs, fail updater --check-env while another update holds the lock, keep the running image when adopting a node, judge rollout liveness on the service clock, deny the CI roles changes to their own permissions, and validate the Terraform plan class before acting on it 2026-10-24 09:34:01 +02:00
Matthew Meszaros 9c2cffae58 feat: deploy stable releases to AWS automatically with provenance checks, a migration gate, alarm-watched waves and reverse rollback, move worker and consumer hosts in canary rollouts with host-side health rollback and opt-in env refresh, rotate instance keys inside AWS through a rotator task and the rotate-keys workflow, plan and apply the AWS Terraform from CI with approval for replacements, and add the admin Keys tab and the matching runbooks 2026-10-24 09:10:52 +02:00
Matthew Meszaros 5863b206a2 feat: verify AWS credential read keys, MFA purpose keys and the complete organization key inventory 2026-10-22 12:57:44 +02:00
Matthew Meszaros fdea749005 feat: preserve encrypted credentials and MFA enrollments across staged key rotation with scoped token grace periods 2026-10-22 12:37:19 +02:00
Matthew Meszaros 8e3e827b02 feat: validate that GitHub-enabled AWS runtimes define both deployable application services 2026-10-22 12:21:26 +02:00
Matthew Meszaros f31334bd7a feat: allow the AWS deployment role to pass the backend and consumer task roles to ECS 2026-10-22 12:14:15 +02:00
Matthew Meszaros 05499dba2b feat: derive AWS service and database check tasks from shared Terraform container inputs 2026-10-22 11:16:49 +02:00
Matthew Meszaros ee8581136e feat: require direct database sessions and verify authenticated worker dependencies during AWS deployment 2026-10-22 11:01:04 +02:00
Matthew Meszaros bfad481ade feat: preserve active ECS task revisions throughout the AWS deployment rollback window 2026-10-22 09:12:11 +02:00
Matthew Meszaros 99357a6823 feat: add Terraform AWS hosting with optional Postgres pooling and migration-gated deployments 2026-10-22 08:57:54 +02:00
Matthew MeszarosandDevin AI 9af270a533 feat: add production image-only Coolify Compose deployment with domain, secret, backup and setup instructions
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-10-16 11:47:52 +00:00
Matthew Meszaros 8c06efc6e4 feat: ship the iOS app to TestFlight from GitHub Actions on merges touching ios/ (ios-release workflow with cloud-managed signing, shared Warmbly scheme, ExportOptions, iPhone-only 1.0.0, privacy manifest), add in-app account deletion and a read-only billing page, let admins exempt an existing account from the emailed login code for App Review, read Apple .p8 keys as PEM or base64 through one applekey parser for Sign in with Apple and APNs, and document the release flow and Apple web sign-in setup 2026-10-12 10:38:14 +02:00
Matthew MeszarosandDevin AI caa1633dac feat: verify main and TAC assessment dashboard origins share configurable backend CORS and OAuth routing
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-10-05 12:24:11 +00:00
Matthew Meszaros f23c04ddd8 Merge remote-tracking branch 'origin/main' into feature/slack-app-agent
# Conflicts:
#	deploy/slack/manifest.json
#	docs/content/docs/development/slack-app.mdx
#	docs/content/docs/guides/slack.mdx
#	internal/app/integration/slack.go
#	internal/app/slackapp/agent.go
#	internal/app/slackapp/client.go
#	internal/app/slackapp/slackapp_test.go
#	web/src/lib/integrations/oauthPopup.ts
2026-10-04 06:54:19 -07:00
Matthew Meszaros 801c87ffe1 feat: make the Warmbly Slack bot a mention-driven agent that DMs the account link, answers the held question once linked, auto-links the installer, reacts while working, lets teammates take over a thread, sets up Slack and links integrations through permission-gated assistant tools with strict approvals in channels, drops the /warmbly slash command, and replaces the integration connect drawer and label field with a connect popup 2026-10-04 05:15:07 -07:00
Matthew Meszaros e852a106a9 feat: serve every node-only internal route (data keys, message map, sync lookups, worker config, fleet heartbeat) on NODE_BROKER_TOKEN and leave only tracked links, domain redirects, page hits and forms on INTERNAL_API_TOKEN, render nodes only the node token when one is set, and have the installer generate a distinct NODE_BROKER_TOKEN for new installs and its own UPDATER_TOKEN 2026-10-04 03:21:58 -07:00
Matthew Meszaros 8bb60e9449 feat: accept only Salesforce domains as an org's API host and call it through the SSRF-guarded client, register the Warmbly Cloud link only on self-hosted instances behind the instance admin with a second factor and dial it through safehttp with fixed error text, keep automation signing secrets in the sealed connection config, answer integration service failures with fixed messages, add security headers to the forms, tracking and docs origins and TLS 1.2+ to the nginx template, compare the captcha bypass in constant time, require TLS 1.2 for IMAP probes, and drop the unused RSA helpers 2026-10-04 02:58:43 -07:00
Matthew Meszaros e1c4a91ad1 feat: confirm a Slack link only for the Warmbly member whose email matches the Slack account's confirmed profile email (read via users.info with the new users:read and users:read.email scopes, refused as slack_link_email_mismatch), show the Slack account's name and avatar on the link page and after linking, return the Slack connection in GET /integrations/slack/status only to manage_settings or use_integrations, scope agent-thread lookups by organization, and check a Draft a reply card's conversation belongs to the clicker's workspace before starting the assistant 2026-10-04 02:45:04 -07:00
Matthew Meszaros 61d16dfe9a feat: add Warmbly for Slack with the AI assistant in DMs, mentions and the assistant pane, unified inbox threads with reply, AI draft and lead actions, per-category notification routing and DMs, account linking, dashboard Slack settings, manifest and docs 2026-10-03 15:35:11 +02:00
Matthew Meszaros 41d43f64be feat: check that a verified root redirect actually reaches visitors by opening the domain over http and https and naming what answered instead (Traefik, nginx, Caddy, a rewritten Host header, a missing certificate, a closed port) with per-proxy fix steps in the sending domain drawer, and let a self-hosted instance linked to Warmbly Cloud have Cloud serve and certify its redirects (connect in place from the redirect tab, the bulk dialog or a page banner), with Cloud-side linked-instance redirect endpoints under a per-instance limit, migration 000237, labelled tracking answers and a 503 when the redirect lookup is unavailable, and the sending domains, Warmbly Cloud, data control, install, OpenAPI, API and error code docs updated 2026-09-30 06:04:00 -07:00
Matthew Meszaros f4f219b7c5 feat: run contact file imports as background jobs (upload once, a whole-file check of new, existing, repeated and invalid rows, a chunked leased runner with live CONTACT_IMPORT_PROGRESS, history, cancel, a draft that autosaves and survives a reload, remembered mappings, and a failed-rows CSV under the file's own headers), match existing contacts across the workspace, batch updates and fail a bad row alone, keep imported verdicts on update, scope every import write to the importing workspace, rebuild the import wizard with icons and inline segment creation, show company logos and the inbox provider on the contact avatar, and hide contact columns the list has no data for 2026-09-27 21:50:10 -07:00
Matthew Meszaros e58921484d feat: rebuild mailbox import around column mapping and automatic host and sign-in detection (CSV, XLSX, pasted lists, saved mappings, retryable rows with fixes, migrations 000205-000206), connect whole Google Workspace domains and Microsoft 365 organizations through a proved administrator grant, import from inbox vendors (InboxKit, Zapmail, Mailforge, Infraforge, Maildoso, Cheap Inboxes, ScaledMail) with vendor-managed forwarding and DNS, add a sending domains page with per-domain tracking and verified root redirects, unify Add account into one Google and one Microsoft entry with per-method choices, mark per-mailbox Google sign-in as retiring with in-place moves to the admin grant or an app password, allow the loopback security mode in the credential columns (migration 000207), read semicolon-separated CSVs, and add a mock vendor API to the sandbox 2026-09-23 08:41:01 -07:00
Matthew Meszaros e668a2a36b feat: complete the ADA CASA v2.1.1 AL1 control set across authentication, sessions, access control, cryptography, input validation and configuration, adding a breached-password denylist and per-account login throttling, enforced multi-factor authentication on the admin panel, step-up confirmation before an action that mints a lasting credential, purpose-scoped session tokens, single-use TOTP steps, tenant verification on every cross-referenced identifier, security headers on every surface, encrypted webhook signing secrets, per-organization idempotency, PKCE and a minimal two-scope Gmail consent on the mailbox OAuth flow, bounded spreadsheet and archive decoding, a patched Go toolchain with govulncheck in CI, and the evidence pack under compliance/casa 2026-09-19 08:18:35 +02:00
Matthew Meszaros e36a070595 Merge remote-tracking branch 'origin/main' into fix/worker-capacity-account-distribution 2026-09-17 08:02:45 -07:00
Matthew Meszaros 8a60c6bab3 feat: build realtime on OTP 27 and update vulnerable Elixir dependencies 2026-09-17 07:54:34 -07:00
Matthew Meszaros c1e45b194a feat: merge latest main before worker capacity queueing 2026-09-17 06:26:07 -07:00
Matthew Meszaros b733d09f89 feat: make inbox follow-up labels safe for manual labels and automated mail 2026-09-17 04:57:52 -07:00
SUMAN JANA 3b8761d361 feat(inbox): explain every tag on hover, label live mail, and keep follow-ups working with no external service 2026-09-17 04:57:52 -07:00
SUMAN JANA 260898bf20 feat(inbox): automatic tagging and relevance scoring for inbound mail, optional and off by default 2026-09-17 04:57:24 -07:00
Matthew Meszaros bab9f86727 feat: correct worker capacity, mailbox distribution, observed IPv4, fleet pagination, and premium pool promotion 2026-09-17 04:15:05 -07:00
Matthew Meszaros 4784ee7d39 feat: fetch the MaxMind databases instead of requiring a mounted file (#529)
* feat: let the backend, consumer and tracking service fetch their own MaxMind databases from GEODB_URL and TRACKING_SCANNER_ASN_DB_URL, reading the archive shape from the content so a permalink tar.gz, a gzipped mmdb and a bare mmdb all work, never replacing a file already at the path, opening the bytes before installing them so a licence-key error page cannot become the database forever, skipping the AppleDouble sidecars a macOS tar writes ahead of the real file, and treating both URLs as secrets because the permalink carries the licence key

* feat: drop the trailing blank line cargo fmt --check rejects at the end of tracking/src/asndb.rs

* feat: stream the downloaded ASN archive instead of decompressing it whole, sizing each buffer from the gzip footer and the tar header so the member is allocated exactly once, which drops the peak of unwrapping a permalink tar.gz from 38 MB to 11.9 MB, essentially the database itself

* feat: stop the MaxMind licence key reaching the logs through net/http's and reqwest's own error text, which both print the URL they were given and so defeated the redaction beside them, drop userinfo as well as the query when redacting, refuse plain http for a URL carrying a credential and refuse an https-to-http redirect, and apply the size cap to the decoded database rather than the compressed transfer so a gzip bomb cannot fill the disk

* feat: strip basic-auth userinfo as well as the query when the tracking service redacts its database URL, parsing it rather than cutting at the first question mark so where a credential sits is the URL library's problem and not a guess
2026-09-15 03:06:34 -07:00
Matthew Meszaros 0a1ed6f04e feat: resolve scanner ASNs from a GeoLite2-ASN database so the catalogue's asn: entries match without a Cloudflare transform rule, ship the Proofpoint, Mimecast and Cisco ASNs enabled behind a new probable certainty that widens the consumer's machine window instead of deciding the verdict, make the tracking event dedupe claim one coalesced operation, and report an ASN database that opened cleanly but resolves nothing (#440) 2026-09-15 01:48:45 -07:00
Matthew Meszaros 7300b3b021 feat: full PostHog coverage: identify the signed-in user and workspace in the dashboard and admin panel with autocapture, heatmaps, dead and rage clicks, web vitals, network timing, console capture and session replay masking only password fields, send server-side signup, trial and subscription events under the user id with the organization as a group, keep the marketing site and form pages cookieless while capturing everything stateless plus a form funnel, upload the form app's source maps, and add WARMBLY_POSTHOG_SESSION_REPLAY 2026-09-13 20:58:22 -07:00
Matthew Meszaros d456bc48c6 feat: fix the Warmbly Cloud pool link across both roles (#262): take an enrolled mailbox out of this instance's own warmup pool so local partners stop writing to it and their unverifiable warmup stops landing in the owner's unibox, recognise the cloud's warmup mail whose verify header did not survive delivery through a new warmup-deliveries lookup that ignores consumed_at because instance and cloud read the same mailbox, move the managed-mailbox access token route behind NODE_BROKER_TOKEN so the internet-facing tracking and forms services can no longer mint a live provider token, scope pause and resume to the caller's workspace, keep an enrolled mailbox listed once it goes inactive, release the cloud copy when the local mirror row cannot be written, refuse the one-time handshake when CREDENTIALS_ENCRYPTION_KEY is missing, blank an expired code's plaintext instance token, and stop errx answering 200 for a status outside its table 2026-09-12 06:58:25 -07:00
Matthew Meszaros a4f503f620 fix: realtime cannot complete either of its TLS connections (#452)
* fix: let the realtime service actually establish its TLS connections, adding the wildcard hostname match fun Redix needs for every managed Redis and a CA bundle option for an RDS Postgres that the system trust store cannot verify

* feat: drop the redundant wget package from the realtime image since busybox already provides it, and fix a comment that read as literal interpolation

* fix: treat a blank DATABASE_SSL_CA_FILE as unset in the realtime config, because compose passes every optional variable through as empty and cacertfile with an empty path is no CA source at all rather than a fallback to the system store
2026-09-12 00:29:34 -07:00
Matthew Meszaros 280a3e64ac feat: publish Kafka-linked image variants (#448)
* feat: publish Kafka-linked images for backend, consumer, worker and tracking as -kafka tag variants built per-arch on native runners, because the cgo librdkafka link cannot cross-compile on the build-go path

* feat: carry BUILT_AT inside the Go Kafka targets' build-args instead of the shared build-native block, so tracking and realtime stop warning about an unconsumed arg, and emit the matrix with printf because echo expands the separating backslash-n in some shells
2026-09-11 22:31:41 -07:00
Matthew Meszaros a962683511 feat: ship AWS's RDS truststore in the backend and consumer images and point the docs at it, because Amazon RDS chains to a root that is in no public trust store so the sslmode=verify-full those docs recommended failed every connection with x509 certificate signed by unknown authority, while deliberately not setting PGSSLROOTCERT by default since an RDS-only store would break a Postgres fronted by a public CA 2026-09-11 05:23:23 +02:00