169 Commits
Author SHA1 Message Date
Matthew Meszaros 6e71801feb feat: show fleet autoscaling status failures with a retry in the admin Capacity tab, derive the fleet cap ceiling from fleet_max_size when unset, warn through a fleet_cap check when fleet_max_size no longer matches the group's cap, and bound the cap and ceiling to int32 2026-11-13 16:10:34 +01:00
Matthew Meszaros 5c56833ff9 feat: let the admin panel's Fleet capacity tab show the worker Auto Scaling Group's size against its cap and change the cap between the running fleet and a Terraform-set warmbly:max-ceiling tag, warning near and at the cap, through FLEET_AUTOSCALING_GROUP and GET/PATCH /admin/fleet/scale 2026-11-13 15:56:06 +01:00
Matthew Meszaros cfc9ec24ab Merge pull request #1253 from warmbly/feat/terraform-stack-foundation-vars
feat: let the CI Terraform runner plan and apply the foundation stack
2026-11-13 05:46:27 +00:00
Matthew Meszaros 063228aa38 feat: declare state_bucket in the foundation Terraform root so the CI stack runner can plan and apply it like runtime and workers, and fail the stack runner test when a stack it runs does not declare it 2026-11-12 19:29:59 +01:00
Matthew Meszaros ea38686862 feat: say in the EC2 worker docs that unlimited credits bill only CPU time beyond earned credits 2026-11-12 19:10:40 +01:00
Matthew Meszaros b4353e7987 feat: run the fixed EC2 worker hosts on unlimited CPU credits so a busy worker bills its surplus instead of being throttled to the 20% baseline, and document the cost signal to watch 2026-11-12 19:01:57 +01:00
Matthew Meszaros d42c664dd1 feat: answer a workspace archive check in seconds by matching existing rows on their primary key index and bounding the advisory conflict counts, always send empty preflight lists as arrays, show a failed check next to its button, and exempt the archive import routes from the 10 MB request cap in the API and the nginx example 2026-11-11 20:42:35 +01:00
Matthew Meszaros 1048431673 feat: document moving a portal zone from one route for every hostname to a route for each, keeping the old route for a day 2026-11-10 14:10:34 +01:00
Matthew Meszaros 9128106295 feat: give each agency portal domain its own Worker route when CLOUDFLARE_SAAS_PORTAL_WORKER names the portal Worker, added with the domain's custom hostname and removed with it, so a Cloudflare for SaaS zone shared with tracking and redirect domains sends only portal domains to the portal 2026-11-10 13:35:47 +01:00
Matthew Meszaros a79a196912 feat: add Warmbly for Agencies with a separate agency view, agency accounts, a shared monthly send pool, a shared agency AI balance with per-client caps and transfers, agency billing and top-ups, agency API keys across clients, client mailbox connect links and shareable and scheduled client reports served by a separate portal service on portal domains, white-label branding with a branded client dashboard on agency portal domains and website embeds, copy campaigns and templates across workspaces, and a cross-client inbox 2026-11-10 08:31:31 +01:00
Matthew Meszaros 2385a1f3dd feat: hand reader services only public web-port pages and stop them redirecting off their own host, read MCP search tools' structuredContent and isError whatever the content shape, refuse a search when the server's credentials cannot be opened, honour a tool's minimum result count, keep nested results, multiline text and duplicate links' snippets, offer only search-capable tools in the web search picker and say it runs without approval, forward the per-provider search URLs in docker-compose.yml, and scope the robots.txt note to the lead finder 2026-11-10 05:01:14 +01:00
Matthew Meszaros f4c7e73176 feat: read lead finder sites that answer 401/403/406/429/503 through configurable reader services (WEB_READER: Crawl4AI, Jina Reader, DDGS, custom) and a workspace Jina AI integration, search the web for missing websites and blocked sites' addresses through a picked MCP search tool, the workspace Jina key or SEARCH_PROVIDER (now also ddgs, jina, brave and tavily, comma-separated to fall back), add web_search to enrichment runs and search_tool to MCP servers in migration 000373, show unread sites as no info available with their reason, and document it all 2026-11-10 05:01:14 +01:00
Matthew Meszaros 26dac7af94 feat: run the bundled PostgreSQL in install.sh, docker-compose.yml and the Coolify compose with max_locks_per_transaction=256 so the single-transaction warmblyctl restore has lock-table headroom, and say so in data-control.mdx 2026-11-09 17:11:03 +01:00
Matthew Meszaros 81982ac3d0 feat: make warmblyctl backup and restore pick the pg_dump and psql matching the server's PostgreSQL major from the 16, 17 and 18 clients the backend image now ships, skip preamble settings the target server lacks such as transaction_timeout when replaying a dump, run the schema drop and load in one transaction so a failed restore changes nothing, and add a live backup and restore round trip test to CI 2026-11-09 17:05:23 +01:00
Matthew Meszaros c319580aea feat: drop create_before_destroy from the consumer quiet alarm, which Terraform would propagate to the consumer service and its fixed-name target groups 2026-11-08 12:56:28 +01:00
Matthew Meszaros 576c0b99bd feat: keep the consumer backlog alarm under warmbly-production-consumer- so a release that stops consuming fails its deploy bake, leave only the quiet alarm outside it, and create a renamed quiet alarm before deleting the old one 2026-11-08 12:47:16 +01:00
Matthew Meszaros 73ef60dd00 feat: name the consumer scaling alarms warmbly-production-scale-consumer-backlog and -quiet so the deploy bake, which watches every alarm under the service's own name prefix, never reads an idle consumer's quiet alarm as a failed release 2026-11-08 06:38:53 +01:00
Matthew Meszaros c0cf2555e7 feat: default BOX_GOOGLE_OAUTH_CONNECT to empty in the Coolify compose file so configured Google credentials enable mailbox sign-in, and document the Warmbly Cloud brokered sign-in and shared-parent passkey cases 2026-11-07 11:21:50 +01:00
Matthew Meszaros bc85154d2d feat: show Google mailbox sign-in in the dashboard whenever the backend reports gmail_oauth_connect, removing the WARMBLY_GMAIL_OAUTH_CONNECT and VITE_GMAIL_OAUTH_CONNECT dashboard flag and the assessment-dashboard setup from config, compose files, docs and tests 2026-11-07 11:10:37 +01:00
Matthew Meszaros d448cdb20d feat: add map-based lead finder under /app/leads with pluggable OpenStreetMap, Google Places, Foursquare and Outscraper sources, website reads that find emails, socials and the people who run each business with pattern-built verified addresses, filters, saved searches, per-business or per-person CSV/XLSX/JSON export and add-to-contacts 2026-11-07 09:21:41 +01:00
Matthew Meszaros 527b982111 feat: register every verified tracking host and redirect domain (root and www) as a Cloudflare for SaaS custom hostname when CLOUDFLARE_SAAS_* is set, show a root CNAME and its activation TXT behind that edge, point the www record at the root, and let the AWS tracking listener forward unmatched hosts to tracking behind tracking_serves_unmatched_hosts 2026-11-07 08:17:45 +01:00
Matthew Meszaros 617e9b0ee2 feat: state in the runtime README that services left out of autoscaling keep their count, and in the Fargate guide that a consumer image without the backlog publisher leaves its scaling inactive 2026-11-06 13:21:02 +01:00
Matthew Meszaros b6c31852c2 feat: scale the AWS consumer service between one and six tasks on a WorkerEventBacklog CloudWatch metric the consumer publishes when CONSUMER_METRICS_CLOUDWATCH is set, and let backend, tracking and forms opt into CPU target tracking through the runtime root's autoscaling map 2026-11-06 13:11:13 +01:00
Matthew Meszaros a64a9e3bbf feat: retain negotiated origin cipher outcomes through bounded process deadlines and verify open-peer behavior 2026-11-04 21:24:41 +01:00
Matthew Meszaros fdd9a1fb33 feat: record bounded peer alert and ClientHello evidence in the origin TLS diagnostic while omitting session secrets 2026-11-04 20:23:48 +01:00
Matthew Meszaros 71fb768eca feat: isolate AWS origin TLS probe and cleanup credentials with phase policies and failure-gated finalization 2026-11-04 20:02:09 +01:00
Matthew Meszaros 0dca1f84f2 feat: verify exact origin TLS diagnostic ownership during running and stopped task recovery and private log collection 2026-11-04 19:22:44 +01:00
Matthew Meszaros d77383adce feat: add a secretless Fargate origin TLS diagnostic with pinned interfaces and private versioned evidence 2026-11-04 19:07:57 +01:00
Matthew Meszaros 343f8a2ec5 Merge pull request #1149 from warmbly/feat/origin-alb-policy-config
feat: add exact origin ALB policy plans and scoped configuration through ordinary AWS infrastructure
2026-11-04 17:55:01 +00:00
Matthew Meszaros ed9c340f79 feat: add main-only origin ALB policy planning, exact scoped apply and rollback through ordinary AWS infrastructure workflow 2026-11-04 18:25:39 +01:00
Matthew Meszaros 6642f339ea feat: merge main after the Postgres load work landed, number the mailbox push channels migration 000351 after it, and share one arrival batch size between the claim and the delivery loop 2026-11-04 18:03:46 +01:00
Matthew Meszaros 710d857a4b feat: keep an unsent mailbox push report ahead of the control plane's stored row and resend it during the retry backoff, and plan-test the largest accepted fleet worker capacity 2026-11-04 16:58:54 +01:00
Matthew Meszaros daaed572cc feat: power off a fleet worker or NAT instance whose boot script fails so the Auto Scaling group replaces it, stop an in-flight node update before the lifecycle watcher stops a terminating worker, cap fleet_worker_capacity at the 99999999 the worker accepts with a plan test, bound each CloudWatch fleet metrics publish to one interval, and document FLEET_METRICS_CLOUDWATCH as a fleet enable step and join token renewal every three weeks 2026-11-04 16:32:27 +01:00
Matthew Meszaros 9545a26e29 feat: verify hosted backend preflight task identities with existing deployment permissions and preserve rollback counts 2026-11-04 14:26:27 +01:00
Matthew Meszaros 1cb2bfa3b3 feat: add verified hosted backend attachment configuration with retained release settings and guarded rollback 2026-11-04 13:56:27 +01:00
Matthew Meszaros f9b8fa2255 feat: scale the AWS worker fleet on worker memory and CPU instead of mailbox count, publish WorkerMemoryPercent and WorkerMemoryMaxPercent from heartbeats, and let WARMBLY_WORKER_CAPACITY=auto size each worker's mailbox target from its measured memory 2026-11-03 18:11:18 +01:00
Matthew Meszaros d241a836f8 feat: sync mailboxes in realtime through Graph change notifications, Gmail Pub/Sub watches and IMAP IDLE that wake the owning worker with SYNC_WAKE, drain stored arrivals continuously, and add an opt-in autoscaling AWS worker fleet without public IPv4 scaled on a published fleet utilization metric 2026-11-03 15:11:24 +01:00
Matthew Meszaros 7cec9bf137 Merge pull request #1137 from warmbly/feat/static-edge-tls-routing
feat: serve hosted static frontends through explicit HTTPS Pages origin mappings
2026-11-03 13:33:42 +00:00
Matthew Meszaros fb7546e141 Merge pull request #1131 from warmbly/feat/bounce-cause-keeps-valid-addresses
feat: decide bounce suppression from the DSN status code and server reply through a shared dsn.Blame rule so mailbox-full, policy, reputation and system bounces keep the address and lead while address-invalid ones suppress, carry the per-recipient status and diagnostic in inbound bounce reasons, reuse the rule in emailverify, and add an optional TYPESAFE_ENDPOINT
2026-11-03 13:15:28 +00:00
Matthew Meszaros d3afcc9fa4 feat: serve hosted static frontends through explicit HTTPS Pages origin mappings with credential-free forwarding and staged custom-domain verification 2026-11-03 14:03:10 +01:00
Matthew Meszaros d1ae1e2e32 feat: decide bounce suppression from the DSN status code and server reply through a shared dsn.Blame rule so mailbox-full, policy, reputation and system bounces keep the address and lead while address-invalid ones suppress, carry the per-recipient status and diagnostic in inbound bounce reasons, reuse the rule in emailverify, and add an optional TYPESAFE_ENDPOINT 2026-11-03 08:31:36 +01:00
Matthew Meszaros 31f6db7083 feat: monitor the NATS logical stream limit and alert before queued storage reaches capacity 2026-11-03 07:14:16 +01:00
Matthew Meszaros 9c5a6c3387 Merge pull request #1125 from warmbly/chore/review-kafka-costs
feat: add private AWS NATS operations and a compatible Kafka migration
2026-11-03 06:03:01 +00:00
Matthew Meszaros 0dfd746dc7 feat: verify all stored credential domains from a bounded consistent database snapshot 2026-11-03 06:50:19 +01:00
Matthew Meszaros 0d80e9e1d0 feat: identify each NATS notification policy statement for SNS validation 2026-11-03 06:48:45 +01:00
Matthew Meszaros a735bc50fe feat: tighten private NATS operations with durable quorum checks and scoped application permissions 2026-11-03 06:47:47 +01:00
Matthew Meszaros 19c1040549 feat: validate stored mailbox credentials with their organization keys in the AWS deployment gate 2026-11-03 06:36:50 +01:00
Matthew Meszaros b63a853723 feat: automate private AWS NATS backups and serial security patching with quorum checks and operational alarms 2026-11-02 19:17:00 +01:00
Matthew Meszaros 999597508c feat: provision private three-zone NATS JetStream and add gated Kafka migration readers with compatible codecs 2026-11-02 15:25:15 +01:00
Matthew Meszaros 9b0b319ee7 feat: enforce issued password format and add scoped private attachment downloads with restricted runtime TLS ciphers 2026-11-02 13:05:45 +01:00