Commit Graph
142 Commits
Author SHA1 Message Date
Matthew Meszaros f4c7e73176 feat: read lead finder sites that answer 401/403/406/429/503 through configurable reader services (WEB_READER: Crawl4AI, Jina Reader, DDGS, custom) and a workspace Jina AI integration, search the web for missing websites and blocked sites' addresses through a picked MCP search tool, the workspace Jina key or SEARCH_PROVIDER (now also ddgs, jina, brave and tavily, comma-separated to fall back), add web_search to enrichment runs and search_tool to MCP servers in migration 000373, show unread sites as no info available with their reason, and document it all 2026-11-10 05:01:14 +01:00
Matthew Meszaros ebf918aa74 Merge remote-tracking branch 'origin/main' into fix/hide-warmup-inbox 2026-11-09 20:46:53 +01:00
Matthew Meszaros e3b5253cbe feat: hide other services' warmup from the inbox with a Settings > Sending switch for unverified Warmbly warmup markers plus warmup tags (applied on arrival and swept from stored mail), add an opt-in per-mailbox warmup filter tag appended to warmup a mailbox receives (migration 000373, GET/PUT/DELETE /emails/:id/warmup-filter-tag, drawer row), and show provider send throttles as a calm Warmup tab note instead of a red failure 2026-11-09 20:19:19 +01:00
Matthew Meszaros 8575bce7ca feat: make the Partnership plan Cloud-only by removing instance self-execution, and add the Partnership mailbox connect API (SMTP/IMAP with warmup, one-sign-in OAuth grant handoff, sign-in links), workspace-owned Google and Microsoft OAuth apps refreshed on the backend, Partnership access to placement tests and API keys, and the Partnership guide 2026-11-09 15:58:47 +01:00
Matthew Meszaros 4716ab963f feat: replace AI credits with a dollar balance charged on actual usage (provider price per model and unit plus an admin-set markup), with dollar plan allowances, one-click top-ups on the saved card billed as Stripe invoices, Checkout that saves the card, automatic reload with soft-decline retries and a monthly limit, low and empty balance notices, dollar spend limits, an invoices list, the Balance tab, header meter, banner and out-of-balance dialog, an admin pricing editor, the iOS billing screen and the balance guide, keeping the credit columns in step by trigger for mixed versions and rollback 2026-11-07 09:47:50 +01:00
Matthew Meszaros b6c31852c2 feat: scale the AWS consumer service between one and six tasks on a WorkerEventBacklog CloudWatch metric the consumer publishes when CONSUMER_METRICS_CLOUDWATCH is set, and let backend, tracking and forms opt into CPU target tracking through the runtime root's autoscaling map 2026-11-06 13:11:13 +01:00
Matthew Meszaros f5d974bfd2 feat: run consumer worker events on per-mailbox queues that share a lane limit so a slow mailbox holds only its own lane, and size that limit from the consumer's database pool unless CONSUMER_EVENT_CONCURRENCY is set 2026-11-05 13:30:39 +01:00
Matthew Meszaros 67d61f24ba feat: handle worker events on ordered per-mailbox consumer lanes (CONSUMER_EVENT_CONCURRENCY, default 8) and hold warmup receipt locks on direct sessions behind a transaction pooler so a held lock never occupies a pooled server connection that receipt acceptance needs 2026-11-05 09:50:32 +01:00
Matthew Meszaros 377e42a58c feat: keep free warmup only for workspaces that exist today through subscriptions.keeps_free_warmup (migration 000352), give new unpaid workspaces a zero mailbox allowance with warmup off, send inbox review notices only to workspaces whose plan opens the inbox, and update the locked screen, allowance dialog, billing, site pricing and docs to drop the hosted free plan and recommend Warmbly Cloud over self-hosting for deliverability 2026-11-04 20:27:44 +01:00
Matthew Meszaros d241a836f8 feat: sync mailboxes in realtime through Graph change notifications, Gmail Pub/Sub watches and IMAP IDLE that wake the owning worker with SYNC_WAKE, drain stored arrivals continuously, and add an opt-in autoscaling AWS worker fleet without public IPv4 scaled on a published fleet utilization metric 2026-11-03 15:11:24 +01:00
Matthew Meszaros fb7546e141 Merge pull request #1131 from warmbly/feat/bounce-cause-keeps-valid-addresses
feat: decide bounce suppression from the DSN status code and server reply through a shared dsn.Blame rule so mailbox-full, policy, reputation and system bounces keep the address and lead while address-invalid ones suppress, carry the per-recipient status and diagnostic in inbound bounce reasons, reuse the rule in emailverify, and add an optional TYPESAFE_ENDPOINT
2026-11-03 13:15:28 +00:00
Matthew Meszaros 36b77487e3 feat: read the per-recipient DSN group with its Status field as the deciding code, keep policy-tailed and sender-scoped refusals off recipient suppression, reopen completed campaigns when a refused lead is kept, and require https for TYPESAFE_ENDPOINT outside loopback and private hosts 2026-11-03 13:27:07 +01:00
Matthew Meszaros dd6a17ab6f Merge pull request #1135 from warmbly/feat/webhook-catalog-emitted-or-deprecated
feat: emit campaign.completed, inbox.reply_received, email_account.disconnected/error/synced, automation.run and bulk_operation.* webhooks from their control-plane sources, deprecate the five catalog events nothing can deliver, hide them from pickers, and add a source-scan test that every catalog event is emitted or deprecated
2026-11-03 12:01:15 +00:00
Matthew Meszaros 6afc5844b4 feat: emit campaign.completed, inbox.reply_received, email_account.disconnected/error/synced, automation.run and bulk_operation.* webhooks from their control-plane sources, deprecate the five catalog events nothing can deliver, hide them from pickers, and add a source-scan test that every catalog event is emitted or deprecated 2026-11-03 09:47:34 +01:00
Matthew Meszaros 7a9645123b feat: record the step A/B arm each contact was sent on campaign_contact_progress (migration 000338), report contact-based per-step arm stats with last-step attribution and a Bonferroni-corrected two-proportion z-test winner in GET /campaigns/:id/ab-analysis, implement ab_testing.auto_promote_winner as an audited consumer loop, and show per-arm results, winner and promotion in the step editor and Settings > Sending 2026-11-03 09:12:56 +01:00
Matthew Meszaros d1ae1e2e32 feat: decide bounce suppression from the DSN status code and server reply through a shared dsn.Blame rule so mailbox-full, policy, reputation and system bounces keep the address and lead while address-invalid ones suppress, carry the per-recipient status and diagnostic in inbound bounce reasons, reuse the rule in emailverify, and add an optional TYPESAFE_ENDPOINT 2026-11-03 08:31:36 +01:00
Matthew Meszaros 999597508c feat: provision private three-zone NATS JetStream and add gated Kafka migration readers with compatible codecs 2026-11-02 15:25:15 +01:00
Matthew Meszaros f400ff3b2b feat: authenticate each fleet node by its own NODE_CREDENTIAL on every node-only internal route (X-Warmbly-Node-Id and X-Warmbly-Node-Credential sent by every worker and consumer client, node id carried in the request context, named in broker access logs and bound to the heartbeat and node env), add NODE_REQUIRE_NODE_CREDENTIAL to refuse the fleet token once every node sends its credential, seed a dev worker credential, and document the rollout 2026-10-31 11:18:51 +01:00
Matthew Meszaros e326406549 feat: isolate unconfirmed campaign sends with portable contact protection and staged automatic recovery 2026-10-30 06:40:48 +01:00
Matthew Meszaros 81d1a95b98 feat: add the admin-offered $90 Partnership plan with server-priced offer checkout, an admin-granted credential-free Cloud warmup mode where a linked instance keeps every mailbox credential and executes Cloud-planned warmup sends and engagement from a sealed per-link command queue, and Cloud verification of instance reports through recipient-bound arrivals, receiver-proven sends, a placement cross-check reporter-trust audit, liveness, expiry and revocation 2026-10-26 17:49:32 +01:00
Matthew Meszaros 421a5ced5f feat: run the consumer's background mailbox credential reseal under instance sealing too, sealing any OAuth token still stored in plaintext under CREDENTIALS_ENCRYPTION_KEY as the configuration docs describe 2026-10-26 06:31:07 +01:00
Matthew Meszaros 537a622409 feat: seal mailbox SMTP/IMAP logins and OAuth tokens under KMS-protected workspace DEKs behind MAILBOX_CREDENTIALS_SEALING, carry them sealed in ADD_EMAIL and token refreshes, reseal stored credentials in a resumable consumer job, and cache DEKs in process with singleflight in front of Redis and KMS 2026-10-22 19:11:23 +02:00
Matthew Meszaros ee8581136e feat: require direct database sessions and verify authenticated worker dependencies during AWS deployment 2026-10-22 11:01:04 +02:00
Matthew Meszaros 99357a6823 feat: add Terraform AWS hosting with optional Postgres pooling and migration-gated deployments 2026-10-22 08:57:54 +02:00
geetansh 4f926b3efc feat: make Warmbly Cloud the owner of an enrolled mailbox's warmup ramp, window and days by relaying PATCH /emails/:id and warmup lifecycle edits to Cloud, mirroring Cloud's reported settings into the local columns on enroll, listing and a 15-minute consumer sync, keeping re-enrollment from overwriting them, and showing the Cloud-managed state on the Warmup tab 2026-10-19 10:53:05 +02:00
Matthew Meszaros fa6c935b12 feat: withdraw the reply intents and untouched follow-up tasks that mail from before its sender's contact produced through a one-time consumer cleanup, and credit an unthreaded contact message only to a campaign email sent before it arrived (#960) 2026-10-16 06:59:40 +02:00
Matthew Meszaros 74d4c1eb02 feat: enforce current workspace membership on owner-addressed realtime events and notification ingress, stored feeds, deferred email and push delivery 2026-10-10 10:54:59 +00:00
Devin AI 445e014b5e feat: recover mailbox synchronization across Gmail Outlook and IMAP with encrypted arrival replay and safe worker lifecycle 2026-10-09 14:24:10 +00:00
Matthew MeszarosandDevin AI e02655fd89 feat: coordinate TypeSafe billing recovery across backend and consumer, attribute token usage and reuse copy judgments during cache outages
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-10-08 04:28:37 +00:00
Matthew MeszarosandDevin AI b8303060dc feat: integrate current main with a single consumer schema and relation visibility guard while preserving migration cancellation diagnostics and regression coverage
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-10-07 07:49:50 +00:00
Matthew MeszarosandDevin AI dfed1e1593 feat: validate API UUIDs, recover expired Gmail history and guard consumer startup and stale frontend assets
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-10-07 07:19:15 +00:00
Matthew MeszarosandDevin AI 3acfe64109 feat: preserve self-host upgrades with exact realtime CSP discovery, consumer migration readiness and backend-compatible fleet targets
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-10-07 06:53:14 +00:00
Matthew MeszarosandDevin AI 5fe22b3b6a feat: preserve warmup identity across mailbox reconnects and retry provider filing with pre-disconnect cleanup
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-10-06 12:03:17 +00:00
Matthew Meszaros 36149cdebd feat: add Pipedrive CRM mode (migration 000262) with write-through deals, activities and notes, outreach logged as Pipedrive activities, per-connection webhooks, filter import, a Warmbly person panel and modals, a provider registry behind the shared /crm routes, a provider-aware dashboard, and centered popups replacing the integration side drawer 2026-10-04 19:20:48 +02:00
Matthew Meszaros e852a106a9 feat: serve every node-only internal route (data keys, message map, sync lookups, worker config, fleet heartbeat) on NODE_BROKER_TOKEN and leave only tracked links, domain redirects, page hits and forms on INTERNAL_API_TOKEN, render nodes only the node token when one is set, and have the installer generate a distinct NODE_BROKER_TOKEN for new installs and its own UPDATER_TOKEN 2026-10-04 03:21:58 -07:00
Matthew Meszaros d112657cf2 feat: wire suppression, subscription and hold dependencies into the consumer's Salesforce service, give Salesforce token refreshes their own single-flight keyspace, release the outbox lease in memory after each terminal write so writeback notes land and only on rows with a logged Task, drop stale Salesforce import previews, keep keys from the import row menu from opening the edit dialog, and document the Salesforce callback URL and OAuth variables 2026-10-04 09:10:37 +02:00
Matthew Meszaros 6b7e254e56 feat: add native Salesforce sync with Lead and Contact matching and links, a leased activity outbox that logs sends, replies, bounces, opt-outs and meetings as Tasks, Lead Status and Email Opt Out writeback, a pull loop with CRM pause rules, list view and Campaign imports, sandbox and My Domain OAuth that refreshes expired sessions, a Salesforce settings page with contact and inbox cards in web, and docs 2026-10-04 08:54:27 +02:00
Matthew Meszaros 37e91fc89f feat: add HubSpot CRM mode where HubSpot deals, pipelines, tasks, notes and owners are mirrored and written through, sends and replies log as HubSpot emails, exit rules hold campaigns, list import, sync health, setup wizard in web, HubSpot app project with record card and workflow action, and docs 2026-10-04 07:47:51 +02:00
Matthew Meszaros 61d16dfe9a feat: add Warmbly for Slack with the AI assistant in DMs, mentions and the assistant pane, unified inbox threads with reply, AI draft and lead actions, per-category notification routing and DMs, account linking, dashboard Slack settings, manifest and docs 2026-10-03 15:35:11 +02:00
Matthew Meszaros 57a26c4ee6 feat: attribute a received warmup email moved to spam on the evidence around it before charging anyone, never for mail that arrived in spam (warmup_received.landed_spam), holding each move 30 minutes in warmup_spam_moves and deciding provider on a cross-workspace correlation or a move straight after arrival with nobody there, owner on provider-reported read, unread or star activity in mailbox_owner_activity or a repeated uncorrelated pattern in a used mailbox, nobody otherwise, weighting a spam move as one strike, withdrawing owner verdicts a later correlation explains, clearing the strikes behind a hold an admin lifts or an appeal approves, migration 000233, docs and guide 2026-09-30 06:52:32 +02:00
Matthew Meszaros c5a981d86c feat: record a warmup deletion strike only after the worker searches the mailbox and finds the message in the trash or gone, so a move reported as a removal (Graph, a provider filter, a mailbox rule, a second Warmbly instance, our own filing) is never charged, withdraw strikes whose message is still in the mailbox and lift the pause or block they imposed, recheck deletion strikes recorded before the search once via warmup_tampering_events.verified_at (000229), and clarify the tampering reasons and warmup guide 2026-09-29 04:28:34 -07:00
Matthew Meszaros a749a8e353 feat: hold Warmbly Cloud's warmup standing on a linked self-hosted instance, syncing each enrolled mailbox's health into cloud_link_mailboxes so campaign gates, send plans, lifecycle, risk pacing, account health and the Advisor enforce a cloud quarantine, block or throttle, fire the usual health webhooks and realtime events, and carry a hold into the local pool on unenroll 2026-09-28 08:46:40 -07:00
Matthew Meszaros 7f324a38ac feat: open inbox placement tests to workspaces with probes rendered like the campaign send and paced as placement tasks, Message-ID matching instead of a subject token and warmup header, instance, workspace and Warmbly Cloud seed panels, a tracking comparison, scheduled campaign monitors with alerts and optional auto-pause, monthly allowances, realtime updates and org transfer registration 2026-09-25 20:48:15 -07:00
Matthew Meszaros 6a13fd5238 feat: report the Message-ID Gmail stamps on a send and resolve a reply's campaign through the Gmail thread and In-Reply-To so campaign replies stop tagging as sales pitches (with a warmblyctl --recheck-cold-inbound backfill that clears the stale labels and a tasks thread index), let a workspace add its own yes/no and pick-one tagging questions with plain-word labels and hold/stop/task actions, and add tagging languages (44 named to the classifier; for 27, only once chosen, tagging also cuts their quoted history and reads their away and non-delivery subjects), documented in the inbox tagging guide and the OpenAPI schema 2026-09-24 21:08:35 -07:00
Matthew Meszaros 256010725b feat: count only email steps in every sent total (contact drawer, Leads view and statuses, campaign progress, guardrails, org conduct, verification evidence, segments, admin, advisor), keep line breaks in synced body text and strip quoted history in any shape, classify delivery failures before out-of-office and tag them as bounces, record statusless failure notices from X-Failed-Recipients as permanent bounces, limit reply and inbox-tag opt-outs to people answering our outreach (never bounces, auto-replies or list mail), and recheck earlier reply opt-outs, lifting with an audit entry only those the message that wrote them no longer supports 2026-09-24 09:44:17 -07:00
Matthew Meszaros b196b3c8be feat: count warmup placements exactly once through a consumer sweep that seeds history in batches instead of a locking migration backfill, add a concurrent unplaced-receipt index, publish WARMUP_PLACEMENT only when a receipt is counted, share one spam-flag list across warmup, placement tests and unibox folders, name ranked mailboxes from their own rows, default from to 30 days before to, keep ScrollStrip scrolling to itself, refresh account statuses at most every 5 minutes, and document rescued as requested 2026-09-24 05:28:51 -07:00
Matthew Meszaros b2d54fc873 feat: record where every warmup email lands (inbox, Gmail tab, spam) per sender, day and recipient host in warmup_placement_daily, serve it from GET /analytics/warmup/placement, cap mailbox health at the measured 7-day inbox rate, and show it as an Inbox column, a mailbox Deliverability tab and a workspace placement section; replace every visible native checkbox in the dashboard with a themed Checkbox and make the mailbox drawer's tab bar scroll 2026-09-24 05:28:51 -07:00
Matthew Meszaros 1513419a2a feat: delete warmup mail from each mailbox once past a per-mailbox retention window (email_accounts.warmup_retention_days, else retention.warmup_mail_days, default 30) via a consumer sweep that retires the receipt and sender copy and a worker delete action that trashes on Gmail, deletes on Graph, expunges on IMAP and drops the stored body, prune per-message warmup records after retention.warmup_event_days, and count a warmup deletion as tampering only within 24 hours of arrival and never for a retired message, judging Gmail's Trash label on the same rule 2026-09-21 00:52:02 -07:00
Matthew Meszaros 6026168334 feat: stop blocking boot on the GeoIP download and swap the database in when it lands, retry a refused mirror with backoff honouring Retry-After, revalidate a database older than a week with If-Modified-Since instead of keeping the first copy forever, and add a twice-weekly job mirroring both MaxMind editions to a private bucket so the fleet stops spending a 30-a-day allowance per boot 2026-09-20 17:55:38 +02:00
Matthew Meszaros d1a4add567 feat: add the new CHECK constraints NOT VALID and validate them in migration 000189 so campaign_leads is not scanned under a write lock, keep a step's copy-judgment findings open when the Advisor's judge cap or an outage left it unjudged, validate every TypeSafe answer's type and bounds and version the copy-judgment cache key, mirror the reads-as-bulk rule in the editor's all-clear, and refuse inbox classification without a configured client 2026-09-19 23:59:19 -07:00