Matthew Meszaros bea3f21da2 feat: replace the worker's flat mailbox-sync rate limiter, which fetched every message of every IMAP folder oldest first on first sight and then deactivated the mailbox after 100 new emails in five minutes with no resume state while Gmail and Outlook imported nothing historical at all, with a per-mailbox sync governor and a real backfill on every provider: three Redis fixed-window lanes shared across workers (priority for mail in a conversation the mailbox owns, resolved through the internal own-conversation endpoint and never held behind ordinary inbound mail; live for new mail after connect with burst, hourly, per-mailbox daily and per-organization daily budgets; backfill paced per minute and charged to the org budget) that defer over-budget mail with the provider cursor pinned before the first deferred message rather than dropping it, escalate to the existing EMAIL_RATE_LIMITED path only for a flood (SyncFloodPerHour new live messages seen in one hour, deduped through a lane cache so a held backlog is not re-counted every pass) or chronic overage (per-mailbox daily budget exhausted on three of the last seven days), cancel the mailbox context on termination so the sync goroutine no longer keeps ticking against a removed entry, and fail open on a Redis outage; an initial import newest first inside the policy window and cap with a resumable per-provider cursor relayed as SYNC_STATE, walking IMAP via UID SEARCH SINCE per eligible folder (skipping trash, drafts, junk and Gmail's All Mail) below a saved UID floor, Gmail via messages.list with a page token, and Graph via a filtered ordered folder listing over inbox, sent items and archive; IMAP live sync moved from CHANGEDSINCE window fetches to UID SEARCH MODSEQ plus envelope-only fetches with bodies read only for admitted messages, first sight of a folder now baselining its HIGHESTMODSEQ instead of walking it, and saved folder cursors seeded from the payload; Gmail history and Graph delta callbacks reshaped to offer ids first so dedupe and admission happen before hydration, with history checkpoints and delta links only advancing past fully stored records and pages, bounded pages per pass, Graph now tracking sent items live so a thread shows both sides, Graph priming walking uncapped and unpersisted to its deltaLink, and a Graph 404 on a just-deleted message treated as a skip; Gmail and Graph bodies capped at MaxEmailBodySize like IMAP already was; a provider 429 during sync backing the loop off instead of being relayed as a rate-limit event that deactivated the mailbox; the loop interval now adaptive with jitter and a five minute ceiling while held; a republished ADD_EMAIL applying a changed policy to an already loaded mailbox; and the dead StartImapWorker removed

2026-08-18 08:43:35 -07:00

..

backend

feat: add the control plane for mailbox sync fair use, so a mailbox syncs under an operator-editable policy and its progress survives worker replacement: a sync section on the instance settings document (backfill window in days, backfill cap per mailbox, daily new-mail budget per mailbox and per organization, each clamped on read and write with compiled defaults in constants.go), models.SyncPolicy and models.SyncState with a provider-shaped jsonb SyncCursor, a new email_sync_state table plus an index on tasks.message_id that the reply lookup was scanning sequentially without, an EmailSyncStateRepository whose Put also stamps email_accounts.last_synced_at which nothing had written since the baseline so every admin and dashboard Last synced surface read NULL, an OrganizationID and Sync block on the ADD_EMAIL payload resolved by the loader from instance settings and the saved state and, for IMAP, the saved unibox_mailboxes folder cursors that the loader had never populated so every worker restart re-walked every folder from scratch, a SYNC_STATE consumer handler that persists the relay and publishes ACCOUNT_SYNC_STATE plus a warning when the import completes or fair use flips, an internal own-conversation endpoint the worker's priority lane asks whether a new message replies to a campaign task, a mapped message or a stored thread, GET /emails/:id/sync for the dashboard, and SYNC_FLOOD and SYNC_FAIR_USE mail error codes with user copy for the two patterns that deactivate a mailbox

2026-08-18 08:43:20 -07:00

consumer

feat: add the control plane for mailbox sync fair use, so a mailbox syncs under an operator-editable policy and its progress survives worker replacement: a sync section on the instance settings document (backfill window in days, backfill cap per mailbox, daily new-mail budget per mailbox and per organization, each clamped on read and write with compiled defaults in constants.go), models.SyncPolicy and models.SyncState with a provider-shaped jsonb SyncCursor, a new email_sync_state table plus an index on tasks.message_id that the reply lookup was scanning sequentially without, an EmailSyncStateRepository whose Put also stamps email_accounts.last_synced_at which nothing had written since the baseline so every admin and dashboard Last synced surface read NULL, an OrganizationID and Sync block on the ADD_EMAIL payload resolved by the loader from instance settings and the saved state and, for IMAP, the saved unibox_mailboxes folder cursors that the loader had never populated so every worker restart re-walked every folder from scratch, a SYNC_STATE consumer handler that persists the relay and publishes ACCOUNT_SYNC_STATE plus a warning when the import completes or fair use flips, an internal own-conversation endpoint the worker's priority lane asks whether a new message replies to a campaign task, a mapped message or a stored thread, GET /emails/:id/sync for the dashboard, and SYNC_FLOOD and SYNC_FAIR_USE mail error codes with user copy for the two patterns that deactivate a mailbox

2026-08-18 08:43:20 -07:00

migrate

feat: guard warmup reconciliation by access

2026-06-01 11:19:44 +02:00

sandbox

feat: add the sandbox seeder and simulator (cmd/sandbox, internal/sandbox) - seeds the Sunrise Labs showcase org (paid plan, six live mailboxes with sealed mailpit/dovecot credentials, active tracked campaigns, pre-verified contacts, warmup pool membership) and repairs every fixture smtp_imap account and .test contact; the simulator plays the internet by polling mailpit, appending mail into dovecot inboxes for the real IMAP sync to pick up, hitting tracking pixels and click tickets, and replying as hash-stable contact personas with correct In-Reply-To threading

2026-07-11 17:31:42 +02:00

seed

feat: seed the dev org as a mid-flight workspace: new internal/seed dev_org.go + dev_history.go give dev@warmbly.com 4 warmed premium-pool mailboxes, folders/tags/categories with real bindings (email_tags, campaign_folders, contact_categories, unibox thread labels), ~30 contacts with suppression states, an active 3-step campaign with 24 leads on a deterministic funnel and NOW()-relative sends including today, 14d campaign/warmup/daily stats rollups, a 13-message unified inbox that always resolves a dev-org sending mailbox, CRM pipeline with deals/tasks/notes/activity, reply templates, notifications, AI credit ledger history, and idempotent campaign logs

2026-07-18 12:00:24 +02:00

warmblyctl

feat: organization data export and import for moving a workspace between instances (#132)

2026-08-18 07:53:39 -07:00

worker

feat: replace the worker's flat mailbox-sync rate limiter, which fetched every message of every IMAP folder oldest first on first sight and then deactivated the mailbox after 100 new emails in five minutes with no resume state while Gmail and Outlook imported nothing historical at all, with a per-mailbox sync governor and a real backfill on every provider: three Redis fixed-window lanes shared across workers (priority for mail in a conversation the mailbox owns, resolved through the internal own-conversation endpoint and never held behind ordinary inbound mail; live for new mail after connect with burst, hourly, per-mailbox daily and per-organization daily budgets; backfill paced per minute and charged to the org budget) that defer over-budget mail with the provider cursor pinned before the first deferred message rather than dropping it, escalate to the existing EMAIL_RATE_LIMITED path only for a flood (SyncFloodPerHour new live messages seen in one hour, deduped through a lane cache so a held backlog is not re-counted every pass) or chronic overage (per-mailbox daily budget exhausted on three of the last seven days), cancel the mailbox context on termination so the sync goroutine no longer keeps ticking against a removed entry, and fail open on a Redis outage; an initial import newest first inside the policy window and cap with a resumable per-provider cursor relayed as SYNC_STATE, walking IMAP via UID SEARCH SINCE per eligible folder (skipping trash, drafts, junk and Gmail's All Mail) below a saved UID floor, Gmail via messages.list with a page token, and Graph via a filtered ordered folder listing over inbox, sent items and archive; IMAP live sync moved from CHANGEDSINCE window fetches to UID SEARCH MODSEQ plus envelope-only fetches with bodies read only for admitted messages, first sight of a folder now baselining its HIGHESTMODSEQ instead of walking it, and saved folder cursors seeded from the payload; Gmail history and Graph delta callbacks reshaped to offer ids first so dedupe and admission happen before hydration, with history checkpoints and delta links only advancing past fully stored records and pages, bounded pages per pass, Graph now tracking sent items live so a thread shows both sides, Graph priming walking uncapped and unpersisted to its deltaLink, and a Graph 404 on a just-deleted message treated as a skip; Gmail and Graph bodies capped at MaxEmailBodySize like IMAP already was; a provider 429 during sync backing the loop off instead of being relayed as a rate-limit event that deactivated the mailbox; the loop interval now adaptive with jitter and a five minute ceiling while held; a republished ADD_EMAIL applying a changed policy to an already loaded mailbox; and the dead StartImapWorker removed

2026-08-18 08:43:35 -07:00