Store delayed warmup engagement actions in Postgres and drain them from the consumer so read, important, and star actions survive worker restarts.
Keep foldering and spam rescue immediate while routing delayed actions to the mailbox's current worker at fire time.
Wire warmup health webhooks in the consumer and log skipped warmup actions when a recipient mailbox has no assigned worker.
Run foldering and spam rescue immediately while keeping only low-stakes engagement signals behind the dwell delay.
Add star-rate settings to warmup content controls and include star actions in generated engagement plans.
Execute Gmail stars via STARRED labels while keeping IMAP behavior a no-op to avoid duplicate flagging.
Record warmup spam placements by recipient provider and surface provider breakdowns in the admin health summary.
Track warmup replies and use stable static conversation ids so warmup analytics can correlate content reliably.
Add warmup content generation and admin review surfaces, plus mailbox warmup appeal/status APIs.
Track warmup engagement and tampering signals so unsafe mailboxes can be handled by the warmup flow.
Update account risk bands as warmup monitoring signals arrive, including spam placement, user complaints, invalid tokens, and worker rate-limit abuse. Trigger risk rebalancing after warmup health sweep transitions so risky accounts move out of clean pools sooner.
Brings in PR #15 (email warmup process 4) plus its preceding commits:
customer-defined warmup routing on premium pool, free-trial warmup +
1 inbox for 14 days, customer webhook subscriptions with HMAC signing
+ retry, bumped default API rate limits to 100 req/s with flat per-
user/per-plan caps, plus dev-fixture additions.
One real conflict: internal/client/smtpimap/imap/client.go added
distinct imports on each side (this branch added 'net' for the
*net.TCPAddr BindIP field; main added 'sync' for a Mutex). Kept both.
Everything else auto-merged additively:
cmd/backend/main.go - imports + handler fields + DI lines
internal/api/handler/handler.go - new fields next to existing ones
internal/api/routes.go - new route group next to existing ones
Full build + test suite pass (no regressions).
Workers emit a WorkerHealth event every 30s with assigned mailbox count,
IMAP IDLE connections, memory, goroutines, and rolling 1h send/bounce/
complaint/auth-error/rate-limit counters. Consumer writes them to
worker_health_samples.
Schema additions on workers: egress_kind (cold_smtp / oauth_api /
warmup_only), health_state (healthy / watch / throttled / quarantined /
blocked), load_score (weighted utilization).
worker_capacity_view aggregates the latest hour of samples into a
per-worker capacity row used by the assignment loop. Effective capacity
= base_ceiling(egress_kind) × health_multiplier × age_ramp_multiplier
so a fresh worker earns its way up to base capacity over 72h, and a
worker with rising bounces or complaints automatically gets less load.
MailboxWeight returns 1.0 for cold_smtp, 0.05 for Gmail/Graph API
(worker IP doesn't matter), 0.4 for warmup-only. AssignWorkerToEmail
resolves the mailbox's weight and SelectSharedWorker filters by
headroom + sorts by utilization, so a 200-mailbox OAuth worker and a
16-mailbox cold worker balance fairly.
UnassignWorkerFromEmail refunds the load_score symmetrically.
introduce two distinct warmup_spam_reports report types:
- spam_placement: provider classifier put the message in Junk on arrival
- user_complaint: recipient explicitly flagged the warmup message
detect placement at warmup arrival via the incoming message flags and
record it through a new WarmupService.RecordSpamPlacement. evaluate the
two signals independently in the health sweep — user complaints now
have their own watch/quarantine/block thresholds (0.5 / 1.5 / 3.0%)
since they are a stronger negative signal per event than placement.
rename outbound warmup verification header from X-Warmbly-Token to a
generic X-Mailtrace-Verify and accept both on receive during rollout.
add slot-based subject synthesis that fires ~40% of the time, yielding
thousands of unique strings on top of the static corpus so vendors
cannot trivially cluster on a fixed subject list.
mark_read, mark_important, remove_from_spam, and move_to_warmbly now run
on outlook and custom smtp/imap mailboxes, not just gmail. routes by
provider in the worker handler. carries source mailbox UIDValidity on
the action so the worker can SELECT the right folder before mutating.
Disable the linters that fire on legacy code without flagging real
bugs: `unused` (orphan repos kept for future feature flags),
`unconvert` (defensive type conversions), `gosimple` (style
suggestions in code we don't want to touch).
govet: disable `shadow` (idiomatic `err :=` re-decls in transaction
patterns) and `nilness` (legitimate defensive nil checks that look
tautological to the analyzer).
Ran `gofmt -w internal/ cmd/` — every Go file now passes
gofmt -l with no output.
Kept: govet, staticcheck, ineffassign, typecheck, bodyclose, noctx,
sqlclosecheck, gofmt, goimports, misspell — the real-bug checks.
New background job in the consumer process:
1. Pulls up to 1000 mailbox candidates joined with their worst warmup
health state (across all pools they participate in) and their
current worker's risk_pool. Dedicated workers are excluded — single
tenant, segregation not applicable.
2. Recomputes risk_band from health state via RiskBandFromHealth.
If it changed, writes the new band.
3. If the band's matching pool doesn't equal the worker's pool, picks
a new worker via SelectSharedWorkerForBand and migrates the mailbox.
Increments/decrements account counts.
4. Logs each migration to admin_audit_log with action=
"risk_rebalance_migrate" so operators see what moved and why.
Boot-time run + hourly ticker. Rebalancing is intentionally batch, not
event-driven: warmup health states change on a slow rolling-window basis
(warmup_health_sweep is also hourly), so reacting in real time gains
nothing and would cause thundering-herd migrations.
JobsService gets an AssignmentService dep. Nil disables the job (lets
self-hosters opt out by simply not wiring it).
When the dead-worker job reassigns email accounts from a worker whose
heartbeat expired, write a row into admin_audit_log so the dashboard's
audit viewer surfaces these system actions alongside admin-driven ones.
admin_user_id is uuid.Nil (the platform identity), so admins searching
the log can distinguish "system did this" from "an admin did this" by
filtering on that ID. Details include the replacement worker, account
count, and reason.
JobsService gets an optional AdminRepo dep. Nil disables logging — keeps
the contract loose for any other call site that doesn't have one.
Workers heartbeat into Redis every 90s as RFC3339 timestamp values with a
3-min TTL. The dashboard surfaces liveness based on workers.last_seen_at,
but until now nothing populated that column — the "Live" badge was always
red.
New 60s job in the consumer reads each active worker's Redis heartbeat
value, parses the timestamp, and writes it to workers.last_seen_at. Runs
on its own interval (separate from the 5-min dead-worker detection job,
which does heavier reassignment work) so the UI sees fresh data within a
minute.
- Replace all fmt.Printf calls in Kafka consumer/producer, tracking consumer,
and user email task with structured zerolog (log.Warn/Error/Info)
- Fix hashURL using SHA-256 instead of naive first-8-chars+length approach
which was collision-prone for deduplication
Matthew Meszaros