Carry attachment references inside the stored email body blob, resolve bytes on workers, and encode attachments for Gmail API and SMTP sends without changing the Kafka send-email contract.
Remove the worker-side dwell timer now that delayed warmup engagement is scheduled durably by the consumer.
Keep DelaySeconds only for wire compatibility and run received warmup action batches immediately on the worker.
Wire warmup health webhooks in the consumer and log skipped warmup actions when a recipient mailbox has no assigned worker.
Run foldering and spam rescue immediately while keeping only low-stakes engagement signals behind the dwell delay.
Add star-rate settings to warmup content controls and include star actions in generated engagement plans.
Execute Gmail stars via STARRED labels while keeping IMAP behavior a no-op to avoid duplicate flagging.
Add warmup content generation and admin review surfaces, plus mailbox warmup appeal/status APIs.
Track warmup engagement and tampering signals so unsafe mailboxes can be handled by the warmup flow.
Reserve dedicated worker allocation for the control plane, auto-promote spare capacity when needed, and keep risky or quarantined mailboxes off clean shared workers.
Add explicit warmup participant roles so connected accounts can provide recipient-only pool capacity without becoming senders. Cap warmup scheduling by eligible recipients, avoid same-day recipient reuse, reschedule exhausted pools, and reconcile missing warmup task chains.
Go CI fails on golangci-lint's gofmt check. Ran gofmt -w against
every file the linter named plus a handful of others that drifted
during the autonomous-fleet work. No semantic changes — alignment
of struct field whitespace and one mis-indented import block.
gofmt -l ./... is now empty; go build + go vet are clean.
Brings in PR #15 (email warmup process 4) plus its preceding commits:
customer-defined warmup routing on premium pool, free-trial warmup +
1 inbox for 14 days, customer webhook subscriptions with HMAC signing
+ retry, bumped default API rate limits to 100 req/s with flat per-
user/per-plan caps, plus dev-fixture additions.
One real conflict: internal/client/smtpimap/imap/client.go added
distinct imports on each side (this branch added 'net' for the
*net.TCPAddr BindIP field; main added 'sync' for a Mutex). Kept both.
Everything else auto-merged additively:
cmd/backend/main.go - imports + handler fields + DI lines
internal/api/handler/handler.go - new fields next to existing ones
internal/api/routes.go - new route group next to existing ones
Full build + test suite pass (no regressions).
Three small follow-ups that turn the fleet management system from 'all
the pieces ship green' into 'actually produces telemetry':
cmd/worker/main.go: go workerService.RunHealth(ctx, 30s) alongside
Heartbeat. The sampler snapshots rolling 1m counters into a WorkerHealth
event via the existing event bus + codec path.
cmd/backend/main.go: background goroutine refreshes
worker_capacity_view every minute via REFRESH MATERIALIZED VIEW
CONCURRENTLY. The assignment loop, Rebalancer, Scaler, and
QuarantineEvaluator all read from the view, so it's the freshness gate
for the whole system.
internal/app/worker/event_send_email.go + health_record.go: classify
every wmail.SendResult into the right counter (auth / rate-limit /
bounce-hard / bounce-soft / success) and record SMTP latency. Falls
back to free-text message classification when the error code is
generic, so signal stays useful as new error paths are added.
End-to-end: a worker that bounces 10% of sends now lands in the
'quarantined' band within 5min of the QuarantineEvaluator tick,
auto-drains via Rebalancer, and triggers a Scaler alert if its
removal drops fleet capacity below the warning threshold.
Workers emit a WorkerHealth event every 30s with assigned mailbox count,
IMAP IDLE connections, memory, goroutines, and rolling 1h send/bounce/
complaint/auth-error/rate-limit counters. Consumer writes them to
worker_health_samples.
Schema additions on workers: egress_kind (cold_smtp / oauth_api /
warmup_only), health_state (healthy / watch / throttled / quarantined /
blocked), load_score (weighted utilization).
worker_capacity_view aggregates the latest hour of samples into a
per-worker capacity row used by the assignment loop. Effective capacity
= base_ceiling(egress_kind) × health_multiplier × age_ramp_multiplier
so a fresh worker earns its way up to base capacity over 72h, and a
worker with rising bounces or complaints automatically gets less load.
MailboxWeight returns 1.0 for cold_smtp, 0.05 for Gmail/Graph API
(worker IP doesn't matter), 0.4 for warmup-only. AssignWorkerToEmail
resolves the mailbox's weight and SelectSharedWorker filters by
headroom + sorts by utilization, so a 200-mailbox OAuth worker and a
16-mailbox cold worker balance fairly.
UnassignWorkerFromEmail refunds the load_score symmetrically.
WorkerService now holds eventbus.EventBus + codec.Codec instead of
*kafka.Producer / *kafka.Consumer. Receive() satisfies the
eventbus.Handler signature; Produce() goes through Codec.Serialize +
Bus.Publish.
events.Publisher likewise switches to (bus, codec) and stops
serializing via *kafka.Avrov2 directly.
The kafka package and its Avrov2/Producer/Consumer types remain for
the few non-worker call sites (tracking consumer, validate_credentials)
that haven't been migrated yet; bundled with KafkaBus.Producer() so
existing Avro framing on Kafka is preserved.
Worker boot wiring is split into cmd/* entry-point commits.
Seven call sites stop reaching through the embedded *s3.Client and
instead use the high-level storage.Store methods. Same runtime behavior
on the AWS path; opens the door to the Filesystem backend for
self-hosters.
avatar.go retains an S3-specific path for public-ACL + cache-control
on uploaded avatars and falls back to ServiceUnavailable on non-S3
backends. A future PublicStore interface could clean that up.
unibox/storage.go GetBody now propagates the emsg.DecodeBinary error
that the original code dropped on the floor.
rename outbound warmup verification header from X-Warmbly-Token to a
generic X-Mailtrace-Verify and accept both on receive during rollout.
add slot-based subject synthesis that fires ~40% of the time, yielding
thousands of unique strings on top of the static corpus so vendors
cannot trivially cluster on a fixed subject list.
mark_read, mark_important, remove_from_spam, and move_to_warmbly now run
on outlook and custom smtp/imap mailboxes, not just gmail. routes by
provider in the worker handler. carries source mailbox UIDValidity on
the action so the worker can SELECT the right folder before mutating.
Disable the linters that fire on legacy code without flagging real
bugs: `unused` (orphan repos kept for future feature flags),
`unconvert` (defensive type conversions), `gosimple` (style
suggestions in code we don't want to touch).
govet: disable `shadow` (idiomatic `err :=` re-decls in transaction
patterns) and `nilness` (legitimate defensive nil checks that look
tautological to the analyzer).
Ran `gofmt -w internal/ cmd/` — every Go file now passes
gofmt -l with no output.
Kept: govet, staticcheck, ineffassign, typecheck, bodyclose, noctx,
sqlclosecheck, gofmt, goimports, misspell — the real-bug checks.
New method on WorkerAssignmentService picks the least-loaded shared
worker whose risk_pool matches the mailbox's risk band. Three-step
fallback chain:
1. Exact match: same pool, same tier
2. Fall back to clean pool of the same tier when no matching-pool
worker is available (better to land risky mailboxes on clean
workers than refuse; the rebalancer will move them later)
3. Last resort: any worker of the right tier (preserves legacy
behavior for installations that haven't provisioned risky/
quarantine pools)
Existing SelectSharedWorker is unchanged so call sites that don't
know about risk bands keep working. The next commit (background
rebalancer) is the first consumer of the new method.
The free-tier-vs-paid separation in AssignWorkerToEmail is one of those
rules that's silently load-bearing: if a free org ever slips onto a
premium worker, the IPs of paying customers absorb the deliverability
hit. The code is correct today (strict isPaidOrg check at line 67, free/
premium pool sync at line 116), but nothing was guarding against a
regression.
Five table-thin tests, hand-rolled stub repos (embed the interface as a
nil field so unused methods panic loudly):
- free org → free shared worker → free warmup pool
- paid org → premium shared worker → premium warmup pool
- paid org with DedicatedWorkers > 0 + an assignment → dedicated worker
- paid org with DedicatedWorkers > 0 but no assignment → falls back to
premium shared (not free!)
- SelectSharedWorker with no workers → ErrNoAvailableWorkers
No code changes — this commit is documentation.
Matthew Meszaros