Matthew Meszaros
d456bc48c6
feat: fix the Warmbly Cloud pool link across both roles ( #262 ): take an enrolled mailbox out of this instance's own warmup pool so local partners stop writing to it and their unverifiable warmup stops landing in the owner's unibox, recognise the cloud's warmup mail whose verify header did not survive delivery through a new warmup-deliveries lookup that ignores consumed_at because instance and cloud read the same mailbox, move the managed-mailbox access token route behind NODE_BROKER_TOKEN so the internet-facing tracking and forms services can no longer mint a live provider token, scope pause and resume to the caller's workspace, keep an enrolled mailbox listed once it goes inactive, release the cloud copy when the local mirror row cannot be written, refuse the one-time handshake when CREDENTIALS_ENCRYPTION_KEY is missing, blank an expired code's plaintext instance token, and stop errx answering 200 for a status outside its table
2026-09-12 06:58:25 -07:00
Matthew Meszaros
ced741e352
feat: make PostHog the default error tracker across every runtime while keeping Sentry fully supported alongside or instead of it, by turning internal/observability/errs into a two-sink fan-out with a local-log fallback, adding $exception capture to the Go services, the Rust tracking service, the Elixir realtime service and the dashboard, admin and form apps, reporting gin panics with their route, request id, workspace and user, attaching that identity plus a route and failed-request trail to browser exceptions, and wiring POSTHOG_ERROR_TRACKING, the node join env, compose, source-map upload and the docs to match
2026-09-10 19:11:32 +02:00
Matthew Meszaros
494996ea8a
feat: stop the backend image copying scripts/install-worker.sh, which the pull-based fleet change deleted, and clear the rest of that removal's dangling references out of the backend systemd unit, deploy/README, architecture and troubleshooting docs
2026-09-09 08:56:05 -07:00
Matthew Meszaros
704fecdcc0
feat: settle on WARMBLY_NODE_REGION as the one region variable for every role, because the join script wrote it while the worker still read WORKER_REGION, so a joined worker reported no region and the placement geography term stayed dead for exactly the machines that had been given one
2026-09-09 05:04:55 -07:00
Matthew Meszaros
435dbb522f
feat: replace the worker tier/type/risk-pool/egress categories with a scored placement model and make the fleet pull-based, so a machine joins with one command, workers and consumers share one node registry with usage and liveness, nodes self-update to the version the control plane resolves, and the Hetzner provisioning, worker profiles and SSH orchestrator are removed
2026-09-09 04:54:01 -07:00
Matthew Meszaros
d6ddf1f170
feat: fix implicit-TLS SMTP on 465 and IMAP STARTTLS on 143 behind a stored per-mailbox security mode that accepts any port, stop worker ID churn orphaning mailbox assignments via flock-claimed persistent worker ids, give the unibox a standard mail-folder sidebar (inbox/sent/drafts/archive/spam/trash) backed by a provider-derived folder column, and expose the AI tool registry over REST for non-MCP function-calling agents ( #283 )
2026-09-01 03:53:19 -07:00
Matthew Meszaros
2a831e9783
feat: let a linked self-hosted instance sign Google and Microsoft mailboxes in through Warmbly Cloud's own OAuth apps and send with cloud-brokered access tokens: the cloud runs the consent (pool_link_mailboxes.managed, brokered state in Redis, the existing /addresses/*/callback completes it and redirects to the instance's /cloud-oauth/done), keeps the refresh grant, mints short-lived tokens at /pool-link/instance/mailboxes/:id/token and refuses them for revoked links, removed, inactive or blocked mailboxes; the instance mirrors such mailboxes without a credential (cloud_link_mailboxes.managed), ships them to the worker as brokered so goog/msgraph init on a token source that pulls from /api/v1/internal/cloud-link/token/:id, lets the consumer ignore cloud warmup tokens for enrolled mailboxes, and can adopt mailboxes connected directly on the workspace; Add account shows the cloud path and the adoptable list, and the Warmbly Cloud guide documents the model
2026-08-29 09:50:52 -07:00
Matthew Meszaros
6b614ad4bb
feat: never hand a send to a worker that is not heartbeating (registry row plus the Redis heartbeat key, which the worker now sets at boot instead of 90 seconds later), because a command queued for a dead worker is neither executed nor answered; reload a worker's mailboxes the moment its boot heartbeat arrives instead of waiting for the reconciler's republish window, make the publisher fail instead of silently succeeding when no bus or object storage is configured, and log a send that could not reach a worker to the campaign feed as a red, retrying entry
2026-08-23 10:18:50 -07:00
Matthew Meszaros
a75ea012a0
feat: import a mailbox's recent history on connect and govern sync by fair use: a backfill on every provider (newest first, inside an operator-editable window and cap, resumable through a durable per-provider cursor relayed as SYNC_STATE), a per-mailbox sync governor with priority, live and backfill lanes on shared Redis windows that defers over-budget mail with the cursor held instead of dropping it and only deactivates a mailbox for a flood or chronic daily overage, sync.* budgets on the admin instance settings shipped inside ADD_EMAIL, saved IMAP folder cursors and last_synced_at finally written, a Sync card in the mailbox drawer fed by GET /emails/:id/sync, and docs
2026-08-18 09:09:52 -07:00
Matthew Meszaros
8bd2c2b57a
feat: make self-hosting work end to end and rewrite the guide around what was tested ( #97 )
2026-08-13 09:47:46 +02:00
Matthew Meszaros
543d97f6f3
feat: wire the no-cloud providers into the consumer and worker
2026-07-20 09:56:29 +02:00
Matthew Meszaros
52c5f224fa
feat: rebuild the provider oauth2 config locally on the worker so delegated mailbox tokens refresh, and ship BOX_ google/outlook client credentials to provisioned workers through the orchestrator env
2026-07-04 08:07:49 +02:00
Matthew Meszaros
2430095197
feat: move infrastructure state off dynamodb
...
Remove DynamoDB-backed storage paths, add Postgres/HTTP repositories for mailbox state maps, wire the internal message-map API, and add provisioning runner/migration plumbing.
2026-06-02 15:54:12 +02:00
Matthew Meszaros
d8a546eca5
feat: add worker enrollment install
2026-05-30 05:10:27 +00:00
Matthew Meszaros
e790818d35
fleet: wire health collection so the autonomous loops actually have data
...
Three small follow-ups that turn the fleet management system from 'all
the pieces ship green' into 'actually produces telemetry':
cmd/worker/main.go: go workerService.RunHealth(ctx, 30s) alongside
Heartbeat. The sampler snapshots rolling 1m counters into a WorkerHealth
event via the existing event bus + codec path.
cmd/backend/main.go: background goroutine refreshes
worker_capacity_view every minute via REFRESH MATERIALIZED VIEW
CONCURRENTLY. The assignment loop, Rebalancer, Scaler, and
QuarantineEvaluator all read from the view, so it's the freshness gate
for the whole system.
internal/app/worker/event_send_email.go + health_record.go: classify
every wmail.SendResult into the right counter (auth / rate-limit /
bounce-hard / bounce-soft / success) and record SMTP latency. Falls
back to free-text message classification when the error code is
generic, so signal stays useful as new error paths are added.
End-to-end: a worker that bounces 10% of sends now lands in the
'quarantined' band within 5min of the QuarantineEvaluator tick,
auto-drains via Rebalancer, and triggers a Scaler alert if its
removal drops fleet capacity below the warning threshold.
2026-05-27 16:03:35 +00:00
Matthew Meszaros
5463e3986e
worker: multi-IP support (one systemd unit per Primary IP)
...
Hetzner CX32 + 16 Primary IPs becomes 16 sending identities with one
install command, without expanding ops complexity.
cmd/worker/main.go: WORKER_ID now resolves via 4-tier precedence:
1. WORKER_ID env (explicit UUID)
2. WORKER_BIND_IP env (derive UUIDv5 from the bound IP)
3. hostname-as-UUID (legacy single-IP VPS)
4. generated UUID (local dev fallback)
Boot also constructs the chosen Codec + EventBus + EncryptedKeyStore
via the FromEnv factories from earlier commits, so a worker process is
fully configured by its envelope env file plus the runtime config it
pulls from the backend on first boot.
scripts/install-worker.sh gains --ips <ipv4,ipv4,...> which:
- writes a warmbly-worker@.service systemd template
- drops a per-instance env file at /etc/warmbly/instances/<dashed-ip>.env
with WORKER_BIND_IP and WORKER_ID
- shares one /etc/warmbly/worker.env for the common config
- --status, --update, --uninstall now multi-IP aware
- single-IP mode preserved when --ips is absent
5 worker tests pin the UUIDv5 derivation against the installer's
uuidgen --sha1 output so the two never drift.
docs/MULTI_IP_WORKERS.md is the operator runbook with the Hetzner
recipe, OS-level IP attachment, rDNS automation, day-2 ops, and the
25%-of-fleet blast-radius rule.
2026-05-27 14:43:20 +00:00
Matthew Meszaros
a741b4939a
feat: implement worker HandleAddEmail and HandleRemoveEmail handlers
2026-04-10 06:56:35 +00:00
Matthew Meszaros
0799020dac
feat: add metrics, warmup content variety, tz-aware scheduling, org budget, admin stubs, and bug fixes
2026-04-09 14:33:45 +00:00
Matthew Meszaros
c12fd93def
Add local Sentry logging
2026-02-14 05:49:49 +01:00
Matthew Meszaros
141bc54974
Add sample auth UI theme
2026-02-10 19:30:47 +01:00
Máté Mészáros (Laptop)
7ed5719c15
Config & Deploy
2026-01-29 08:50:38 +01:00
Máté Mészáros (Laptop)
41624a6f79
Analytics & Tracking
2026-01-29 05:59:04 +01:00
Matthew Meszaros
772c19820d
New Repository: Add Backend Code
2026-01-17 14:11:14 +00:00
Matthew Meszaros