Commit Graph
98 Commits
Author SHA1 Message Date
Matthew Meszaros da82993846 feat: add campaign workflow backend support
Add sequence action-node storage and execution, template conditional rendering, lead progress state, profile updates, webhook fan-out throttling, and supporting repository fixes.
2026-06-06 07:49:52 +02:00
Matthew Meszaros e92ec7c661 feat: add ai writing credit backend
Add an organization credit ledger with idempotent consumption, plan credit grants, Anthropic/OpenAI writing providers, and a gated generation endpoint for campaign writing assistance.
2026-06-05 06:03:32 +02:00
Matthew Meszaros 3c440a9f0c feat: add campaign attachment endpoints
Persist campaign attachment metadata, enforce attachment storage quotas, and expose list, upload, and delete routes backed by object storage.
2026-06-05 06:03:13 +02:00
Matthew Meszaros 630e69037a feat: add per-step campaign ab variants
Scope campaign A/B variants to individual sequence steps and select step variants deterministically per contact and step. Keep campaign-level variants on the existing assignment flow.
2026-06-05 06:03:00 +02:00
Matthew Meszaros 2a2e248ad9 feat: add campaign sequence branching backend
Store per-step branching conditions, validate branch targets, and select campaign contact steps by following the flow graph. Preserve linear progression for steps without branch conditions.
2026-06-05 06:02:55 +02:00
Matthew Meszaros 5b9029d2a8 feat: add campaign schedule windows backend
Store per-day schedule windows on campaigns and validate updates before persistence. Use the schedule windows in campaign scheduling, including legacy day/time derivation for existing campaigns.
2026-06-04 18:56:43 +02:00
Matthew Meszaros d6f055da19 feat: add lead sync backend 2026-06-04 11:49:55 +02:00
Matthew Meszaros e1e02c5c18 feat: add campaign send controls backend 2026-06-04 08:53:23 +02:00
Matthew Meszaros 1afe453eb0 feat: add deliverability controls
Add seed inbox-placement testing with admin management, placement result polling, and seed mailbox persistence.

Add pre-send email verification, invalid-recipient skipping, warmup-health campaign gating, and RFC 8058 one-click unsubscribe headers/endpoints.
2026-06-03 16:47:53 +02:00
Matthew Meszaros b0b712a43f feat: refine warmup content admin
Treat warmup content as a shared library in settings and admin tables, and realign the conversation pick index with segment-based selection.

Add generation job details and fix warmup placement displays so backend percentages and provider counts render correctly.
2026-06-03 11:32:09 +02:00
Matthew Meszaros 2009780beb feat: add warmup batch generation
Add OpenAI Batch API support for warmup content generation, including job metadata, polling, cancellation, and completed-batch ingestion.

Share the generation prompt between sync and batch modes and humanize/lint generated threads before storing them in the warmup content library.
2026-06-03 11:14:33 +02:00
Matthew Meszaros 35fbf73347 feat: persist warmup engagement dwell
Store delayed warmup engagement actions in Postgres and drain them from the consumer so read, important, and star actions survive worker restarts.

Keep foldering and spam rescue immediate while routing delayed actions to the mailbox's current worker at fire time.
2026-06-03 11:14:02 +02:00
Matthew Meszaros de7e28d294 feat: add campaign deliverability breaker
Track complaint events in campaign progress and evaluate rolling bounce and complaint rates before auto-pausing campaigns.

Emit an early warning webhook when campaign deliverability enters the warning band.
2026-06-03 06:26:56 +02:00
Matthew Meszaros 47bdf66899 feat: improve warmup health analytics
Record warmup spam placements by recipient provider and surface provider breakdowns in the admin health summary.

Track warmup replies and use stable static conversation ids so warmup analytics can correlate content reliably.
2026-06-03 06:26:39 +02:00
Matthew Meszaros 15ef9d4994 feat: add warmup content controls
Add warmup content generation and admin review surfaces, plus mailbox warmup appeal/status APIs.

Track warmup engagement and tampering signals so unsafe mailboxes can be handled by the warmup flow.
2026-06-03 05:05:53 +02:00
Matthew Meszaros 2430095197 feat: move infrastructure state off dynamodb
Remove DynamoDB-backed storage paths, add Postgres/HTTP repositories for mailbox state maps, wire the internal message-map API, and add provisioning runner/migration plumbing.
2026-06-02 15:54:12 +02:00
Matthew Meszaros 559d05a458 feat: use dot-separated event-bus topic names for Kafka compatibility 2026-06-01 12:49:24 +02:00
Matthew Meszaros d92e7f1b42 feat: squash migrations into a single baseline schema 2026-06-01 12:35:41 +02:00
Matthew Meszaros 83eae84d49 feat: merge main into warmup branch
Resolve the main-branch conflict in realtime event publishing by keeping both warmup account-health and audit-created events. Renumber the warmup migrations after the current main migration tail.
2026-06-01 11:36:04 +02:00
Matthew Meszaros a04d7450bc feat: guard warmup reconciliation by access
Make the warmup task reconciler re-check account state and org warmup access before scheduling replacement tasks. Remove pool membership for accounts whose org can no longer use warmup to avoid repeated no-access task churn.
2026-06-01 11:19:44 +02:00
Matthew Meszaros fc18b8e470 Merge pull request #23 from warmbly/feature/passkey-login-experience-2 2026-06-01 09:15:21 +02:00
Matthew Meszaros 218af13dd9 feat: add session management
Add authenticated session listing and revocation APIs, track the auth provider on sessions, and expose active session controls in account security settings.
2026-06-01 05:53:49 +02:00
Matthew Meszaros e237986f46 feat: merge main into integrations oauth 2026-06-01 04:49:50 +02:00
Matthew Meszaros 4500aeba0f feat: add integration oauth connections
Adds OAuth-backed integration connection management across the API, repository, event dispatch, migrations, docs, and dashboard UI.

Includes realtime invalidation and small dashboard type compatibility fixes needed for the web typecheck gate.
2026-06-01 04:25:14 +02:00
Matthew Meszaros e0ced97ba1 feat: add organization audit trail
Move audit logs into org-scoped Postgres storage, wire audit events across backend handlers, and surface actor details in the dashboard activity log.

Add realtime audit invalidation and retention pruning so the trail stays current and bounded.
2026-06-01 04:23:00 +02:00
Matthew Meszaros 87f9bf58b0 feat: add Safari-ready passkey login
Add passkey enrollment and login wiring, including a Safari-safe explicit login path that prefetches the WebAuthn challenge before the click and calls the credential ceremony immediately from the user gesture.
2026-06-01 03:09:17 +02:00
Matthew Meszaros 0cb53a25a6 feat: improve warmup pool scheduling
Add explicit warmup participant roles so connected accounts can provide recipient-only pool capacity without becoming senders. Cap warmup scheduling by eligible recipients, avoid same-day recipient reuse, reschedule exhausted pools, and reconcile missing warmup task chains.
2026-05-31 18:07:13 +02:00
Matthew Meszaros 7b5a87d1d1 feat: add unibox scheduled send backend
Add Unibox overview, snooze, and scheduled-send endpoints with task repository support, queue caps, execution-time guards, and snooze persistence.
2026-05-31 04:26:26 +00:00
Matthew Meszaros 0740a8bbbe feat: fix local migrations and kafka topics 2026-05-30 14:11:47 +00:00
Matthew Meszaros d08c984c42 feat: snapshot current dashboard changes 2026-05-30 13:56:27 +00:00
Matthew Meszaros 44c837d8ac Merge branch 'main' into feature/dashboard-plans
# Conflicts:
#	AGENTS.md
2026-05-30 09:49:12 +00:00
Matthew Meszaros cb0c2b4fac feat: harden webhook endpoints 2026-05-30 04:33:43 +00:00
Matthew Meszaros 640da62b32 feat: add api idempotency keys 2026-05-30 04:31:43 +00:00
Matthew Meszaros 60773b3d8e feat: make dashboard realtime 2026-05-30 04:17:10 +00:00
Matthew Meszaros ff763c4483 feat: add role and team-size onboarding questions and rework site SEO me 2026-05-29 14:47:16 +00:00
Matthew Meszaros 6766031cc5 feat: add discount code support for checkout and plan changes 2026-05-29 05:49:19 +00:00
Matthew Meszaros e7ef328b2d Merge pull request #18 from warmbly/feature/admin-management
feat: admin management surface with overrides, ban scope, and throttles
2026-05-29 05:01:00 +02:00
Matt bd6a045751 feat(admin): outreach composer (platform mailer + reply-to + audit log)
Adds a dedicated admin path for sending platform email — distinct from
the campaign emailsend service (which sends through customer mailboxes)
so the two abuse surfaces never share code paths.

Schema (000047) adds admin_outreach_messages: every send is recorded
with sent_by, the resolved to_email, the optional reply_to, subject,
body, and a queued → sent/failed status. Failed sends keep their error
column populated for the audit log.

Extends notify.EmailNotificationService with SendOutreach so both
backends (SES + SMTP) support custom Reply-To: SES via the native
ReplyToAddresses field, SMTP via a forged Reply-To header. The
existing transactional Send() remains unchanged so no other caller is
affected.

Service (internal/app/adminoutreach) resolves recipients three ways:
to_email (raw address), to_user_id (sends to the user's account email),
or to_org_id (sends to the workspace owner). Persist-then-send-then-
mark ensures the audit row exists even if the mailer hangs, and
mark-failed captures the error string verbatim.

Routes:
  POST /admin/outreach            manage_organizations
  GET  /admin/outreach            view_organizations

Admin UI: composer with recipient mode picker (email / user_id / org_id),
configurable Reply-To (defaults to support@warmbly.com so customers can
actually reply), subject + HTML body editor, and an outreach log below
showing the last 50 sends with status badges and error details. Sidebar
entry under Accounts (Send icon).
2026-05-28 12:25:18 +02:00
Matt e6753f7c24 feat(limits): limit-increase request workflow (backend)
Wire the customer self-serve path for asking "please give me more
mailboxes / campaigns / contacts." Migration 000046 adds
limit_increase_requests with a partial unique index ensuring only one
pending request per (org, field) so the queue can't be spammed, plus
a CHECK requiring requested > current_effective so no-op rows never
reach an admin.

Service layer:
  - SubmitLimitIncreaseRequest validates membership, rejects unknown
    fields, snapshots the user's current effective limit at submission
    time so the queue row carries the context the admin needs.
  - CancelLimitRequest lets the original submitter walk back a pending
    request; approved/rejected rows are immutable as the audit record.
  - ApproveLimitRequest stamps the row and writes the corresponding
    column on organization_limit_overrides via SetLimitOverrides —
    same write path direct admin overrides use, so granted_by and
    notes carry through and the audit log treats both flows uniformly.
  - RejectLimitRequest stamps the row with required review notes.

Routes:
  POST   /v1/organization/:orgId/limit-requests
  GET    /v1/organization/:orgId/limit-requests
  DELETE /v1/limit-requests/:id                 (submitter only)
  GET    /admin/limit-requests?status=pending
  POST   /admin/limit-requests/:id/approve
  POST   /admin/limit-requests/:id/reject

Admin approval and rejection both fire admin audit log entries with
field + requested + notes so the decision history survives any future
reorg of the request table.

UI (admin queue page + dashboard request form) plus the ToS clause
giving Warmbly the right to refuse any increase land in the next commit.
2026-05-28 12:05:32 +02:00
Matt 7de29b0fb0 feat(admin): ban scope bitmask (schema + UI; enforcement is staged)
Add users.ban_scope INT NOT NULL DEFAULT 0 in migration 000045 so admins
can describe what a ban concretely stops (login / workspace creation /
outbound send) instead of relying on a single boolean banned_at flag
that meant "everything".

Wire flags in the BanScope enum (kept in sync with the migration) plus
a CHECK constraint guaranteeing non-negative values. Existing bans
backfill to BanScopeLogin so the historical "you can't log in"
semantics is preserved exactly — no behaviour changes silently at
deployment.

BanUserRequest gains an optional scope field, BanUser threads it through
the service to the repo write, and the UserBanDialog grows a checkbox
group with one option per flag. Reason still required; at least one
scope must be picked. Audit details now include the scope bitmask.

Runtime enforcement (refusing login when BanScopeLogin is set, etc.) is
intentionally separate from this commit — the existing codebase doesn't
yet have an active ban check anywhere, so wiring that lives across the
auth middleware, org-create handler, and emailsend service. This slice
ships the schema, the audit story, and the UI vocabulary so the
enforcement PR can land without database churn.
2026-05-28 10:00:29 +02:00
Matt 8d3b6b5d05 refactor(integration): drop Postmaster, SNDS, DMARC, DNS providers
The deliverability-data and DNS-write integrations were over-engineered
for the cold-email segment. Postmaster and SNDS require sending volume
our base typically does not hit, and no comparable cold-email tool
exposes DMARC ingestion or native DNS writes. Replaces the catalog with
the standard set: HubSpot, Salesforce, Pipedrive, Close, Zapier, Make,
n8n, Slack, Discord, Calendly, Cal.com, Google Sheets.

Removes dmarc_reports, dmarc_record_rows, postmaster_snapshots, and
dns_verifications tables from the migration. Deletes dmarc.go, dns.go,
cloudflare.go, postmaster.go from the integration package. Prunes the
matching repository methods and HTTP handlers.
2026-05-28 08:53:39 +02:00
Matt 11f1563e08 feat(admin): organization permission bits + limit overrides migration
Add two new admin permission bits — view_organizations (bit 20) and
manage_organizations (bit 21) — and switch the existing read-only
/admin/organizations routes off the borrowed AdminPermViewUsers bit they
were using as a placeholder. Backfill the three predefined roles
(support, ops, analyst) with view_organizations so existing role
mappings still resolve cleanly; super continues to pick up everything
via AllAdminPermissions.

Migration 000044 adds organization_limit_overrides, the table the next
commit's write path will target. Schema follows the "0 = inherit from
plan" convention from the design discussion: each numeric column
defaults to 0 and a CHECK constraint enforces non-negative values, so
reverting an override is a write of 0 (preserving the granted_by audit
trail) rather than a DELETE.

Bumping the permission count changes every role's numeric bitmask, so
update the make grant-admin role table to match — super is now
4194303, support 1086401, ops 1062960, analyst 1055233.
2026-05-28 08:47:57 +02:00
Matt 20f1e93c4b feat(integration): backend foundation for tier 1+2 integrations
Adds an integrations app module covering the providers from the tier 1/2
plan: Calendly, Cal.com, Google Sheets, Google Postmaster, Microsoft SNDS,
DMARC ingestion, and Cloudflare/GoDaddy/Namecheap DNS. One unified
migration provisions integration_connections, dmarc_reports + record
rows, postmaster_snapshots, dns_verifications, and meeting_bookings.

The service exposes a generic CRUD surface for connection state with
per-provider files for parsing (calendly.go, dmarc.go), HTTP clients
(cloudflare.go, postmaster.go, google_sheets.go), and DNS verification
(dns.go). Inbound webhook routes use per-org URL-embedded secrets so
Calendly/Cal.com/DMARC providers post directly without Warmbly auth.
DNS verifier resolves SPF/DKIM/DMARC + tracking CNAME and surfaces
fixes when a record is missing.
2026-05-28 08:19:10 +02:00
Matthew Meszaros d2414ad29f ci(go): gofmt all flagged files
Go CI fails on golangci-lint's gofmt check. Ran gofmt -w against
every file the linter named plus a handful of others that drifted
during the autonomous-fleet work. No semantic changes — alignment
of struct field whitespace and one mis-indented import block.

gofmt -l ./... is now empty; go build + go vet are clean.
2026-05-27 16:40:23 +00:00
Matthew Meszaros 8347237547 merge: resolve main into feature/workers-support
Brings in PR #15 (email warmup process 4) plus its preceding commits:
customer-defined warmup routing on premium pool, free-trial warmup +
1 inbox for 14 days, customer webhook subscriptions with HMAC signing
+ retry, bumped default API rate limits to 100 req/s with flat per-
user/per-plan caps, plus dev-fixture additions.

One real conflict: internal/client/smtpimap/imap/client.go added
distinct imports on each side (this branch added 'net' for the
*net.TCPAddr BindIP field; main added 'sync' for a Mutex). Kept both.

Everything else auto-merged additively:
  cmd/backend/main.go     - imports + handler fields + DI lines
  internal/api/handler/handler.go - new fields next to existing ones
  internal/api/routes.go  - new route group next to existing ones

Full build + test suite pass (no regressions).
2026-05-27 16:29:18 +00:00
Matthew Meszaros 73cabf6f5e worker: WorkerHealth event, capacity view, smarter SelectSharedWorker
Workers emit a WorkerHealth event every 30s with assigned mailbox count,
IMAP IDLE connections, memory, goroutines, and rolling 1h send/bounce/
complaint/auth-error/rate-limit counters. Consumer writes them to
worker_health_samples.

Schema additions on workers: egress_kind (cold_smtp / oauth_api /
warmup_only), health_state (healthy / watch / throttled / quarantined /
blocked), load_score (weighted utilization).

worker_capacity_view aggregates the latest hour of samples into a
per-worker capacity row used by the assignment loop. Effective capacity
= base_ceiling(egress_kind) × health_multiplier × age_ramp_multiplier
so a fresh worker earns its way up to base capacity over 72h, and a
worker with rising bounces or complaints automatically gets less load.

MailboxWeight returns 1.0 for cold_smtp, 0.05 for Gmail/Graph API
(worker IP doesn't matter), 0.4 for warmup-only. AssignWorkerToEmail
resolves the mailbox's weight and SelectSharedWorker filters by
headroom + sorts by utilization, so a 200-mailbox OAuth worker and a
16-mailbox cold worker balance fairly.

UnassignWorkerFromEmail refunds the load_score symmetrically.
2026-05-27 15:56:01 +00:00
Matthew Meszaros a829b41c88 infra(cloudprovider): pluggable cloud-VPS abstraction + Hetzner Cloud impl
cloudprovider.Provider interface (Locations, ServerTypes, Images,
Verify, CreateServer/DeleteServer, CreatePrimaryIP/AssignPrimaryIP/
UnassignPrimaryIP/DeletePrimaryIP/SetReverseDNS). One impl today
(Hetzner Cloud); adding OVH or Vultr later means implementing the same
six surfaces.

hetzner.Client is a minimal idiomatic Go REST client over
https://api.hetzner.cloud/v1. Bearer-token auth. Returns provider-
native IDs as strings so the orchestration layer can persist them for
rollback.

9 tests against httptest.Server covering token transmission, error
surfacing, parsing, request-body shape, and interface conformance.
2026-05-27 15:54:36 +00:00
Matthew Meszaros fdf79a07a9 schema: cloud_credentials, worker_profiles, provisioning_templates, provisioning_jobs, provisioning_policy, decision_log
Foundation for autonomous fleet management.

cloud_credentials stores encrypted API tokens per cloud provider.
worker_profiles bundles the env vars that get rendered into
/etc/warmbly/worker.env at install time.
provisioning_templates is a customizable saved config — every Hetzner
option the admin form exposes lives here, so the cheapest-US-single-IP
setup is a one-click pick once you've saved it.
provisioning_jobs is the state machine (pending -> creating_server ->
creating_ips -> assigning_ips -> setting_rdns -> installing ->
verifying -> completed | failed -> rolling_back).
provisioning_policy is per-provider budget caps + the auto_provision
toggle the scale loop checks.
decision_log records every automated action so admins can audit what
the system did and why.
2026-05-27 15:54:21 +00:00
Matthew Meszaros e0e4a010a0 admin: storage_backends registry + settings/dek/worker-config endpoints
New storage_backends table is the runtime inventory of pluggable
infrastructure choices (KMS, encrypted_keys, blob, eventbus, cache).
Each kind has exactly one active row, enforced via a partial unique
index. Read-only rows are env-var driven; UI-mutable rows can be
flipped via SetActive.

settings.Registrar reflects boot-time backend choices into the table
so the admin UI sees what's actually running.

New admin endpoints under /admin/settings/backends:
  GET    /settings/backends?kind=...
  GET    /settings/backends/active/:kind
  POST   /settings/backends/:id/activate

New internal endpoints under /api/v1/internal:
  GET  /worker/config       - workers fetch runtime config on boot
  POST /worker/heartbeat    - liveness ping
(DEK endpoints added in the encryptedkeys commit.)

handler.Handler grows EncryptedKeys + StorageBackendRepo fields.

5 registrar tests cover create / update-and-activate / skip-when-active /
lookup-error propagation / RegisterAll stop-on-first-error using a
mock repository.
2026-05-27 14:43:36 +00:00
Matthew Meszaros 4080258606 infra(codec): transport-agnostic Codec (Avro + JSON)
Codec interface (Serialize / Deserialize / Name) lets payload encoding
decouple from the transport choice. Two implementations:

  AvroCodec - wraps the existing kafka.Avrov2 Schema Registry client
              via NewAvroFromClient. Preserves identical wire format
              for production deployments already on Kafka + SR.
  JSONCodec - encoding/json based. No external dependency, suitable
              for self-hosters who don't want a Schema Registry.

Factory FromEnv selects via CODEC_PROVIDER (default avro).

Once codec.Codec is in the worker boot and publisher, self-hosters can
pick EVENTBUS_PROVIDER=nats CODEC_PROVIDER=json for a Schema-Registry-
free deployment.

11 tests cover JSON round-trip, nil guards, factory paths, and Avro
interface conformance.
2026-05-27 14:42:47 +00:00