Matthew Meszaros 93e8451738 feat: organization data export and import for moving a workspace between instances (#132)
* feat: add the org_export_jobs and org_import_jobs tables plus the models behind them, so a whole organization can be written to a portable archive and read back on another instance, keeping the option columns typed (a text[] of data groups, an include_secrets boolean, a conflict_strategy check constraint) rather than a settings blob because the option set is small and fixed, and reserving jsonb only for the genuinely free-form parts that are read back for display alone (the source archive's manifest, per-table row counts, the import warning list), with partial indexes on the in-flight and expiring rows so the maintenance sweep stays cheap however much transfer history accumulates, an OrgDataGroup catalog that names the twelve slices of a workspace and carries the dependencies between them, and an org_archive audit entity so an export or import rides the existing audit spine into every teammate's dashboard

* feat: add the schema-generic repository behind workspace archives, which reads and writes tables by name rather than through typed structs because that is the only way an archive stays correct as the schema grows, moving rows as jsonb in both directions via to_jsonb on the way out and jsonb_populate_recordset on the way in so Postgres performs every type conversion and no hand-written Go column mapping can drift from arrays, jsonb, tsvector, inet or enums, lifting the pool's 60s statement_timeout inside the export transaction because a full inbox read legitimately runs longer than that, introspecting generated, identity and not-null columns plus primary keys and foreign keys from the catalog rather than trusting a compiled list, and treating identifier safety as structural: table names come from the compiled registry and column names are always intersected against the destination catalog before reaching a query, so nothing out of an uploaded archive is ever interpolated

* feat: add the workspace archive registry and on-disk format, covering all 110 organization-owned relations with their scope SQL, dependency order and per-table policy, plus 10 explicitly excluded ones each carrying the reason it must never travel (the KMS-wrapped org data key, in-flight OAuth handshakes, the websocket outbox, live sessions, a pending deletion that would otherwise schedule the destination workspace for destruction), naming the two key domains separately because Warmbly seals mailbox credentials under the instance CREDENTIALS_ENCRYPTION_KEY and everything else under the per-organization DEK and confusing them produces mailboxes that authenticate against nothing, defining the archive as a plain zip of newline-delimited JSON so an operator can unzip it and read the data in a text editor and so the manifest can be written last yet still be read first, and sealing archive secrets under an argon2id passphrase key with parameters deliberately heavier than the login hash since it is derived once per archive and guards every credential in the workspace against offline grinding

* feat: implement the workspace export and import engines, streaming rows straight through untouched for the tables that have neither secrets nor blobs so a million-row inbox export stays cheap and only decoding the rows that must change, opening every sealed value against whichever key domain wrote it and re-sealing it under the archive passphrase on the way out then against the destination's own keys on the way in, blanking a credential rather than sinking the whole export when one mailbox cannot be read and clearing the guard flag alongside it so no row is left claiming ciphertext it no longer holds, applying an import inside a single transaction because a half-applied workspace is far worse than a long-running one, rewriting the organization id and matching members to destination accounts by email with unresolvable people blanked where the column is nullable and redirected to the importer where it is not, and running transfers in the accepting process rather than through a queue for the one reason that matters: the passphrase is then never written down anywhere

* feat: make the per-organization DEK cache nil-safe in internal/app/cipher so a process built without Redis falls through to KMS on every call instead of dereferencing a nil cache handle, which is what lets warmblyctl run the workspace export and import commands at all: it deliberately attaches Redis as optional because the whole point of that CLI is working while the rest of the instance is down, and the decrypted-key cache was always an optimisation rather than a requirement

* feat: add the hourly workspace-archive maintenance job that deletes finished archives past their seven-day retention window, since each one is a complete copy of a workspace sitting in object storage and must not accumulate, and closes out any export or import whose process died mid-run, which is the necessary counterpart to executing transfers in the accepting process so the passphrase is never persisted: without this sweep a restart would leave a job reporting running forever

* feat: expose workspace export and import over the JWT-only organization routes and wire the service into the backend, gating every endpoint on workspace ownership through the existing requireOrgOwner check rather than a permission bit because an export with credentials is the single most sensitive artifact this product can produce and an import rewrites the workspace wholesale, so both belong at the same level as deleting it, spooling uploads to a temporary file since a zip needs random access and a length that a multi-gigabyte archive cannot supply from memory, handing that file's ownership to the background import so it outlives the request and is closed exactly when the job ends, streaming downloads with the archive's sha256 in a response header, and constructing the service with both key domains plus object storage so an archive can be opened, re-keyed and stored

* feat: add warmblyctl org list, export and import so a self-hoster can move a workspace from the box without a browser, running the same engine in-process against Postgres and adding no HTTP surface to a CLI whose entire trust model is container or host access, resolving --org from whichever handle the operator has (id, slug, or the owner's email), streaming the archive to a file or to stdout so it can be piped straight into ssh with progress still readable on stderr, prompting for the credential passphrase twice through the existing password prompt so the terminal and pipe rules stay identical across every command, and defaulting the import path to a preflight report that names what already exists here and which members have no account before anything is written, with --dry-run to stop there

* feat: add the dashboard API layer for workspace archives, fetching the data-group catalog from the server rather than restating it in the client so a new group appears the moment the backend knows about it, mirroring the server's group-dependency closure in expandGroups so the toggles a user sees always match what the archive actually gets, polling only while a transfer is in flight and dropping to no interval the moment none are active since a running job has no realtime event of its own, and downloading a finished archive as a blob through the authenticated client because the endpoint is bearer-authenticated and a plain anchor href cannot carry the token

* feat: build the Settings and Data dashboard page for exporting and importing a workspace, following the settings section conventions and the in-app confirm rather than window.confirm, defaulting the export to every data group because a migration that quietly leaves data behind is worse than one that takes a while, marking the heavy groups so nobody exports a decade of inbox history unaware, requiring the credential passphrase twice behind a confirm that states plainly what the file will contain, and making the import a two-step flow where a preflight reads the archive and reports its origin, row counts, unsealable credentials, existing rows and unknown members before a single byte is written, so confirming is never a leap of faith

* feat: register the Data settings section in the dashboard rail, route and realtime spine, placing it under Advanced beside the danger zone and gating it to the workspace owner so the nav matches what the endpoints actually allow, and mapping the new org_archive audit entity to the export and import query keys in useRealtimeEvents so an archive starting or landing refreshes the page for every teammate through the existing audit spine rather than a bespoke event

* feat: document workspace export and import as a customer guide registered under Account and team, covering what each of the twelve data groups contains and which four dominate archive size, why credentials need a passphrase to travel at all and what happens to mailboxes when they do not, how members are matched to destination accounts by email and what becomes of anyone without one, the difference between keeping existing rows and replacing them, and a table of what deliberately does not import with the reason for each, because billing, plan overrides, worker placement, sync checkpoints and warmup pool membership belong to an instance rather than to a workspace

* feat: document org list, export and import in the warmblyctl reference and point the deployment guide at them as the supported route between a self-hosted install and the hosted service in either direction, adding every flag with what it does, the two extra environment variables those commands read and the difference between them (a missing KMS provider stops the command because sealed values cannot be opened, while a missing CREDENTIALS_ENCRYPTION_KEY is only a warning that mailbox credentials will not move), the behaviour when Redis is down, and the warning that an archive carrying credentials is the most sensitive file this product produces

* feat: record in AGENTS.md that a migration adding an organization-scoped table is not finished until that table is registered in internal/app/orgtransfer/spec.go, either in Tables with its group and scope or in ExcludedTables with the reason it must not travel, because data left out of the registry is silently absent from every archive and nobody discovers it until a customer's migration lands on the other side missing a feature's data, and spelling out the four things that are easy to get wrong when adding one: dependency order, the group boundary that needs a Requires entry only when a NOT NULL foreign key crosses it, which of the two key domains seals a ciphertext column, and which columns name something only the source instance knows
2026-08-18 07:53:39 -07:00
feat: make self-hosted onboarding survivable by fixing invite_only, which could not onboard anyone (the accept route is JWT-only, so redeeming the invitation that would create your account required already having one, making the self-host default silently identical to fully closed), threading the invitation token through registration so an invited person lands in the inviting organization instead of a stray workspace, gating SSO just-in-time provisioning behind DISABLE_REGISTRATION (it bypassed the gate entirely, so an instance set to true was still open to anyone the IdP would assert) with SSO_AUTO_PROVISION as the opt-out, correcting the OIDC redirect URL that pointed at /api/v1 against a route at /v1 and 404'd every SSO login, scoping the first-launch exemption so it no longer overrides an explicit lockdown, preserving the remaining TTL when restoring a losing setup token so a public endpoint cannot hold the claim window open forever, replacing a generic 403 with typed registration_invite_only, registration_closed, invitation_invalid, setup_token_invalid and setup_already_complete codes that name the next step, logging why no claim link was issued on an already-claimed instance instead of staying silent, adding a warmblyctl operator CLI (status with health checks and a non-zero exit, reissuable setup-link, user create/list/reset-password/grant-admin/revoke-admin/disable-2fa, hash-password) so a locked-out operator no longer needs hand-written psql, adding read-only instance configuration over 104 environment variables with structural secret redaction and fingerprints, 35 health checks, a database-backed settings tier for the three keys no environment variable owns, hiding the signup form when the config already says invite_only rather than failing the whole form with a toast, and documenting first run, accounts and access, configuration, instance health and troubleshooting alongside the root .env.example the README told operators to write but never shipped (#114)
2026-08-16 05:58:11 +02:00
2026-08-16 07:54:45 +02:00
feat: make self-hosted onboarding survivable by fixing invite_only, which could not onboard anyone (the accept route is JWT-only, so redeeming the invitation that would create your account required already having one, making the self-host default silently identical to fully closed), threading the invitation token through registration so an invited person lands in the inviting organization instead of a stray workspace, gating SSO just-in-time provisioning behind DISABLE_REGISTRATION (it bypassed the gate entirely, so an instance set to true was still open to anyone the IdP would assert) with SSO_AUTO_PROVISION as the opt-out, correcting the OIDC redirect URL that pointed at /api/v1 against a route at /v1 and 404'd every SSO login, scoping the first-launch exemption so it no longer overrides an explicit lockdown, preserving the remaining TTL when restoring a losing setup token so a public endpoint cannot hold the claim window open forever, replacing a generic 403 with typed registration_invite_only, registration_closed, invitation_invalid, setup_token_invalid and setup_already_complete codes that name the next step, logging why no claim link was issued on an already-claimed instance instead of staying silent, adding a warmblyctl operator CLI (status with health checks and a non-zero exit, reissuable setup-link, user create/list/reset-password/grant-admin/revoke-admin/disable-2fa, hash-password) so a locked-out operator no longer needs hand-written psql, adding read-only instance configuration over 104 environment variables with structural secret redaction and fingerprints, 35 health checks, a database-backed settings tier for the three keys no environment variable owns, hiding the signup form when the config already says invite_only rather than failing the whole form with a toast, and documenting first run, accounts and access, configuration, instance health and troubleshooting alongside the root .env.example the README told operators to write but never shipped (#114)
2026-08-16 05:58:11 +02:00
2026-08-16 07:54:45 +02:00
2026-08-16 07:54:45 +02:00
2026-01-17 09:19:43 +00:00
feat: make self-hosted onboarding survivable by fixing invite_only, which could not onboard anyone (the accept route is JWT-only, so redeeming the invitation that would create your account required already having one, making the self-host default silently identical to fully closed), threading the invitation token through registration so an invited person lands in the inviting organization instead of a stray workspace, gating SSO just-in-time provisioning behind DISABLE_REGISTRATION (it bypassed the gate entirely, so an instance set to true was still open to anyone the IdP would assert) with SSO_AUTO_PROVISION as the opt-out, correcting the OIDC redirect URL that pointed at /api/v1 against a route at /v1 and 404'd every SSO login, scoping the first-launch exemption so it no longer overrides an explicit lockdown, preserving the remaining TTL when restoring a losing setup token so a public endpoint cannot hold the claim window open forever, replacing a generic 403 with typed registration_invite_only, registration_closed, invitation_invalid, setup_token_invalid and setup_already_complete codes that name the next step, logging why no claim link was issued on an already-claimed instance instead of staying silent, adding a warmblyctl operator CLI (status with health checks and a non-zero exit, reissuable setup-link, user create/list/reset-password/grant-admin/revoke-admin/disable-2fa, hash-password) so a locked-out operator no longer needs hand-written psql, adding read-only instance configuration over 104 environment variables with structural secret redaction and fingerprints, 35 health checks, a database-backed settings tier for the three keys no environment variable owns, hiding the signup form when the config already says invite_only rather than failing the whole form with a toast, and documenting first run, accounts and access, configuration, instance health and troubleshooting alongside the root .env.example the README told operators to write but never shipped (#114)
2026-08-16 05:58:11 +02:00
feat: add a warmblyctl reference page at docs/content/docs/development/warmblyctl.mdx documenting all nine commands with every flag, because the README and the recovery sections only ever showed 'warmblyctl user create --email ... --admin' without saying where the password comes from, leaving self-hosters with an account they could not sign in to and no page that answered it, covering that user create prompts for the password twice on a terminal and refuses on a non-TTY unless --password-stdin is passed, that docker compose exec allocates the TTY those prompts need unless -T is given and piped input needs -T precisely because it removes it, that the password rule is the dashboard's own 8 to 128 characters, and that signing in afterwards needs nothing else on a stock self-host since AUTH_LOGIN_CODE defaults to off and REQUIRE_EMAIL_VERIFICATION to false when self-hosted, captcha stays off without TURNSTILE_SECRET, and --admin opens the panel on ADMIN_URL rather than APP_URL, plus the status JSON contract and exit codes, the four environment variables each command reads, the per-command behaviour when Redis is down, the admin role masks read from AdminRolePermissions, and the make wrappers, registering it in the Development meta.json between accounts-and-access and configuration, linking it from the four pages that already print these commands, correcting the super-admin mask in the first-run sample output from 4294967295 to the 4194303 that AllAdminPermissions actually is since it is (1 << 22) - 1 and not the full uint32 range, and shortening the README self-hosting section by folding the three-bullet gotcha list into a four-row table that also names make doctor and dropping the duplicated make dev claim warning already stated above it, keeping every fact (#128)
2026-08-16 08:39:57 +02:00

Warmbly

The open-source agentic cold email and warmup platform.

Discord Follow @WarmblyHQ on X Docs CI status Latest release License

Features · How it works · Quick start · Self-hosting · Docs · Community · Support

Help us reach more senders and grow the Warmbly community. Star this repo!

Warmbly

Warmbly runs cold email campaigns from the mailboxes you already own and warms them so they keep landing in the inbox. Opens, clicks, and replies land in a shared dashboard the moment they happen, and it's AI-native, so your team and its agents work in it together, live.

https://github.com/user-attachments/assets/378a510a-bb99-425f-925e-04300184938b

Features

  • Campaigns - multi-step sequences with per-mailbox caps and spacing
  • Unified inbox - every mailbox and reply in one place
  • CRM - contacts, pipelines, deals, tasks, meetings
  • Warmup - a pool of monitored mailboxes, not throwaway accounts
  • Deliverability - bounces, complaints, suppression, inbox placement
  • Automations - visual reply playbooks with AI steps
  • Integrations - HubSpot, Slack, Zapier, REST API, webhooks
  • Realtime - live presence and edits across your team

Campaigns Unified inbox

How it works

Warmbly splits into a control plane (backend API, consumer, Postgres, Redis, and the event bus) that owns all state, and an execution plane of interchangeable Go workers that send and sync mail. Workers never touch Postgres, and outbound mail leaves through each mailbox's own provider, not the worker's IP, so you add throughput by running more workers.

flowchart LR
  MB["Your mailboxes"] --> API
  subgraph CP["Control plane"]
    direction TB
    API["Backend API"] --> DB[("Postgres")]
    API --> BUS{{"Event bus"}}
  end
  BUS --> W1["Worker"]
  BUS --> W2["Worker"]
  BUS --> W3["Worker"]
  W1 --> P["Gmail · Microsoft · SMTP"]
  W2 --> P
  W3 --> P
  P --> R["Recipients"]

Secrets use envelope encryption, with a local AES master key by default or AWS KMS if you prefer. Full write-up in the architecture docs.

Quick start

You need Docker, Go 1.25, and pnpm.

git clone https://github.com/warmbly/warmbly && cd warmbly
make dev

One command brings up the backing services in Docker, applies migrations, seeds demo data, and starts the backend, worker, and dashboard natively. Open http://localhost:5173 and log in with dev@warmbly.com / password123, then read the login code out of Mailpit at http://localhost:18025 (the native dev stack keeps the emailed code on so the flow stays exercised; a self-hosted install does not). Full setup lives in the local development guide.

Warning

make dev and make up share one Docker Compose project, one volume, and one warmbly_dev database, and make dev seeds fixture accounts by default. Those accounts become your instance's accounts, which permanently retires the first-run claim link make up prints. Use make dev SEED=false on a database you intend to self-host from. See first run.

Self-hosting

Runs on Docker Compose

Warmbly runs with no cloud account of any kind: no AWS, no GCP, no Stripe, no Kafka. One command brings up the whole platform on local, open-source pieces:

git clone https://github.com/warmbly/warmbly && cd warmbly
make up

That is the whole install. make up waits for the backend and prints a one-time link that claims the instance and makes you its admin. Open it, pick a password, and you are in. You need Docker with Compose v2 and about 10 GB of free disk; the first run builds the images once, which takes roughly 6 minutes.

Nothing else is required: no SMTP relay, no captcha keys, no cloud account, no .env to hand-write, and no separate command to grant yourself admin. To change something, cp .env.example .env and edit it. The template boots as-is and carries the commands that generate your own secrets; set those and APP_ENV=prod before anyone else can reach the instance.

If Then
The claim link is gone It is single use and lasts 24 hours. make claim prints a fresh one
No link was printed at all The database already has accounts, so the instance is claimed. make cli ARGS="user create --email you@example.com --admin" adds you, prompting for a password to set
You would rather skip the link Set WARMBLY_BOOTSTRAP_EMAIL and WARMBLY_BOOTSTRAP_PASSWORD_HASH before the first start and the owner exists when it comes up
Something is wrong make doctor prints the instance state and every failing check

Those all run warmblyctl, the operator CLI baked into the backend image. It talks to the database directly, so it works when signing in does not.

Note

Signing in never depends on outbound mail. Platform email defaults to MAIL_TRANSPORT=log under Compose, so password resets and invitations go to the backend logs until you point SMTP_* at a relay. Invitations still work without one: invite the person under Settings > Members, then copy the link from their row and send it yourself. See accounts and access.

➡️ Follow the step-by-step self-hosting guide for the full walkthrough: your own secrets, verifying the stack is healthy, mail and single sign-on, HTTPS, connecting Gmail and Microsoft mailboxes, scaling workers, backups, and a troubleshooting table.

Every external dependency is picked by an environment variable, so you swap in a cloud service only if you want one:

Concern Self-host default Optional / cloud
Database PostgreSQL 16 RDS / Cloud SQL, any Postgres
Cache Redis (or Valkey) ElastiCache
Event bus NATS JetStream Kafka (-tags kafka)
Blob storage Filesystem S3, MinIO, R2, B2
KMS / root key Local AES master key AWS KMS
Payments Off (everything unlocked) Stripe

Scaling is by mailboxes and workers, not IPs.

Documentation

The full docs live at docs.warmbly.com.

Read this To learn
Self-hosting guide Step-by-step install, then production, backups, and scaling the worker fleet
First run Claiming the instance, reissuing the setup link, and what to do when accounts already exist
Accounts and access Registration modes, inviting people with or without a mail relay, SSO, and recovering access
warmblyctl The operator CLI: creating accounts, setting passwords, granting admin, and instance status
Configuration reference Every environment variable, its default, and whether changing it needs a restart
Instance health The checks the admin panel runs against your deployment, and make doctor
Troubleshooting The errors self-hosters actually hit, and the command that fixes each one
Local development Every make target, the native services, and how seeding works
Architecture How the control plane and workers split the job, plus the encryption model
API reference Endpoints, auth, permissions, and webhooks

Community

Have a question, found a bug, or want to shape where Warmbly goes next?

  • Discord - chat with the team and other senders
  • GitHub Issues - report bugs and request features
  • X / @WarmblyHQ - follow along for updates and releases
  • Email - reach us at team@warmbly.com

Support and enterprise

Note

Need a hand? We are happy to help. Ask in Discord, open a GitHub issue, or email team@warmbly.com, and someone on the team will get back to you.

Running Warmbly at scale, or would you rather we run it for you? We offer enterprise support and managed infrastructure: we can host and operate the whole platform for your organization, help you deploy and scale the worker fleet, tune deliverability, migrate your sending onto Warmbly, and stand behind it with a support agreement built around your team. Tell us what you need at team@warmbly.com or reach out on X.

Follow @WarmblyHQ on X Join the Discord Email the team

Star the repository

warmbly-star

Contributing

Pull requests are welcome. Keep each one to a single logical change, and open an issue first for larger design or product changes. Before you open a PR, run the checks for the tree you touched (make fmt and make lint for Go, pnpm typecheck and pnpm lint for the frontends). See CONTRIBUTING.md.

Security

Found a vulnerability? Email team@warmbly.com instead of opening a public issue. We prefer responsible disclosure and credit reporters in the release notes.

License

Apache License 2.0. Copyright 2026 Mindroot Ltd. See LICENSE.

Languages
Go 41.1%
TypeScript 36.6%
Swift 10%
Astro 9.4%
Elixir 0.9%
Other 1.9%