warmbly/docs/content/docs/development/warmblyctl.mdx

---
title: warmblyctl
description: Local operator recovery, remotely approved administrator diagnostics, and public API-key commands for a Warmbly instance.
---

<Callout type="info" title="There are two CLIs. This is the administrator's one.">
`warmblyctl` is for running an instance: accounts, health, recovery, backups. Local operator commands use the database inside the backend container. Remote administrator commands use a separately approved MFA session from your own machine, without database credentials.

If you want to use the product from your terminal (campaigns, contacts, mailboxes, the inbox), you want [`warmbly`](/api/cli/): a signed-in, multi-host client you install on your own machine, with `warmbly auth login` instead of an exported key.
</Callout>

`warmblyctl` is the CLI for a Warmbly instance, with three deliberately separate trust models.

The operator commands answer two questions: what state is this install in, and how do I get back in. They read and write the database directly, so they keep working when the sign-in page does not. Their authorization is container or host access, the same trust model as Sentry's `createuser`, Gitea's `admin user create` and authentik's `ak changepassword`, and it is the right one when the identity system is the thing that is broken.

The [API commands](#the-api-commands) drive a running instance over its public REST API with an API key, so they work from any machine against any instance you hold a key for, the hosted service included. They exist so scripts and AI agents can operate the product itself: campaigns, contacts, mailboxes, the inbox, settings. Everything they can do is bounded by the key's scopes.

The [remote administrator commands](#remote-administrator-commands) use an independent signed-in session approved on the existing administrator website. They require live platform permissions, session MFA, and the server's existing fresh-auth and audit guards. Platform administrator access does not imply organization membership, select a workspace, or issue an API key. These commands never fall back to local database authority or `WARMBLY_API_KEY`.

The CLI never serves HTTP. The API commands are a client of the already-gated public API, which adds no new surface to your instance.

Those API commands predate the [`warmbly` CLI](/api/cli/) and still work exactly as documented below. For day-to-day product work prefer `warmbly`: it signs in for you, holds a credential per host, prints tables rather than raw JSON, and runs on your laptop rather than inside a container.

## Running it

The binary ships inside the backend image at `/usr/local/bin/warmblyctl`, so it is on the path in every runtime. Running it inside the backend is the documented path because the environment there is already correct.

```bash
docker compose -p warmbly exec backend warmblyctl status   # docker compose
docker exec -it warmbly-backend warmblyctl status          # plain docker
kubectl exec -it deploy/warmbly-backend -- warmblyctl status
warmblyctl status                                          # bare binary
```

Local operator examples use the compose form. Remote administrator examples run the binary on your own machine.

For a compose install, `make cli` is the same thing with less typing. Flags go through `ARGS`, because make reads a bare `--email` as one of its own options:

```bash
make cli status
make cli setup-link
make cli ARGS="user create --email you@example.com --admin"
```

### Terminals and pipes

This is the one piece of shell mechanics the CLI cannot hide from you.

`docker compose exec` allocates a TTY unless you pass `-T`. Commands that prompt need that TTY. Commands you pipe into need it gone.

| You want to | Run |
|---|---|
| Be prompted for a password | `docker compose -p warmbly exec backend warmblyctl ...` |
| Pipe a password in | `printf '%s' 'your-password' \| docker compose -p warmbly exec -T backend warmblyctl ... --password-stdin` |

A command that would set a password refuses on a non-TTY unless you passed `--password-stdin`, rather than creating an account nobody can sign in to.

### What it reads from the environment

| Variable | Needed by | If it is missing |
|---|---|---|
| `PRIMARY_DB` | local database operator commands except `hash-password` | The command stops and tells you to run it inside the backend container |
| `REDIS` | `setup-link` always, `reset-password` to mint a link | `setup-link` fails. Everything else degrades to a warning and continues |
| `AUTH_SECRET` | `reset-password` when it mints a link | The link cannot be signed with the key the backend verifies, so it is refused |
| `APP_URL` | every printed link and sign-in hint | Links are built against `https://app.warmbly.com`, which is the hosted service and not your instance |
| `KMS_PROVIDER` and its key | `org export`, `org import` | The command stops: a workspace's sealed values cannot be opened, so an archive would be useless |
| `CREDENTIALS_ENCRYPTION_KEY` | `org export`, `org import` | A warning, and mailbox credentials are neither read nor written. Everything else still moves |
| `CREDENTIALS_ENCRYPTION_KEY`, `KMS_LOCAL_MASTER_KEY` | `backup`, `restore` | `backup` warns and leaves them out of the bundle. `restore` refuses when this host's differ from the bundle's |
| `BLOB_FS_ROOT` | `backup`, `restore` | Defaults to `/data/blobs`, which is where the compose stack mounts it |
| `WARMBLY_API_KEY` | every API command | The command stops and explains where a key comes from |
| `WARMBLY_API_URL` | every API command | Falls back to the instance's own `API_PUBLIC_URL`, then the hosted service |

`backup` and `restore` also need `pg_dump` and `psql`, which the backend image ships for exactly this reason. Inside the backend container all of these are already set, except the API key, which is yours. Outside it, export what the command needs, and match `AUTH_SECRET` to the backend's exactly.

Remote administrator commands require none of those instance secrets. They do not read `API_PUBLIC_URL`, `WARMBLY_API_URL` or `WARMBLY_API_KEY` to choose an instance or authenticate. Their only configuration-directory dependency is the operating system's user configuration location (`XDG_CONFIG_HOME` on Linux when set).

## Remote administrator commands

### Installing on another machine

The backend image still includes `warmblyctl` for local emergency recovery. To use the remote commands from a laptop or separate VM, compile just this binary from a trusted source checkout with the repository's Go toolchain, then install it on your path:

```bash
go build -o ./warmblyctl ./cmd/warmblyctl
install -m 0755 ./warmblyctl "$HOME/.local/bin/warmblyctl"
```

Create `$HOME/.local/bin` first if it does not exist. This is not the [`warmbly` installer](/api/cli/); that installer provides the separate public product CLI. Remote credential storage currently supports Linux, macOS and BSD on filesystems with verifiable POSIX ownership, modes and file locking. Windows and other unsupported platforms refuse remote storage rather than saving unprotected credentials.

Deploy the backend device-login routes and the administrator website's `/device` consent page first. An older backend can refuse the new routes without affecting any existing local operator or API-key commands. No database migration or `PRIMARY_DB` access is required by the remote client.

### Login, identity and logout

```bash
warmblyctl login --url https://api.example.com --server laptop --client-name 'work laptop'
warmblyctl whoami --server laptop
warmblyctl logout --server laptop
```

`--url` is the backend origin, not the administrator frontend URL. It must be a bare HTTPS origin without credentials, path, query or fragment. HTTP is allowed only for loopback development. Redirects are refused, including same-origin redirects; point directly at the backend's canonical origin. There is no insecure TLS flag or environment-selected default server.

Login prints the selected backend, requester, lookup code and `/device` verification path to **stderr**. Open your configured administrator website's `/device` page on any machine, sign in through the unchanged OAuth, passkey or password flow with MFA, enter the lookup code, and explicitly approve the named requester and instance. The administrator website may be on a different frontend origin; the CLI never sends a credential there. Merely viewing the request does not approve it.

The client compares the returned instance identity with the selected backend, waits the supplied poll interval, and stores only the resulting ordinary session token pair. The private device secret stays in process memory and is never printed, saved, placed in a URL or given to a browser. The CLI opens no browser and serves no HTTP. Approval expires after five minutes; a lost claim response requires a new login, not a replay. If the source session's original authentication proof ages out, perform a fresh browser authentication and begin a new request. There is no password or MFA-bypass option in the remote client.

`whoami` reads `/v1/auth/me` and prints the safely projected identity UUID, live admin permissions and MFA state. Arbitrary display names, addresses and free text are redacted. Login success prints session expiry timestamps, never token values.

Every remote request requires `--url` or a registered `--server`. A URL-only login uses the `default` profile *for that exact origin*; it never sets a global default. Named profiles are bound to the origin registered by `login --server NAME --url URL`. A conflicting URL is rejected; use another profile name for another instance or administrator session. Logging out leaves that non-secret server mapping intact.

Refresh happens before expiry or after one rejected read. Rotated credentials are atomically saved before use and local processes serialize refresh with a file lock. A failed or ambiguous refresh discards only the affected profile because the previous refresh token may already have been consumed. Reads retry at most once; writes are never automatically repeated. Permission, MFA and fresh-auth denials do not trigger refresh. Refresh is not fresh authentication: for `reauth_required`, log out and start a new explicitly browser-approved login, or use a new named profile.

Normal logout revokes the independent CLI session and removes its local credential only after confirmed success. A transport failure leaves the credential available for a later explicit attempt. An authentication rejection forgets it. `logout --local-only` is explicit local forgetting, not revocation; its JSON says `revoked:false` and the server session remains valid until separately revoked or expired.

Revoke **all** sessions for this user on this backend only deliberately:

```bash
warmblyctl logout --server laptop --all --write \
  --confirm 'POST /v1/auth/logout-all'
```

The server's existing logout-all behavior applies, including trusted-device forgetting. Matching local profiles for the same user and instance are removed; other instances and users are untouched. After device issuance, revoking the approving browser alone does not revoke the independently issued CLI session. Use normal session management or revoke-all.

### Private credential files

No supported OS keychain library is already used by this binary, so the fallback is an atomic `0600` credential file inside an owner-only `0700` directory. On Linux the location is `$XDG_CONFIG_HOME/warmblyctl/credentials.json`, or `$HOME/.config/warmblyctl/credentials.json`; on macOS it is under the user's `Library/Application Support/warmblyctl`. Tokens are plaintext secrets protected by filesystem ownership and modes, not encrypted by an application key.

The client checks owner identity, regular-file type, exact permissions, and hard links. It refuses unsafe existing storage instead of silently fixing modes. Writes use a private temporary file, sync, atomic rename and directory sync. A pinned directory handle bounds file access. The owner-only lock file protects rotating refresh tokens from concurrent processes. Passwords are never accepted in remote argv or saved configuration. Do not copy this directory into a repository, diagnostic archive or another user's account.

Existing configuration ancestors, including a symlinked home or config directory, resolve once to their canonical path before creating or opening storage. Every canonical ancestor must be owned by root or the effective user and refuse group/other writes, except a sticky directory such as `/tmp`. The final pinned credential directory must still be owned by the effective user with exactly `0700`; final-directory, lock-file and credential-file symlinks remain forbidden. Credential files require exactly `0600` and a single hard link. An untrusted target or dangling ancestor fails closed without creating credential storage.

### Read-only operational helpers

```bash
warmblyctl admin --help
warmblyctl admin workers list --server laptop
warmblyctl admin workers stats --id <worker-uuid> --server laptop
warmblyctl admin fleet nodes --server laptop
warmblyctl admin fleet capacity --server laptop
warmblyctl admin fleet decisions --server laptop
warmblyctl admin logs --node-id <worker-uuid> --level warn --limit 100 --server laptop
warmblyctl admin broker --node-id <worker-uuid> --server laptop
warmblyctl admin mailbox list --q example.com --server laptop
warmblyctl admin sync list --state stalled --server laptop
warmblyctl admin sends in-flight --server laptop
warmblyctl admin recovery dead-letters --status pending --server laptop
warmblyctl admin recovery failures --server laptop
warmblyctl admin recovery webhooks --server laptop
warmblyctl admin jobs list --server laptop
warmblyctl admin health monitoring --server laptop
```

Run `--help` after a helper for its typed UUID, time, enum and limit filters. Logs also accept `--after`, `--before` (inclusive RFC3339) and `--mailbox-id`. Lists with pagination accept the exact `next_cursor` returned by that endpoint, not page numbers. Helpers reject invalid filters before accessing credentials. Additional existing filters are available through the catalog request command below.

Keep AI diagnosis read-only: inspect availability, coverage, observation timestamps, dropped events, recent evidence and scope before concluding there is an outage. Empty or unavailable evidence is not proof of health. Node logs are the backend's allowlisted partial event history, not raw journals or provider conversations. Existing workers without an enrollment-bound log credential need an upgrade and authenticated re-enrollment. Rerun the existing join script with the same configuration directory and options; it reads the existing node UUID and preserves local overrides, without deleting the fleet node, moving mailboxes or manually copying secrets. The service restarts. Upgrade alone cannot securely exchange a fleet-wide broker credential for a node-owned identity. Until capture is received, evidence stays unavailable; there is no historical backfill. See [the upgrade procedure](/development/data-control/#upgrade-order-and-coverage).

Broker observations keep `commands` (this worker) separate from `results` (shared result consumer). Lag is committed-offset distance, not delivery success, throughput or proof that a particular worker is broken. Missing permissions, commits or dependencies remain explicitly unavailable. No command resets offsets.

In-flight send rows retain `has_message_id` and age information but omit bodies, headers and free-form failure text. Do not infer that an unknown outcome was never sent from an empty log or a missing message ID. These commands cannot clear uncertain-send holds, disable mailboxes or silently resend.

### Explicit recovery actions

Every admin mutation requires the named action **and** `--write` **and** an exact `--confirm 'METHOD /admin/concrete-target-path'`, excluding query parameters. This is an operator safeguard, not a replacement for server permissions, MFA, fresh authentication or audit logs. There is no generic `--yes` or automatic retry of a write. Transport errors mean the action may have reached the server; inspect state before another attempt.

Provider-throttle recovery and backfill restart use their existing server workflows, not unknown-send clearing:

```bash
warmblyctl admin recovery clear-throttle --id <mailbox-uuid> --server laptop \
  --write --confirm 'POST /admin/sync/<mailbox-uuid>/clear-throttle'
warmblyctl admin recovery restart-backfill --id <mailbox-uuid> --server laptop \
  --write --confirm 'POST /admin/sync/<mailbox-uuid>/restart-backfill'
```

Substitute the same real canonical UUID in both places. Clearing a provider throttle can resume provider work; restarting backfill resets the existing sync backfill state. Neither action establishes sending authority.

Dead-letter replay is never part of diagnosis. Inspect the task, mailbox, provider receipts and uncertain-send protections first. Replay additionally requires a separate risk acknowledgement:

```bash
warmblyctl admin recovery replay --id <dead-letter-uuid> --server laptop \
  --write --acknowledge-send-risk \
  --confirm 'POST /admin/tasks/dead-letters/<dead-letter-uuid>/replay'
```

Acknowledging risk does not prove the message was not delivered or override server recovery guards. Never replay an uncertain send solely because a CLI diagnostic has no evidence. Webhook lease recovery is separately available as `admin recovery reclaim-webhooks`, with exact confirmation for `POST /admin/webhooks/reclaim`.

### Cataloged admin API requests

```bash
warmblyctl admin catalog
warmblyctl admin api GET /admin/users --server laptop --query limit=25
warmblyctl admin api GET /admin/instance/settings --server laptop
warmblyctl admin api PUT /admin/workers/<worker-uuid>/tags --server laptop \
  --data @worker-tags.json --write \
  --confirm 'PUT /admin/workers/<worker-uuid>/tags'
```

The offline, deterministically sorted catalog lists every registered admin method and path template, permission names, fresh-auth requirement, read/write classification, replay-risk gate and unavailable reason. Replace parameter templates with canonical UUIDs, or bounded single-segment words for the few non-UUID parameters. The request command matches only that compiled inventory; it does not accept arbitrary URLs, `/v1` routes, traversal, encoded paths or made-up endpoints. Server guards remain authoritative even if the CLI catalog is older than the server.

Supply `--query key=value` separately for each bounded filter. Mutating JSON input is a single object from `--data @file` or `--data -` for stdin, limited to 1 MiB, without duplicate keys or excessive nesting. Inline JSON is intentionally refused so passwords cannot enter argv. Input, headers and request bodies are not echoed. Responses must be JSON within 8 MiB; there is no raw-output or secret-reveal flag.

Some catalog entries are discoverable but deliberately unavailable: browser device-consent endpoints, fleet join-token and tester credential issuance, binary export downloads and multipart imports, and arbitrary background-job execution. Use the existing browser or transfer/operator tools for those workflows. Unsupported entries fail before network or credential access, even with confirmation.

### JSON, redaction and exit codes

Remote success is one deterministic JSON value on stdout. Progress, approval codes, help and machine-readable errors are on stderr. Object keys are sorted by the JSON encoder; backend list ordering is preserved. Redaction projects only allowlisted diagnostic fields, canonical UUIDs, timestamps, typed enums, known broker scope names, validated cursors, numbers and booleans. Unknown fields are omitted and counted in `_redacted_fields`; arbitrary strings become `[REDACTED]`, including addresses and display names. No provider tokens, passwords, body, header, payload or free-form error subtree is exposed. The existing local emergency and public API commands keep their own documented output behavior unchanged.

Monitoring preserves the compiled source and metric IDs, count units, severity, schema version and bounded coverage reasons so an AI client can identify each observation without free-form labels. New or unrecognized identifiers still fail closed; update the CLI when the backend adds a monitoring metric. Titles, notes, mailbox addresses and provider error text remain redacted.

Errors preserve safe server `code`, HTTP `status` and `request_id`, but never raw backend messages, response bodies, response headers or transport errors. A numeric `retry_after_seconds` can accompany rate limiting.

| Exit | Meaning |
|---|---|
| `0` | Successful command or help |
| `2` | Invalid arguments, malformed input, invalid server selection or rejected HTTP 400 |
| `3` | Permission, MFA, fresh-auth or device approval denied |
| `4` | Login needed, expired/rejected session or invalid device identity |
| `5` | Transport, service, unsafe storage or invalid response failure |
| `6` | Rate limited |
| `7` | Cancelled operation |
| `8` | Missing write confirmation, missing replay-risk acknowledgement or unavailable catalog action |

The [shared authentication reference](/api/authentication/) remains the backend contract. These CLI additions change neither admin permission bits nor organization API scopes.

## The operator commands

| Command | Does |
|---|---|
| [`status`](#status) | Prints the instance state, the platform admins, the health checks, and what to run next |
| [`setup-link`](#setup-link) | Prints a fresh single-use link that claims an instance with no accounts |
| [`user create`](#user-create) | Creates an account, an organization and a trial, optionally a platform admin |
| [`user list`](#user-list) | Lists accounts, and answers whether any platform admin survives |
| [`user reset-password`](#user-reset-password) | Prints a one-time reset link, or sets the password from stdin |
| [`user grant-admin`](#user-grant-admin) | Gives an account a platform admin role |
| [`user revoke-admin`](#user-revoke-admin) | Takes platform admin away from an account |
| [`user disable-2fa`](#user-disable-2fa) | Clears an account's authenticator enrolment |
| [`hash-password`](#hash-password) | Prints an argon2 hash for unattended provisioning |
| [`fleet join-token`](#fleet) | Issues the token a machine needs to join the fleet |
| [`fleet list`](#fleet) | Every worker and consumer: role, version, liveness and usage |
| [`fleet version`](#fleet) | Shows or sets the version every node should run |
| [`fleet channel`](#fleet) | Follows stable or dev releases, or holds the fleet |
| [`fleet pin`](#fleet) | Holds one node at a version, to canary or hold it back |
| [`fleet remove`](#fleet) | Forgets a node; its mailboxes re-place themselves |
| [`backup`](#backup) | Writes the whole instance to one restorable bundle |
| [`restore`](#restore) | Restores a bundle onto this instance, replacing everything on it |
| [`org list`](#org-list) | Lists the workspaces on this instance with their id, owner, and size |
| [`org export`](#org-export) | Writes a whole workspace to a portable archive file |
| [`org import`](#org-import) | Applies an archive to a workspace on this instance |

`warmblyctl --help` lists them, and `warmblyctl <command> --help` prints one command's flags with an example. The [API commands](#the-api-commands) are further down, because they authenticate differently.

## status

```bash
docker compose -p warmbly exec backend warmblyctl status
```

| Flag | Default | Does |
|---|---|---|
| `--json` | off | Prints the state as JSON instead of prose, and always exits 0 |
| `--quiet` | off | Prints only the checks, without the instance summary. Ignored with `--json` |

It prints four blocks in this order: the instance state, the platform admins, how to get in, and the checks. A sample run is on [first run](/development/first-run/#checking-where-you-stand).

The **How to get in** block changes with the state. An unclaimed instance is told to print a setup link; a claimed one is given the recovery commands; an instance with no platform admin is told to promote an account.

The **Checks** block holds the same findings the admin panel's Setup and health page shows. Every one of them is documented on [instance health](/development/instance-health/).

`make doctor` is `warmblyctl status` for a compose install.

### Exit status and JSON

`status` exits non-zero when any check is at error severity, which is what makes `make doctor` usable as the last line of a deploy script.

`--json` is the exception: it always exits 0, because `make claim` uses it to decide whether the backend is answering at all. Read `.summary.error` for the verdict instead.

| Key | Holds |
|---|---|
| `accounts`, `claimed`, `setup_required` | Account count, and whether a setup link can still be issued |
| `admin_count`, `admins` | How many platform admins exist, and who they are |
| `registration`, `registration_source` | The resolved registration mode, and whether it was set or defaulted |
| `mail_transport`, `mail_delivers`, `mail_transport_source` | The transport, whether it puts mail on the wire, and where the setting came from |
| `app_url`, `app_url_source` | The base URL every printed link is built from |
| `next_steps` | The same commands the prose **How to get in** block prints |
| `checks`, `summary` | The findings, and the error, warning and note counts |

Those keys are a contract. They are only ever appended to, never renamed or dropped.

## setup-link

```bash
docker compose -p warmbly exec backend warmblyctl setup-link
```

Prints a single-use link that claims an unclaimed instance. It expires in 24 hours, replaces any outstanding link, and only its hash is stored, so this is the only time it is printed.

It refuses on an instance that already has accounts, by design: a second owner must never be mintable without an existing account. If you get that refusal, you want [`user create`](#user-create) instead.

This is the one command that cannot degrade, because the token lives in Redis. With Redis down, create the owner directly instead.

`make claim` wraps it for a compose install, and finds the link already in the logs before minting a new one.

## user create

```bash
docker compose -p warmbly exec backend warmblyctl user create --email you@example.com --admin
```

| Flag | Default | Does |
|---|---|---|
| `--email` | required | The address of the account to create |
| `--admin` | off | Grants every platform admin permission, the same as `--role super` |
| `--org` | `<name>'s Organization` | Names the new organization |
| `--no-org` | off | Creates the account with no organization and no trial |
| `--password-stdin` | off | Reads the password from stdin instead of prompting |

It creates the account, an organization and a free trial, and with `--admin` grants every platform admin bit. Use `--no-org` when the person will accept an invitation into a workspace that already exists.

An address that already exists is an error, and the message points you at `reset-password`.

### The password is the one you type

There is no generated or default password. On a terminal the command prompts for it twice, with echo off:

```
Password for you@example.com:
Repeat password for you@example.com:
Created account you@example.com (id ...), organization "Your Organization" (id ...),
a free trial, every platform admin permission (mask 4194303).

Next
  Sign in at http://localhost:5173 with the password you just set.
```

It must be between 8 and 128 characters, which is the same rule the dashboard enforces, so the scripted route is never weaker than the interactive one.

From a script, pipe it in so it never reaches your shell history or `ps` output:

```bash
printf '%s' "$PASSWORD" | docker compose -p warmbly exec -T backend \
  warmblyctl user create --email you@example.com --admin --password-stdin
```

### Signing in afterwards

Open the URL the command printed, which is `APP_URL`, and sign in with the address and that password. On the default compose stack that is `http://localhost:5173`.

Nothing else stands in the way on a stock self-hosted instance:

- **No emailed login code.** `AUTH_LOGIN_CODE` defaults to off when self-hosted, and a transport that does not deliver can never gate a login whatever the setting says. See [login codes](/development/accounts-and-access/#login-codes)
- **No email confirmation.** `REQUIRE_EMAIL_VERIFICATION` defaults to off when self-hosted, so the account is usable immediately
- **No captcha** unless you set `TURNSTILE_SECRET`
- **No invitation**, and no dependency on the registration mode. `invite_only` and `true` both govern the sign-up form, which this command does not use

The dashboard and the admin panel are separate apps. `--admin` opens the panel on `ADMIN_URL`, port `5174` in the default stack, not `APP_URL`.

## user list

```bash
docker compose -p warmbly exec backend warmblyctl user list --admin
```

| Flag | Default | Does |
|---|---|---|
| `--admin` | off | Lists only accounts holding platform admin permissions |
| `--limit` | `50` | How many accounts to print, between 1 and 100 |

Prints address, name, admin role and creation date, oldest first. `--admin` answers the narrow question of whether any admin account survives, and when none does it prints the two commands that fix that.

## user reset-password

```bash
docker compose -p warmbly exec backend warmblyctl user reset-password --email you@example.com
```

| Flag | Default | Does |
|---|---|---|
| `--email` | required | The address of the account to reset |
| `--password-stdin` | off | Reads the new password from stdin and sets it immediately |
| `--ttl` | `1h` | How long the printed link stays valid, up to 24 hours |

Without `--password-stdin` it prints a single-use reset URL and changes nothing yet. Open it in a browser and choose the new password there, which keeps it out of your shell history, your scrollback and the process list. It redeems through exactly the same path as the reset link the product emails, and opening it revokes every existing session for the account.

`--ttl` is capped at 24 hours because a reset link is a bearer credential for the account.

The automation path sets the password directly and revokes every existing session the same way:

```bash
printf '%s' "$PASSWORD" | docker compose -p warmbly exec -T backend \
  warmblyctl user reset-password --email you@example.com --password-stdin
```

Reach for that form when Redis is down, since a link cannot be minted without it.

## user grant-admin

```bash
docker compose -p warmbly exec backend warmblyctl user grant-admin --email you@example.com --role super
```

| Flag | Default | Does |
|---|---|---|
| `--email` | required | The address of the account to promote |
| `--role` | required | One of `super`, `support`, `ops`, `analyst` |

| Role | Grants |
|---|---|
| `super` | Every platform admin permission. This is the one that opens every admin screen |
| `support` | Users, campaigns, organizations, the warmup pool, warmup bans, appeals, enterprise inquiries, audit logs |
| `ops` | Workers and worker management, rate limits, analytics, organizations, audit logs |
| `analyst` | Read only: users, campaigns, organizations, analytics, audit logs |

The role replaces the account's existing mask rather than adding to it, and re-granting the role it already holds is reported as no change. The permissions are read from the session, so the account has to sign out and back in before the panel reflects the grant.

The account must already exist. `make grant-admin EMAIL=... ROLE=...` wraps this for a compose install. The admin panel now offers the same under **Accounts > Admins**, by preset or per bit, so this command is no longer the only way to add a second admin.

## user revoke-admin

```bash
docker compose -p warmbly exec backend warmblyctl user revoke-admin --email old@example.com
```

| Flag | Default | Does |
|---|---|---|
| `--email` | required | The address of the account to demote |
| `--force` | off | Allows removing the last remaining platform admin |

Removes every platform admin permission and leaves the account otherwise untouched.

Revoking the only remaining admin is refused without `--force`, because it closes the admin panel for everyone and nothing inside the product can reopen it. Grant someone else first. `make revoke-admin EMAIL=...` wraps it.

## user disable-2fa

```bash
docker compose -p warmbly exec backend warmblyctl user disable-2fa --email you@example.com
```

Clears the authenticator enrolment and the recovery codes, so a lost phone does not become a full password reset. The password still works, and the account can enroll a new authenticator from account security after signing in.

## hash-password

```bash
warmblyctl hash-password
printf '%s' 'your password' | warmblyctl hash-password
```

Prints an argon2 hash for `WARMBLY_BOOTSTRAP_PASSWORD_HASH`, which provisions the owner before the first start of an instance with no accounts. See [unattended provisioning](/development/first-run/#unattended-provisioning).

It prompts when a terminal is attached and reads the pipe when one is not, so the same command works by hand and in a provisioning script. The hash alone goes to stdout, so it can be captured or piped; the explanation goes to stderr. This is the only command that does not need a database.

<Callout type="warn" title="Quote the hash">
An argon2 PHC string contains `$` characters. Docker Compose reads those as interpolation, so a bare hash in `.env` silently loses part of itself. Wrap it in single quotes there, and double every `$` if you paste it into `docker-compose.yml` directly.
</Callout>

## backup

```bash
docker compose -p warmbly exec backend warmblyctl backup --out /data/blobs/warmbly.tar.gz
docker compose -p warmbly cp backend:/data/blobs/warmbly.tar.gz ./warmbly.tar.gz \
  && docker compose -p warmbly exec -T backend rm -f /data/blobs/warmbly.tar.gz
```

`/data/blobs` is the one path the container and the host both see, so it is the hand-off. Move the bundle out and delete it: `backup` excludes its own output from the archive, but a bundle left behind is swept into the next one. The delete is chained with `&&` so a failed copy cannot take the only copy with it.

Writes one bundle holding the three things that only restore together: a `pg_dump` of the database, the blob root (message bodies, attachments, avatars), and the encryption keys.

That combination is the whole reason this is a command rather than a documented list of steps. A dump alone restores an instance whose every mailbox credential decrypts to nothing, because the ciphertext is in the database and the key that opens it is in `.env`. The blob root alone restores bodies nothing points at.

| Flag | Does |
|---|---|
| `--out` | Where to write it. Defaults to `warmbly-backup-<timestamp>.tar.gz` in the working directory |
| `--no-keys` | Leaves the encryption keys out. The bundle is then not restorable on its own |
| `--no-blobs` | Database only |
| `--force` | Overwrite an existing output file |

The bundle is written 0600 and holds every mailbox credential on the instance plus the keys that open them. Treat the file as you would the instance itself: encrypted at rest, off the host, and not in a shared drive.

Blobs travel only on the filesystem provider. An instance storing them in S3 keeps them in the bucket, and the bundle says so rather than pretending to be complete.

<Callout title="Schedule it">
[`install.sh --wizard`](/development/install/) writes a `backup.sh` and a systemd timer that runs exactly this, keeps the last N bundles and can copy each one off the host.
</Callout>

## restore

```bash
docker compose -p warmbly exec backend warmblyctl restore --file /data/blobs/warmbly.tar.gz
docker compose -p warmbly restart
```

Empties the schema and replays the bundle into it, so it replaces every organization, user, campaign and mailbox currently on the instance. It prints what the bundle holds and asks you to type `restore` before it does; `--yes` skips that for scripts.

Before anything is written it compares the bundle's `CREDENTIALS_ENCRYPTION_KEY` and `KMS_LOCAL_MASTER_KEY` against this host's, and refuses to continue when they differ, printing the two lines to put in `.env` first. That check is the point of the command: without it a restore looks like it worked, and every mailbox fails to authenticate days later with no error that names the cause.

| Flag | Does |
|---|---|
| `--file` | The bundle. Required |
| `--yes` | Skip the typed confirmation |
| `--no-blobs` | Restore the database only, leaving the blob root alone |
| `--force` | Restore even though the keys differ. Accepts losing every stored mailbox credential |

Blobs are unpacked over the blob root additively: files the bundle knows about are overwritten, anything else is left alone.

[Data control](/development/data-control/#moving-an-instance) walks the whole move to a new host.

## org list

```bash
docker compose -p warmbly exec backend warmblyctl org list
```

Prints every workspace with its id, name, owner email, and member, mailbox and contact counts. It exists so the next two commands have something to name: `--org` accepts the id, the slug, or the owner's email, whichever you have to hand.

## org export

```bash
docker compose -p warmbly exec backend warmblyctl org export \
  --org you@example.com --out /tmp/workspace.warmbly.zip
```

Writes one workspace to a single archive file: the organization, its members and roles, mailboxes, campaigns, sequences, contacts, suppression, CRM, inbox history, and the send state that stops a migrated mailbox from sending twice its daily volume on the day it moves. It is the same archive the dashboard produces under **Settings > Data**, so either side of a migration can use either tool. The admin panel now offers the same for any organization under **Instance > Transfers**.

| Flag | Does |
|---|---|
| `--org` | The workspace: its id, its slug, or the owner's email. Required |
| `--out` | Where to write the archive. `-` writes to stdout, so it can be piped straight into `ssh` or a bucket. Required |
| `--groups` | Comma-separated data groups to include. Omit for everything |
| `--with-credentials` | Seals mailbox and integration credentials into the archive under a passphrase |
| `--passphrase-stdin` | Reads the passphrase from stdin instead of prompting twice |

`warmblyctl org export --help` prints the group list. `core` is always included; `inbox`, `sending`, `events` and `logs` are the ones that grow without limit, so dropping them is how you get a small archive that still rebuilds a working workspace.

<Callout type="warn" title="An archive with credentials is the most sensitive file this product produces">
It holds every mailbox password and refresh token in the workspace, protected only by the passphrase you type. Warmbly stores that passphrase nowhere, so losing it means exporting again, and anyone holding both the file and the passphrase can send mail as those mailboxes.
</Callout>

Without `--with-credentials` the credential fields travel empty and every mailbox arrives on the destination needing a reconnect. Everything else moves either way.

Progress goes to stderr and the archive to the file, so `--out -` streams cleanly:

```bash
warmblyctl org export --org you@example.com --out - | ssh newhost 'cat > /tmp/workspace.zip'
```

## org import

```bash
docker compose -p warmbly exec backend warmblyctl org import \
  --org you@example.com --file /tmp/workspace.warmbly.zip --dry-run
```

Applies an archive to a workspace on this instance. It always prints what the archive holds, which rows already exist here, and which members have no account, before writing anything.

| Flag | Does |
|---|---|
| `--org` | The destination workspace: id, slug, or owner email. Required |
| `--file` | The archive to read. Required |
| `--groups` | Comma-separated data groups to apply. Omit for everything in the archive |
| `--overwrite` | Replaces rows that already exist here instead of keeping them |
| `--with-credentials` | Prompts for the export passphrase so credentials come across |
| `--passphrase-stdin` | Reads that passphrase from stdin |
| `--dry-run` | Prints the report and writes nothing |

Run it with `--dry-run` first. The report is the same preflight the dashboard shows, and it costs nothing. The admin panel offers the same import, preflight included, under **Instance > Transfers**.

The whole import runs in one transaction: if any part fails, nothing lands and the workspace is untouched. Members are matched to accounts on this instance by email address, and anyone without one has their rows reassigned to the workspace owner, named in the report before you commit. An archive carries no password material, so it can never create an account here.

`org export` and `org import` are the per-workspace tool and [`backup`](#backup) is the instance-level one. They are not interchangeable: a bundle cannot be applied to one workspace, and a workspace archive cannot restore an instance.

Billing history, plan overrides, worker placement, mailbox sync checkpoints and warmup pool membership are exported for the record but never applied: each belongs to the instance rather than to the workspace. [Export and import](/guides/workspace-export-import/) has the full table.

## The API commands

Everything in this section talks to the public REST API with an API key, so it runs from anywhere: your laptop, a CI job, an agent's sandbox. It needs no database access and works against the hosted service and self-hosted instances alike.

```bash
export WARMBLY_API_KEY=wmbly_...                       # Settings > API keys
export WARMBLY_API_URL=https://api.your-instance.com  # omit for the hosted service

warmblyctl me                 # who the key is, and its scopes
warmblyctl campaign list
warmblyctl campaign start --id <uuid>
```

Every command prints the API's JSON response to stdout, untouched, and exits 1 on failure with the API's own `code` and `request_id` on stderr. What a key can do is exactly its granted [permissions](/api/permissions/); `warmblyctl me` shows them.

### The families

| Family | Covers |
|---|---|
| `me` | Identity and granted scopes |
| `campaign` | List, get, create, update, delete, sequence steps, sender pool, preflight, start, stop, test email, send log |
| `contact` | Search, get, lookup by address, create, update, delete, notes, timeline, import, export, custom fields |
| `mailbox` | List, get, update, disconnect, auth check, sync state, sending behaviour, verify, send, warmup start/pause/resume/stop/status |
| `inbox` | List, count, thread, mark seen, reply, compose, agent drafts, scheduled sends |
| `analytics` | Dashboard, deliverability, warmup, accounts, campaigns, usage, audit logs |
| `settings` | Outreach and suppression settings |
| `webhook` | Endpoints, secrets, deliveries, event types |
| `apikey` | Key self-service: list, create, update, revoke, the scope catalog |
| `template` | Reply templates |
| `crm` | Pipelines, deals, tasks |

`warmblyctl <family> --help` lists a family's subcommands, and `warmblyctl <family> <subcommand> --help` prints its flags and the endpoint it calls.

### The raw passthrough

Anything the API can do that has no typed command yet is one `api` call away, so nothing is out of reach:

```bash
warmblyctl api get "/campaigns?limit=10"
warmblyctl api post /contacts --data '{"email":"jane@example.com"}'
warmblyctl api patch "/campaigns/<id>" --data @changes.json
warmblyctl api delete "/webhooks/<id>"
```

Paths are relative to `/v1`. `--data` takes a JSON literal, `-` for stdin, or `@file`.

### Conventions

- List responses are `{"data": [...], "pagination": {"next_cursor", "has_more"}}`. Page with `--cursor` until `has_more` is false; the cursor is opaque.
- Writes accept `--idempotency-key`, and a retried command with the same key can never double-apply. Details on [authentication](/api/authentication/).
- A `429` failure names its `Retry-After`; wait it out and retry.
- `contact list` is a search: `--data` carries the filter body, and omitting it lists everything.

<Callout type="warn" title="Six commands put real mail on the wire">
`campaign start`, `campaign test-email`, `mailbox send`, `inbox reply`, `inbox compose` and `inbox approve-draft` send. Everything else reads or edits drafts. Run `campaign preflight` before `campaign start`; it is free and catches missing senders, empty audiences and broken tracking.
</Callout>

### For AI agents

The repository ships skills under `skills/` (`warmbly-cli` for the [`warmbly` CLI](/api/cli/), `warmbly-api` for the same product surface through `warmblyctl`, `warmbly-ops` for instance administration, `warmbly-install` for standing an instance up and moving it) that teach a coding agent these commands, the conventions above, and the sending-safety rules. Install them the way your agent expects, for example `cp -r skills/warmbly-api ~/.claude/skills/` for Claude Code, or point the agent at the `SKILL.md` directly. An agent given a scoped key and those skills can operate a workspace end to end without touching the database.

## When Redis is down

Account recovery must not depend on the cache, so most commands treat an unreachable Redis as a warning and carry on.

| Command | Without Redis |
|---|---|
| `setup-link` | Fails. The token lives in Redis, so there is nowhere to put it |
| `user reset-password` without `--password-stdin` | Fails. The link is bound to a nonce in Redis. The message points you at the `--password-stdin` form |
| `user reset-password --password-stdin` | Works. The password changes, but existing sessions cannot be revoked |
| `user create` | Works. The new account is not warmed into the cache, which is harmless |
| `status` | Works, and reports `redis_unreachable` as a finding |
| `org export` and `org import` | Work. Decrypted keys are not cached, so each workspace costs one extra KMS round trip |
| Everything else | Works |

## Make targets

Every one of these runs `warmblyctl` inside the backend container of a compose install.

| Target | Runs |
|---|---|
| `make claim` | Finds or prints the first-run claim link, and says what to do instead when the instance is already claimed |
| `make doctor` | `warmblyctl status`, non-zero on any error-severity check |
| `make cli ARGS="..."` | Any command, with a TTY attached so prompts work |
| `make grant-admin EMAIL=... ROLE=...` | `user grant-admin` |
| `make revoke-admin EMAIL=...` | `user revoke-admin` |

## See also

- [First run](/development/first-run/) for claiming a fresh instance and provisioning the owner unattended
- [Accounts and access](/development/accounts-and-access/) for registration modes, invitations, login codes and single sign-on
- [Instance health](/development/instance-health/) for every check `status` can report
- [Troubleshooting](/development/troubleshooting/) for symptoms and the command that fixes each one
- [Export and import](/guides/workspace-export-import/) for what an archive contains and what deliberately does not travel
- [Configuration reference](/development/configuration/) for every variable named here
- [API authentication](/api/authentication/) and [permissions](/api/permissions/) for the keys and scopes the API commands run on

## fleet

Manages the machines running Warmbly. Every process on a machine you own is a node: a `worker` sends and syncs mail, a `consumer` processes events. Both enrol with a token, heartbeat, and pull the version the control plane tells them to run.

Adding a machine is two commands:

```bash
warmblyctl fleet join-token
```

Then on the machine itself:

```bash
curl -fsSL https://<your-instance>/join.sh | sh -s -- \
  --url https://<your-instance> --token <token> --role worker
```

The token is shown once. It joins any number of machines until it expires: seven days by default, or what `--ttl` sets, up to `720h`. Issuing another revokes the previous one; machines that already joined are unaffected. A machine presenting an expired token is refused, so issue a fresh one.

```bash
warmblyctl fleet list
```

```
ROLE      NAME      STATE  VERSION           REGION      MEM    SEEN     ID
worker    box-1     live   v1.4.2            eu-central  128MB  12s ago  de434ce4-...
consumer  events-1  live   v1.0.0 -> v1.4.2  -           96MB   30s ago  2b334317-...
```

`STATE` is `live`, `unreachable` (enrolled but silent) or `stopped`. A version shown as `a -> b` has not picked up the target yet. `--role worker` narrows it; `--json` is machine-readable.

Moving the fleet:

```bash
warmblyctl fleet version              # what the fleet should be on
warmblyctl fleet version v1.4.2       # move everything to a tag
warmblyctl fleet channel stable       # follow releases again
warmblyctl fleet pin <node-id> v1.4.1 # hold or canary one machine
warmblyctl fleet pin <node-id>        # clear that pin
```

Setting a tag also pins the channel, so a release landing later does not undo a deliberate rollback.

```bash
warmblyctl fleet remove <node-id>
```

Forgets a node. Any mailboxes it carried are re-placed within a few minutes. It does not stop the process: a machine still running re-joins on its next heartbeat, so stop the service there too.