Merge branch 'fix/imap-dead-session-reconnect' of https://github.com/rocker1166/warmbly into fix/imap-dead-session-reconnect

This commit is contained in:
SUMAN JANA
2026-09-07 07:15:49 +00:00
470 changed files with 41912 additions and 2622 deletions
+8 -5
View File
@@ -11,14 +11,17 @@
.vscode
.agentd
# Frontend bits — the web service bind-mounts ./web directly, so we
# never want the heavyweight frontend tree to land in the Go build
# context.
web/node_modules
web/.vite
# Frontend trees: the Go images never need them, and a host node_modules
# copied over the forms image's own install makes pnpm abort the build (#292).
**/node_modules
**/.vite
web/dist
web/build
web/coverage
forms/dist
admin/dist
site/dist
site/.astro
# Compiled host binaries that would shadow / inflate the context.
bin
+29
View File
@@ -460,6 +460,35 @@ BILLING_PROVIDER=none
# WORKER_INSTALLER_PATH=
# === Updates ==================================================================
#
# The admin panel's top bar shows the running version and turns into an
# "Update available" pill when a newer release exists. The backend checks
# GitHub Releases on the interval below; the check is one unauthenticated API
# read and can be turned off. Full page: https://docs.warmbly.com/development/updates/
# UPDATE_CHECK_ENABLED=true
# UPDATE_CHECK_INTERVAL=30m # minimum 5m
# UPDATE_CHANNEL=stable # stable | dev (dev also offers prereleases)
# RELEASES_GITHUB_REPO=warmbly/warmbly # point a fork's instance at the fork
# RELEASES_GITHUB_TOKEN= # optional; only raises the API rate limit
#
# "Update and restart" in that pill needs the updater: a sidecar that pulls this
# checkout, rebuilds the images and recreates the containers, then waits for the
# backend to answer again. It holds the docker socket (root on this host), so it
# runs only under the "updater" compose profile. `make up` enables the profile;
# for a plain `docker compose up`, uncomment the next line. Comment it out to
# update by hand with `git pull && make up` instead.
# COMPOSE_PROFILES=updater
#
# The backend reaches it at UPDATER_URL with UPDATER_TOKEN (defaults to
# INTERNAL_API_TOKEN). UPDATER_URL=none makes the panel report-only.
# UPDATER_URL=http://updater:8095
# UPDATER_TOKEN=
# UPDATER_FETCH_INTERVAL=30m # how often the updater git-fetches for the "commits behind" count
# UPDATER_PRUNE=true # docker image prune after a successful update
# UPDATER_ALLOW_DIRTY=false # let an update run over local modifications in the checkout
# WARMBLY_REPO_DIR= # the checkout's absolute path; defaults to $PWD at compose time
# === Tracking service (Rust, open/click) ======================================
#
# The shipped docker-compose.yml pins the listen address, so these two only take
+17 -2
View File
@@ -6,7 +6,7 @@ on:
workflow_dispatch:
inputs:
service:
description: "Service to build (all, backend, consumer, worker, forms, tracking, realtime)"
description: "Service to build (all, backend, consumer, worker, forms, updater, tracking, realtime)"
required: false
default: "all"
@@ -61,6 +61,15 @@ jobs:
- 'cmd/forms/**'
- 'forms/**'
- 'deploy/docker/forms.Dockerfile'
updater:
- 'go.mod'
- 'go.sum'
# internal/** and not internal/updater/**: the binary links
# internal/version and whatever else it grows, and an image that
# silently skips a rebuild is worse than one rebuilt too often.
- 'internal/**'
- 'cmd/updater/**'
- 'deploy/docker/updater.Dockerfile'
tracking:
- 'tracking/**'
realtime:
@@ -75,6 +84,7 @@ jobs:
CONSUMER: ${{ steps.filter.outputs.consumer }}
WORKER: ${{ steps.filter.outputs.worker }}
FORMS: ${{ steps.filter.outputs.forms }}
UPDATER: ${{ steps.filter.outputs.updater }}
TRACKING: ${{ steps.filter.outputs.tracking }}
REALTIME: ${{ steps.filter.outputs.realtime }}
run: |
@@ -82,7 +92,7 @@ jobs:
native=""
if [ "$EVENT" = "workflow_dispatch" ]; then
sel="${SELECTED:-all}"
for s in backend consumer worker forms; do
for s in backend consumer worker forms updater; do
if [ "$sel" = "all" ] || [ "$sel" = "$s" ]; then go="$go\"$s\","; fi
done
for s in tracking realtime; do
@@ -93,6 +103,7 @@ jobs:
if [ "$CONSUMER" = "true" ]; then go="$go\"consumer\","; fi
if [ "$WORKER" = "true" ]; then go="$go\"worker\","; fi
if [ "$FORMS" = "true" ]; then go="$go\"forms\","; fi
if [ "$UPDATER" = "true" ]; then go="$go\"updater\","; fi
if [ "$TRACKING" = "true" ]; then native="$native\"tracking\","; fi
if [ "$REALTIME" = "true" ]; then native="$native\"realtime\","; fi
fi
@@ -132,6 +143,10 @@ jobs:
context: .
file: deploy/docker/${{ matrix.service }}.Dockerfile
push: true
build-args: |
VERSION=dev-${{ github.sha }}
COMMIT=${{ github.sha }}
BUILT_AT=${{ github.event.head_commit.timestamp }}
tags: |
${{ env.IMAGE_PREFIX }}/${{ matrix.service }}:${{ github.sha }}
${{ env.IMAGE_PREFIX }}/${{ matrix.service }}:dev
+44 -1
View File
@@ -34,6 +34,8 @@ jobs:
forms: ${{ steps.filter.outputs.forms }}
make: ${{ steps.filter.outputs.make }}
ios: ${{ steps.filter.outputs.ios }}
installer: ${{ steps.filter.outputs.installer }}
cli-installer: ${{ steps.filter.outputs.cli-installer }}
steps:
- uses: actions/checkout@v4
- uses: dorny/paths-filter@v3
@@ -70,6 +72,16 @@ jobs:
- 'integrations/make/**'
ios:
- 'ios/**'
installer:
- 'site/public/install.sh'
- 'site/public/install.sh.sha256'
- 'scripts/check-installer.sh'
cli-installer:
- 'site/public/cli.sh'
- 'site/public/cli.sh.sha256'
- 'site/public/cli.ps1'
- 'scripts/check-cli-installer.sh'
- 'scripts/build-cli.sh'
migrations-ci:
name: Migrations
@@ -229,6 +241,37 @@ jobs:
- name: Build
run: pnpm build
# The installer is served verbatim from site/public, so what CI checks is
# exactly what a `curl | sh` executes. The checksum is published next to it
# for anyone who would rather download, verify and read before running, and
# it is only useful if it cannot drift.
installer-ci:
name: Installer CI
needs: changes
if: needs.changes.outputs.installer == 'true'
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- name: Check the installer
run: ./scripts/check-installer.sh
cli-installer-ci:
name: CLI Installer CI
needs: changes
if: needs.changes.outputs.cli-installer == 'true'
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
# dash is what /bin/sh is on Debian and Ubuntu, which is what most people
# will pipe this into. Ubuntu runners already have shellcheck and pwsh.
- name: Install dash
run: sudo apt-get update && sudo apt-get install -y dash
- name: Check the CLI installer
run: ./scripts/check-cli-installer.sh
make-ci:
name: Make App CI
needs: changes
@@ -398,7 +441,7 @@ jobs:
ci-status:
name: CI Status
runs-on: ubuntu-latest
needs: [changes, migrations-ci, go-ci, web-ci, admin-ci, site-ci, forms-ci, make-ci, rust-ci, elixir-ci, ios-ci, frontend-images]
needs: [changes, migrations-ci, go-ci, web-ci, admin-ci, site-ci, forms-ci, installer-ci, cli-installer-ci, make-ci, rust-ci, elixir-ci, ios-ci, frontend-images]
if: always()
steps:
- name: Check CI status
+155 -2
View File
@@ -36,7 +36,7 @@ jobs:
strategy:
fail-fast: false
matrix:
service: [backend, consumer, worker]
service: [backend, consumer, worker, forms, updater, cli]
runs-on: ubuntu-latest
permissions:
contents: read
@@ -60,6 +60,10 @@ jobs:
context: .
file: deploy/docker/${{ matrix.service }}.Dockerfile
push: true
build-args: |
VERSION=${{ github.ref_name }}
COMMIT=${{ github.sha }}
BUILT_AT=${{ github.event.head_commit.timestamp }}
tags: |
${{ env.IMAGE_PREFIX }}/${{ matrix.service }}:${{ github.ref_name }}
${{ env.IMAGE_PREFIX }}/${{ matrix.service }}:v${{ needs.validate-tag.outputs.minor }}
@@ -210,17 +214,95 @@ jobs:
-t ${{ env.IMAGE_PREFIX }}/${{ matrix.service }}:prod \
$(printf '${{ env.IMAGE_PREFIX }}/${{ matrix.service }}@sha256:%s ' *)
# The `warmbly` CLI is a plain static binary, so it cross-compiles for every
# platform on one runner.
#
# Assets are named WITHOUT the version, so
# releases/latest/download/warmbly_linux_amd64.tar.gz always resolves. That is
# what lets the install script find the newest build with no GitHub API call,
# which matters because the unauthenticated API is rate limited and a curl
# installer that fails on a busy CI runner is not an installer.
build-cli:
name: Build CLI
needs: validate-tag
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- name: Set up Go
uses: actions/setup-go@v5
with:
go-version-file: go.mod
cache: true
- name: Cross-compile and package
env:
VERSION: ${{ github.ref_name }}
COMMIT: ${{ github.sha }}
BUILT_AT: ${{ github.event.head_commit.timestamp }}
run: |
set -euo pipefail
./scripts/build-cli.sh dist
- name: Upload
uses: actions/upload-artifact@v4
with:
name: warmbly-cli
path: dist/
retention-days: 1
create-release:
name: Create GitHub Release
needs: [validate-tag, build-go, build-frontend, merge-native]
needs: [validate-tag, build-go, build-frontend, merge-native, build-cli]
runs-on: ubuntu-latest
permissions:
contents: write
# The manifest step logs in to GHCR and inspects each published image;
# without read access that fails on a private package.
packages: read
steps:
- uses: actions/checkout@v4
with:
fetch-depth: 0
- name: Download the CLI binaries
uses: actions/download-artifact@v4
with:
name: warmbly-cli
path: /tmp/cli
# The installer verifies what it pulled against this file, so it is what
# makes "curl | sh" checkable after the fact rather than only before it.
# One line per service, because the thing that reads it is a POSIX shell.
- name: Log in to GHCR
uses: docker/login-action@v3
with:
registry: ${{ env.REGISTRY }}
username: ${{ github.actor }}
password: ${{ secrets.GITHUB_TOKEN }}
- name: Publish the image manifest
env:
TAG: ${{ github.ref_name }}
run: |
set -euo pipefail
{
printf '{\n'
printf ' "tag": "%s",\n' "$TAG"
printf ' "registry": "%s",\n' "$IMAGE_PREFIX"
printf ' "images": {\n'
first=1
for s in backend consumer worker forms updater web admin tracking realtime; do
digest=$(docker buildx imagetools inspect "$IMAGE_PREFIX/$s:$TAG" \
--format '{{json .Manifest.Digest}}' | tr -d '"')
[ "$first" = 1 ] || printf ',\n'
first=0
printf ' "%s": "%s"' "$s" "$digest"
done
printf '\n }\n}\n'
} > /tmp/images.json
cat /tmp/images.json
- name: Generate release notes
env:
GITHUB_SERVER_URL: ${{ github.server_url }}
@@ -236,6 +318,35 @@ jobs:
cat /tmp/changelog.md
cat <<'EOF'
## Install
```
curl -fsSL https://warmbly.com/install.sh | sh -s -- --version ${{ github.ref_name }}
```
Add `--wizard` to be asked where each store lives, what is kept and
for how long, and how it is backed up.
## The warmbly CLI
```
curl -fsSL https://warmbly.com/cli.sh | sh # macOS, Linux
irm https://warmbly.com/cli.ps1 | iex # Windows
brew install warmbly/tap/warmbly # Homebrew
scoop install warmbly # Scoop
```
Or take an archive below and unpack it yourself; `checksums.txt`
verifies every one of them. Already installed? `warmbly upgrade`.
`images.json` below lists the manifest digest of every image in this
release. The installer checks what it pulled against it, and you can
too:
```
docker image inspect ${{ env.IMAGE_PREFIX }}/backend:${{ github.ref_name }} --format '{{index .RepoDigests 0}}'
```
## Docker Images
All images are available at `ghcr.io/${{ github.repository_owner }}/warmbly`:
@@ -249,6 +360,9 @@ jobs:
| Realtime | `${{ env.IMAGE_PREFIX }}/realtime:${{ github.ref_name }}` |
| Dashboard (web) | `${{ env.IMAGE_PREFIX }}/web:${{ github.ref_name }}` |
| Admin | `${{ env.IMAGE_PREFIX }}/admin:${{ github.ref_name }}` |
| Forms | `${{ env.IMAGE_PREFIX }}/forms:${{ github.ref_name }}` |
| Updater | `${{ env.IMAGE_PREFIX }}/updater:${{ github.ref_name }}` |
| CLI | `${{ env.IMAGE_PREFIX }}/cli:${{ github.ref_name }}` |
## Deployment
@@ -256,11 +370,50 @@ jobs:
EOF
} > /tmp/release-body.md
# The formula and manifest are generated with the archives, so their
# checksums can never drift from what they describe. Pushing them is
# skipped, loudly, when the tap token is not configured: a release must
# not fail because a downstream package repo is not set up yet.
- name: Publish the Homebrew formula and Scoop manifest
env:
TAP_TOKEN: ${{ secrets.HOMEBREW_TAP_TOKEN }}
TAG: ${{ github.ref_name }}
run: |
set -euo pipefail
if [ -z "${TAP_TOKEN:-}" ]; then
echo "HOMEBREW_TAP_TOKEN is not set; skipping the tap push."
echo "The formula and manifest are still attached to the release."
exit 0
fi
# A prerelease must never become the default `brew install`.
case "$TAG" in
*-*) echo "$TAG is a prerelease; not updating the taps."; exit 0 ;;
esac
git config --global user.name "warmbly-release"
git config --global user.email "release@warmbly.com"
git clone --depth 1 \
"https://x-access-token:${TAP_TOKEN}@github.com/warmbly/homebrew-tap.git" /tmp/tap
mkdir -p /tmp/tap/Formula /tmp/tap/bucket
cp /tmp/cli/warmbly.rb /tmp/tap/Formula/warmbly.rb
cp /tmp/cli/warmbly.json /tmp/tap/bucket/warmbly.json
cd /tmp/tap
git add Formula/warmbly.rb bucket/warmbly.json
if git diff --cached --quiet; then
echo "the tap already describes $TAG"
else
git commit -m "warmbly $TAG"
git push
echo "pushed warmbly $TAG to the tap"
fi
- name: Create Release
uses: softprops/action-gh-release@v2
with:
tag_name: ${{ github.ref_name }}
name: ${{ github.ref_name }}
body_path: /tmp/release-body.md
files: |
/tmp/images.json
/tmp/cli/*
draft: false
prerelease: ${{ contains(github.ref_name, '-') }}
+11
View File
@@ -11,6 +11,12 @@
/worker
/backend
/seed
/forms
/migrate
/updater
/warmblyctl
/warmbly
/cli
# Test binary, built with `go test -c`
*.test
@@ -24,6 +30,10 @@ profile.cov
# Dependency directories (remove the comment below to include it)
vendor/
# Local CLI builds (make warmbly / make warmbly-dist)
/bin/
/dist/
# Go workspace file
go.work
go.work.sum
@@ -100,3 +110,4 @@ build-errors.log
# Xcode per-user state (ios/)
xcuserdata/
.DS_Store
/.worktrees/
+86 -2
View File
@@ -13,6 +13,15 @@ At a product level, the app does four main things:
The backend API is the control plane. Workers are the execution plane.
It ships as a hosted service and as a self-host, and the two are the same code.
The front door for the self-host is one command,
`curl -fsSL https://warmbly.com/install.sh | sh`, which pulls the published
release images and needs no clone and no compiler. `--wizard` turns it into an
interactive install that asks the data-control questions up front: where each
store lives, what is kept and for how long, how it is backed up. The script is
`site/public/install.sh` and it has its own rules below; the docs are
`docs/content/docs/development/install.mdx` and `data-control.mdx`.
## Working In This Repo
CI is strict. `go build ./...` succeeding is not enough — `golangci-lint` runs `gofmt` as part of its checks, and a single unformatted import block or mis-indented doc comment will fail the PR even when the code compiles cleanly. Before declaring any Go change done:
@@ -39,6 +48,7 @@ Docs stay in sync:
- the customer docs site lives in `docs/` (Fumadocs, served at docs.warmbly.com); content is MDX under `docs/content/docs/` in three sections: `guides/` (product behavior), `learn/` (fundamentals), `api/` (API reference)
- any change that alters user-visible behavior must update the matching docs page in the same change: a new or changed endpoint updates `api/endpoints.mdx` (scope map) and, where relevant, `api/authentication.mdx`; a new or changed API permission updates `api/permissions.mdx` including the permission table, presets, and all three language tabs in the constants section; a new or changed error code updates `api/error-codes.mdx`; a new or changed product feature, default, limit, or setting updates the relevant `guides/` page (or adds one, registered in `guides/meta.json` under the right section group)
- removing or renaming a feature, endpoint, or permission means removing or updating its docs too; do not leave stale docs behind
- self-hosting behavior has its own pages under `docs/content/docs/development/`: a change to the installer or to what it asks updates `install.mdx`; a change to where a store lives, how long something is kept, or how an instance is backed up or moved updates `data-control.mdx`; a new environment variable updates `configuration.mdx`, and a new database-backed setting updates its table there as well as the admin panel
- follow the docs conventions: frontmatter `title` is the H1 (no `#` heading in the body), no decorative sidebar icons (pages and `meta.json` sections carry no `icon`; the source loader has the lucide icon plugin disabled, and code-sample tabs use the real language logo instead), sentence-case headings, no em dashes in prose, internal links use trailing slashes (`/guides/mailboxes/`)
- verify with `pnpm types:check` and `pnpm lint` in `docs/` (the site is a fully static export; `pnpm build` writes `out/`)
@@ -81,6 +91,69 @@ Rows move as `jsonb` in both directions, so adding a *column* to an existing tab
The same applies to the customer-facing side of a feature: if it stores org data, its docs page and `docs/content/docs/guides/workspace-export-import.mdx` should agree about whether that data moves.
### The installer is a published artifact
`site/public/install.sh` is the one-command self-host installer, served
verbatim from the static site at `https://warmbly.com/install.sh`. What is in
the repo is byte for byte what a stranger pipes into their shell, which makes
it the highest-consequence file here that is not Go.
It is a wizard: an animated stepper, arrow-key and vim menus, live pull and
health screens, a review pass, and a `--demo` mode that plays the whole thing
while installing nothing. `docs/content/docs/development/install.mdx` documents
it and `data-control.mdx` documents what its questions decide; the
`warmbly-install` skill is the agent-facing version.
Rules, all of them learned from breaking them:
- **POSIX sh, not bash.** It runs under whatever `/bin/sh` the host has, which
on Debian and Ubuntu is dash. A `sh -n` that passes under your own shell
proves nothing about that; `make installer-check` runs `dash -n` and
`shellcheck -s sh`
- **`set -eu`, everything in a function, `main "$@"` on the last line**, so a
truncated download executes nothing. Watch for `[ x ] && y` as a function's
LAST command: it returns non-zero when the test fails, and under `set -e`
that ends the run. Use an `if`, or end with `return 0`
- **Nothing drawn inside a redraw loop may be wider than the terminal.** A
wrapped line is two physical rows while every `ESC[nA` counts logical ones,
so one long option hint makes the menu draw over itself and over whatever was
on screen before it. Everything in a loop goes through `fit`
- **The screen is not ours.** It appends by default, `--clear` is opt-in, and
`ESC[3J` (erase scrollback) is never sent
- **Regenerate the checksum.** `site/public/install.sh.sha256` is what makes
"download, verify, read, run" a real alternative to piping into a shell.
`make installer-sha`, and CI fails when the two disagree
- **Every answer is a flag and a `WARMBLY_*` variable.** An install that can
only be driven by keyboard cannot be driven by Ansible, cloud-init or an
agent, and the wizard exists to be optional
- **Idempotent.** A second run adopts the existing `.env`, never regenerates a
secret (a new `CREDENTIALS_ENCRYPTION_KEY` is permanent data loss) and never
moves an existing data root
Run `make installer-check` before pushing a change to it (POSIX parse,
shellcheck, `--help`, `--demo`, `--print-env`, a compose file per answer shape,
a pty width regression test, and the checksum). `make installer-demo` is how
you see a UI change without installing anything.
`site/public/cli.sh` is the second published script, served at
`https://warmbly.com/cli.sh`, and it installs the `warmbly` CLI rather than an
instance. Every rule above applies to it, plus two of its own:
- **It verifies what it downloads.** The release publishes `checksums.txt`
next to the archives, and a mismatch installs nothing rather than warning.
Never weaken that to a warning
- **Release assets are named without the version**, so
`releases/latest/download/warmbly_<os>_<arch>.tar.gz` resolves with no
GitHub API call. The unauthenticated API is rate limited per IP, which is
what breaks a curl installer on a shared runner. `scripts/build-cli.sh` and
the platform list in `cli.sh` have to agree; `make cli-check` fails when they
do not
`make cli-check` runs the whole thing (POSIX parse, shellcheck, `--help`,
`--dry-run`, a real install from a local mirror, checksum tampering, uninstall,
the PowerShell parse and the checksum), and `make cli-sha` regenerates the
checksum after any edit. `site/public/cli.ps1` is the Windows half.
### Verification: what to run, what to skip
Keep the loop fast. The signals that matter are formatting, lint, and typecheck — not local builds or browser automation.
@@ -92,6 +165,11 @@ Always, before calling a Go change done:
For frontend changes, run `pnpm typecheck` and `pnpm lint` in any tree you touched.
For a change to `site/public/install.sh`, run `make installer-check`; it is the
same script CI runs and it regenerates nothing, so a stale checksum fails there
exactly as it will in CI. For `site/public/cli.sh` or `cli.ps1`, the equivalent
is `make cli-check` (and `make cli-sha` after any edit).
Do not:
- do not run `go build ./...`, `pnpm build`, or docker image builds as a "did it work" check. They are slow and are not what CI gates on. `go run` (via the make dev targets) already compiles; `make fmt` + `make lint` + `pnpm typecheck` are the real signals.
@@ -116,6 +194,8 @@ Infra runs in docker; the Go services and frontends run natively on the host for
- `make web` / `make admin` / `make site` — frontend dev servers (5173 / 5174 / 4321), pointed at the native backend.
- `make seed` — load fixtures (after the backend has applied migrations).
- `make fmt` / `make lint` — format and lint Go.
- `make installer-demo` — walk the self-host installer's wizard with nothing installed: the real questions and review, a played pull and start. No Docker, no network, no file written. `WARMBLY_DEMO_FAST=1` collapses the animations while iterating on them.
- `make installer-check` / `make installer-sha` — everything CI runs against `site/public/install.sh`, and the checksum regeneration that has to follow any edit to it.
Prefer native `make backend` over rebuilding the docker backend image: docker rebuilds are slow because the image bakes in the migrations and the compiled binary, so a one-line change means a full image build + container recreate. The native targets skip all of that. The dockerized hot-reload flow (`make app`) and prod-image smoke test (`make up`) remain available when you specifically need containers.
@@ -197,16 +277,18 @@ API keys with the `REALTIME_SUBSCRIBE` permission (bit 11) can connect to the sa
- `cmd/backend`: API and business orchestration
- `cmd/consumer`: consumes Kafka events and updates platform state
- `cmd/worker`: execution worker for send/sync operations
- `cmd/cli`: the `warmbly` CLI, the customer-facing one. A signed-in, multi-host client of the public REST API (`internal/cli/*` holds its config, HTTP client and renderers). It never serves HTTP and never touches Postgres; `cmd/warmblyctl` is the operator's CLI and keeps the database half. The directory is `cli` and the binary is `warmbly`, so every build target names its output explicitly (`-o warmbly`), and `go install` needs the rename documented in `docs/content/docs/api/cli.mdx`
- `cmd/forms`: the public face of hosted lead-capture forms (`internal/formserver`): serves the built `forms/` app, per-form page shells (with the per-form embed CSP), the embed loader and public submissions on their own origin (`FORMS_DOMAIN`). No database; the backend's internal API is its only dependency, like the tracking service
- `forms/`: the public form page app (React + TanStack Router/Query/Form, Vite CSR build). Renders a published form from the same-origin `/api/forms/:publicID`, submits to `/api/forms/:publicID/submit`; the Go forms service hosts the build
- `tracking/`: open and click tracking service
- `realtime/`: websocket fanout service
- `web/`: in-product frontend (dashboard). Customer-facing only: it holds no platform-admin screens, and operator tooling must not be added back here
- `admin/`: platform admin panel (:5174), the single operator surface. Workers, users, orgs, warmup, campaigns, analytics, audit. Every route sits behind `RequireAdmin` and the backend's `RequireAdminPermission` gates
- `site/`: public marketing site (Astro 5 + Tailwind v4)
- `site/`: public marketing site (Astro 5 + Tailwind v4). `site/public/install.sh` is the self-host installer served at warmbly.com/install.sh and `site/public/cli.sh` is the CLI installer served at warmbly.com/cli.sh (with `cli.ps1` for Windows), each with its checksum next to it; see the rules above before touching either
- `deploy/`: production deploy manifests, infrastructure, and runtime config
- `docs/`: documentation site (docs.warmbly.com); product guides, API reference, and self-hosting/engineering docs under `content/docs/development/`
- `scripts/`: one-off tooling (codegen, migrations, local dev utilities)
- `scripts/`: one-off tooling (codegen, migrations, installer checks, local dev utilities)
- `skills/`: agent playbooks shipped with the repo (`warmbly-cli` for the `warmbly` CLI, `warmbly-api` for the same product surface through `warmblyctl`, `warmbly-ops` for instance administration, `warmbly-install` for standing an instance up and moving it). A command an operator can run is not usable by an agent until it is in one of these
## Worker Topology
@@ -787,6 +869,8 @@ These files are the fastest way to rebuild context:
- `README.md`
- `docs/content/docs/development/architecture.mdx`
- `docs/content/docs/development/install.mdx` and `data-control.mdx` (what a self-hoster is asked, and what each answer decides)
- `site/public/install.sh` (the installer itself)
- `cmd/worker/main.go`
- `internal/app/worker/assignment.go`
- `internal/tasks/email_task.go`
+71 -3
View File
@@ -15,6 +15,21 @@ export PATH := $(GO_BIN):$(PATH)
# fresh clones without any environment setup.
COMPOSE := docker compose -p warmbly
# Build identity stamped into the Go images (shown in the admin panel's top bar
# and read by the update check). Empty outside a git checkout, which the
# binaries report as "dev".
export WARMBLY_BUILD_VERSION ?= $(shell git describe --tags --always --dirty 2>/dev/null)
export WARMBLY_BUILD_COMMIT ?= $(shell git rev-parse HEAD 2>/dev/null)
export WARMBLY_BUILD_TIME ?= $(shell date -u +%Y-%m-%dT%H:%M:%SZ)
# The updater sidecar (one-click "Update and restart" from the admin panel)
# holds the docker socket, so it lives behind a compose profile. `make up`
# turns it on; UPDATER=false leaves it off.
UPDATER ?= true
ifeq ($(UPDATER),true)
UP_PROFILES := --profile updater
endif
GOLANGCI_LINT_VERSION ?= v1.64.8
PROTOC_GEN_GO_VERSION ?= v1.36.11
PROTOC_GEN_GO_GRPC_VERSION ?= v1.6.1
@@ -23,10 +38,11 @@ PROTO_DIR := internal/tasks/proto
PROTO_GEN_FILES := $(PROTO_DIR)/tasks.pb.go
.PHONY: poollink-dev poollink-dev-down poollink-dev-reset setup-tools fmt lint check-migrations proto check-proto \
up claim doctor cli seed-demo seed seed-plan sandbox sandbox-seed sandbox-simulate reset logs status stop down test-seed \
up upgrade claim doctor cli seed-demo seed seed-plan sandbox sandbox-seed sandbox-simulate reset logs status stop down test-seed \
restart restart-go restart-all infra infra-down app app-down app-logs \
backend forms forms-web consumer worker run dev tracking realtime web \
admin site docs grant-admin revoke-admin gen-key db-reset db-wipe migrate
admin site docs grant-admin revoke-admin gen-key installer-sha installer-check installer-demo \
db-reset db-wipe migrate warmbly warmbly-dist cli-sha cli-check
setup-tools:
@echo "Installing required Go tools into $(GO_BIN)"
@@ -34,6 +50,33 @@ setup-tools:
GOBIN=$(GO_BIN) go install google.golang.org/protobuf/cmd/protoc-gen-go@$(PROTOC_GEN_GO_VERSION)
GOBIN=$(GO_BIN) go install google.golang.org/grpc/cmd/protoc-gen-go-grpc@$(PROTOC_GEN_GO_GRPC_VERSION)
# Build the `warmbly` CLI into ./bin, stamped with this checkout's version so
# `warmbly version` reports something meaningful. This is the customer CLI; the
# operator one (warmblyctl) ships in the backend image and runs there.
warmbly:
@mkdir -p bin
go build -ldflags="-s -w \
-X github.com/warmbly/warmbly/internal/version.Version=$(WARMBLY_BUILD_VERSION) \
-X github.com/warmbly/warmbly/internal/version.Commit=$(WARMBLY_BUILD_COMMIT) \
-X github.com/warmbly/warmbly/internal/version.BuiltAt=$(WARMBLY_BUILD_TIME)" \
-o bin/warmbly ./cmd/cli
@echo "built bin/warmbly ($(WARMBLY_BUILD_VERSION))"
@echo "put it on your PATH: sudo install -m 0755 bin/warmbly /usr/local/bin/warmbly"
# Everything a release publishes for the CLI: an archive per platform, the
# checksums, and the Homebrew and Scoop manifests. Same script the release
# workflow runs, so an artifact can be reproduced locally.
warmbly-dist:
./scripts/build-cli.sh dist
# The published installer at https://warmbly.com/cli.sh. Regenerate the
# checksum after any edit to it; CI fails when the two disagree.
cli-sha:
@cd site/public && sha256sum cli.sh > cli.sh.sha256 && cat cli.sh.sha256
cli-check:
@./scripts/check-cli-installer.sh
# Format all Go code. CI's golangci-lint enforces gofmt, so this is the
# formatting signal to run before committing, not `go build`.
fmt:
@@ -79,7 +122,7 @@ ADMIN_URL = http://$(WEB_HOST):5174
# everything detached. Dashboard :5173, admin :5174, API :8080.
up:
@command -v docker >/dev/null || { echo "docker is required: https://docs.docker.com/get-docker/"; exit 1; }
$(COMPOSE) up -d --build
$(COMPOSE) $(UP_PROFILES) up -d --build
@echo ""
@echo "Warmbly is starting. The first run builds the images once."
@echo ""
@@ -88,6 +131,12 @@ up:
@echo " Health: make doctor Logs: make logs"
@echo " Demo data: make seed-demo Guide: https://docs.warmbly.com/development/first-run/"
# The by-hand equivalent of "Update and restart" in the admin panel: move the
# checkout forward, rebuild, recreate what changed. Migrations apply on boot.
upgrade:
git pull --ff-only
@$(MAKE) --no-print-directory up
# Report how to get into this instance, and end on a command that works.
#
# State is queried FIRST (warmblyctl, falling back to a direct count), so an
@@ -605,6 +654,7 @@ forms-web:
consumer:
$(GO_DEV_ENV) \
$(AI_DEV_ENV) \
GEODB_PATH=data/GeoLite2-City.mmdb \
go run ./cmd/consumer
# Send/sync worker. No Postgres by design. WORKER_ID is an explicit UUID
@@ -642,6 +692,24 @@ run:
gen-key:
@openssl rand -base64 32
# The one-command installer, served verbatim at https://warmbly.com/install.sh.
# The published checksum is what makes "download, verify, read, run" a real
# alternative to piping into a shell, so it is regenerated with the script and
# CI fails when the two disagree.
installer-sha:
@cd site/public && sha256sum install.sh > install.sh.sha256 && cat install.sh.sha256
# Everything CI runs against the installer: POSIX parse, shellcheck, --help,
# --print-env, a compose file per answer shape, and the checksum.
installer-check:
@./scripts/check-installer.sh
# Walk the installer's wizard without installing anything: the real questions,
# the real review, and a played pull and start. Writes no file, pulls no image,
# needs no Docker. This is the "what does it look like" target.
installer-demo:
@sh site/public/install.sh --demo
# ─── one-command dev stack ───────────────────────────────────────────────
#
# `make dev` is the "just make it work" target for a fresh clone or a fresh
+21 -5
View File
@@ -109,15 +109,29 @@ the native services, and how seeding works are in the
Warmbly runs with **no cloud account of any kind**: no AWS, no GCP, no Stripe, no
Kafka. One command brings up the whole platform on local, open-source pieces:
```bash
curl -fsSL https://warmbly.com/install.sh | sh
```
That pulls the published release images, writes a real `.env`, starts the stack
and prints the one-time link that claims the instance and makes you its admin.
No clone, no compiler, under two minutes. Add `--wizard` and it asks where each
store lives, what is kept and for how long, and how it is backed up. The script
is checksummed at `install.sh.sha256` if you would rather download, verify and
read it first, `--dry-run` prints every file it would write without touching
anything, and `--demo` walks the whole wizard while installing nothing. Full
flag list: [install](https://docs.warmbly.com/development/install/).
To build it from source instead:
```bash
git clone https://github.com/warmbly/warmbly && cd warmbly
make up
```
That is the whole install: `make up` waits for the backend and prints a one-time
link that claims the instance and makes you its admin. You need Docker with
Compose v2 and about 10 GB of free disk. If anything looks wrong, `make doctor`
prints the instance state and every failing check.
Either way you need Docker with Compose v2; the build path also wants about
10 GB of free disk. If anything looks wrong, `make doctor` prints the instance
state and every failing check.
From here the
[self-hosting guide](https://docs.warmbly.com/development/deployment-guide/)
@@ -134,7 +148,9 @@ The full docs live at **[docs.warmbly.com](https://docs.warmbly.com)**.
| Read this | To learn |
|-----------|----------|
| [Self-hosting guide](https://docs.warmbly.com/development/deployment-guide/) | Step-by-step install, then production, backups, and scaling the worker fleet |
| [Install](https://docs.warmbly.com/development/install/) | The one-command install and its wizard: where the data lives, what is kept, how it is backed up |
| [Self-hosting guide](https://docs.warmbly.com/development/deployment-guide/) | Building from source, then production, backups, and scaling the worker fleet |
| [Data control](https://docs.warmbly.com/development/data-control/) | Every store's location, every retention window, instance backups, and moving to another host |
| [First run](https://docs.warmbly.com/development/first-run/) | Claiming the instance, reissuing the setup link, and what to do when accounts already exist |
| [Accounts and access](https://docs.warmbly.com/development/accounts-and-access/) | Registration modes, inviting people with or without a mail relay, SSO, and recovering access |
| [`warmblyctl`](https://docs.warmbly.com/development/warmblyctl/) | The operator CLI: creating accounts, setting passwords, granting admin, and instance status |
+2
View File
@@ -5,6 +5,8 @@
# the heavy pnpm build runs only once.
FROM --platform=$BUILDPLATFORM node:22-alpine AS build
WORKDIR /app
# No TTY in a build: CI=true makes pnpm reinstall instead of prompting.
ENV CI=true
RUN corepack enable && corepack prepare pnpm@11.9.0 --activate
COPY package.json pnpm-lock.yaml pnpm-workspace.yaml ./
RUN pnpm install --frozen-lockfile
+1 -1
View File
@@ -27,7 +27,7 @@ const KNOWN_ACTIONS = [
];
const KNOWN_TARGETS = [
"worker", "aws_credentials", "worker_profile", "release",
"worker", "aws_credentials", "worker_profile", "release", "instance",
"user", "email_account", "campaign", "plan",
];
+70 -2
View File
@@ -2,14 +2,25 @@
// as decided by the running backend. The endpoint returns only checks that
// are not ok, so an empty response is a real all-clear and not a stub.
import { Link } from "react-router-dom";
import { AlertTriangle, CheckCircle2, Info, RefreshCw, XCircle } from "lucide-react";
import { useState } from "react";
import { Link, useSearchParams } from "react-router-dom";
import {
AlertTriangle,
ArrowUpCircle,
CheckCircle2,
Info,
Loader2,
RefreshCw,
XCircle,
} from "lucide-react";
import { PageHeader } from "@/components/layout/PageHeader";
import { ErrorState } from "@/components/ErrorState";
import { Button } from "@/components/ui/button";
import { Skeleton } from "@/components/ui/skeleton";
import { InstanceFindings } from "./InstanceHealthPanel";
import { UpdateDialog } from "@/components/layout/UpdateDialog";
import { useInstanceHealth } from "@/hooks/useInstanceHealth";
import { buildLabel, isUpdating, useUpdateState } from "@/hooks/useUpdateState";
import type { InstanceHealthSummary } from "@/lib/api/client/admin/instance";
export default function HealthPage() {
@@ -43,6 +54,8 @@ export default function HealthPage() {
</Button>
</PageHeader>
<UpdateCard />
{healthQ.isLoading && (
<div className="space-y-3">
<Skeleton className="h-12 w-full" />
@@ -89,6 +102,61 @@ export default function HealthPage() {
);
}
// Version and update status, above the findings: the same facts as the pill
// in the top bar, on the page an operator opens to ask "is this instance ok".
function UpdateCard() {
const updateQ = useUpdateState();
// ?update=1 is how the dashboard's version pill deep-links an admin
// straight into the dialog.
const [params] = useSearchParams();
const [open, setOpen] = useState(params.get("update") === "1");
const state = updateQ.data;
if (!state) return null;
const updating = isUpdating(state);
const available = state.update_available;
const tone = updating
? "border-sky-200 bg-sky-50/60"
: available
? "border-amber-200 bg-amber-50/60"
: "border-border bg-white";
return (
<div className={`mb-5 flex flex-wrap items-center gap-3 rounded-lg border p-3 ${tone}`}>
{updating ? (
<Loader2 className="size-4 shrink-0 animate-spin text-sky-600" />
) : available ? (
<ArrowUpCircle className="size-4 shrink-0 text-amber-600" />
) : (
<CheckCircle2 className="size-4 shrink-0 text-emerald-600" />
)}
<div className="min-w-0 flex-1 text-[13px]">
<span className="font-semibold text-foreground">
{updating
? "Updating this instance"
: available
? `${state.latest?.tag && state.reason === "release" ? state.latest.tag : "A newer version"} is available`
: "Up to date"}
</span>
<span className="text-muted-foreground">
{" "}
running {buildLabel(state)}
{state.updater.checkout && !state.updater.checkout.detached
? ` on ${state.updater.checkout.branch}`
: ""}
{state.checked_at
? `, checked ${new Date(state.checked_at).toLocaleTimeString()}`
: ""}
</span>
</div>
<Button size="sm" variant={available ? "default" : "outline"} onClick={() => setOpen(true)}>
{updating ? "Progress" : available ? "Update" : "Details"}
</Button>
<UpdateDialog open={open} onOpenChange={setOpen} />
</div>
);
}
function SummaryStrip({
summary,
total,
@@ -76,11 +76,57 @@ const SYNC_FIELDS = [
type SyncFieldKey = (typeof SYNC_FIELDS)[number]["key"];
// Retention windows, mirroring internal/config/constants.go. Each one is also
// how long the personal data in that log is held, so the help text says what
// the data is rather than only what the number does.
const RETENTION_MIN_DAYS = 1;
const RETENTION_MAX_DAYS = 3650;
const RETENTION_FIELDS = [
{
key: "engagementDays",
setting: "engagement_event_days",
label: "Opens and clicks (days)",
help: "Per-event open and click logs, with the client, device and approximate location of each. Campaign counts and routing read a separate summary that is never pruned, so shortening this changes what a contact's timeline can show, not what a campaign does.",
},
{
key: "formDays",
setting: "form_event_days",
label: "Form funnel events (days)",
help: "Views, starts, field-level drop-off and submissions for hosted forms. Funnel reports range up to 90 days, so anything below that shortens the report too. Submitted contacts are unaffected.",
},
{
key: "auditDays",
setting: "audit_log_days",
label: "Audit log (days)",
help: "Who did what, from which IP address and user agent, with the change payload. This window is how long that record is held, and it is the one most likely to be set by a retention policy.",
},
] as const;
type RetentionFieldKey = (typeof RETENTION_FIELDS)[number]["key"];
// The two presets are the ends of the band people actually choose between.
const RETENTION_PRESETS = [
{
id: "default",
label: "Defaults",
description: "365 / 180 / 90 days",
values: { engagementDays: "365", formDays: "180", auditDays: "90" },
},
{
id: "minimal",
label: "Minimal retention",
description: "30 / 30 / 30 days",
values: { engagementDays: "30", formDays: "30", auditDays: "30" },
},
] as const;
interface FormState {
linksEnabled: boolean;
ttlHours: string;
allowInvitedSignup: boolean;
sync: Record<SyncFieldKey, string>;
retention: Record<RetentionFieldKey, string>;
enforceDomainAuth: boolean;
authGraceHours: string;
}
@@ -96,6 +142,11 @@ function toForm(s: InstanceSettings): FormState {
dailyPerMailbox: String(s.sync.daily_messages_per_mailbox),
dailyPerOrg: String(s.sync.daily_messages_per_org),
},
retention: {
engagementDays: String(s.retention.engagement_event_days),
formDays: String(s.retention.form_event_days),
auditDays: String(s.retention.audit_log_days),
},
enforceDomainAuth: s.deliverability.enforce_domain_auth,
authGraceHours: String(s.deliverability.auth_grace_hours),
};
@@ -137,6 +188,12 @@ export default function InstanceSettingsPage() {
!!server &&
!!form &&
SYNC_FIELDS.some((f) => form.sync[f.key] !== String(server.sync[f.setting]));
const retentionDirty =
!!server &&
!!form &&
RETENTION_FIELDS.some(
(f) => form.retention[f.key] !== String(server.retention[f.setting]),
);
const dirty =
!!server &&
!!form &&
@@ -145,9 +202,15 @@ export default function InstanceSettingsPage() {
form.allowInvitedSignup !== server.access.allow_invited_signup ||
form.enforceDomainAuth !== server.deliverability.enforce_domain_auth ||
form.authGraceHours !== String(server.deliverability.auth_grace_hours) ||
retentionDirty ||
syncDirty);
const syncValid =
form !== null && SYNC_FIELDS.every((f) => syncFieldValid(form.sync[f.key], f.min, f.max));
const retentionValid =
form !== null &&
RETENTION_FIELDS.every((f) =>
syncFieldValid(form.retention[f.key], RETENTION_MIN_DAYS, RETENTION_MAX_DAYS),
);
const authGrace = form ? Number(form.authGraceHours) : NaN;
const authGraceValid =
@@ -177,6 +240,12 @@ export default function InstanceSettingsPage() {
toast.error("Every sync budget must be a whole number inside its range");
return;
}
if (!retentionValid) {
toast.error(
`Every retention window must be a whole number of days between ${RETENTION_MIN_DAYS} and ${RETENTION_MAX_DAYS.toLocaleString()}`,
);
return;
}
if (!authGraceValid) {
toast.error(
`The authentication grace period must be a whole number of hours between ${AUTH_GRACE_MIN_HOURS} and ${AUTH_GRACE_MAX_HOURS}`,
@@ -192,6 +261,11 @@ export default function InstanceSettingsPage() {
daily_messages_per_mailbox: Number(form.sync.dailyPerMailbox),
daily_messages_per_org: Number(form.sync.dailyPerOrg),
},
retention: {
engagement_event_days: Number(form.retention.engagementDays),
form_event_days: Number(form.retention.formDays),
audit_log_days: Number(form.retention.auditDays),
},
deliverability: {
enforce_domain_auth: form.enforceDomainAuth,
auth_grace_hours: authGrace,
@@ -370,6 +444,93 @@ export default function InstanceSettingsPage() {
</CardContent>
</Card>
<Card className="lg:col-span-2">
<CardHeader>
<CardTitle>Data retention</CardTitle>
<CardDescription>
How long event-level history is kept on this instance. Every window
below is also how long the personal data in that log is held, so
these are the settings a retention or privacy policy applies to. A
sweep runs a few times a day and reads these values each pass, so a
change takes effect without a restart. Deletion is permanent:
shortening a window removes what already sits outside it on the
next sweep.
</CardDescription>
</CardHeader>
<CardContent className="space-y-4 pt-0">
<div className="flex flex-wrap items-center gap-2">
<span className="text-xs text-muted-foreground">Presets</span>
{RETENTION_PRESETS.map((preset) => {
const active = RETENTION_FIELDS.every(
(f) => form.retention[f.key] === preset.values[f.key],
);
return (
<Button
key={preset.id}
type="button"
size="sm"
variant={active ? "default" : "outline"}
onClick={() =>
setForm({
...form,
retention: { ...preset.values },
})
}
>
{preset.label}
<span className="ml-1.5 text-[11px] opacity-70">
{preset.description}
</span>
</Button>
);
})}
</div>
<div className="grid grid-cols-1 gap-3 md:grid-cols-3">
{RETENTION_FIELDS.map((f) => {
const valid = syncFieldValid(
form.retention[f.key],
RETENTION_MIN_DAYS,
RETENTION_MAX_DAYS,
);
return (
<div key={f.key}>
<Label htmlFor={`retention-${f.key}`}>{f.label}</Label>
<Input
id={`retention-${f.key}`}
type="text"
inputMode="numeric"
autoComplete="off"
value={form.retention[f.key]}
onChange={(e) =>
setForm({
...form,
retention: {
...form.retention,
[f.key]: e.target.value,
},
})
}
aria-invalid={!valid}
className="mt-1"
/>
<p className="mt-1 text-xs text-muted-foreground">
{f.help} Between {RETENTION_MIN_DAYS} and{" "}
{RETENTION_MAX_DAYS.toLocaleString()} days.
</p>
{!valid && (
<p className="mt-1 text-xs text-red-600">
Enter a whole number of days between{" "}
{RETENTION_MIN_DAYS} and{" "}
{RETENTION_MAX_DAYS.toLocaleString()}.
</p>
)}
</div>
);
})}
</div>
</CardContent>
</Card>
<Card className="lg:col-span-2">
<CardHeader>
<CardTitle>Sending-domain authentication</CardTitle>
@@ -0,0 +1,433 @@
// Operator notification channels: where this deployment tells its operator
// that something happened. Discord, Slack, a generic signed webhook, or email.
//
// The channels live in the same instance settings document as the rest of the
// writable configuration, so this page reads and writes /admin/instance/settings
// and only reaches for its own endpoints for the event catalog and the test
// delivery probe.
//
// Targets and secrets come back redacted. An unchanged field is sent back as
// the preview the server returned (or empty) and resolves to the stored value,
// so saving an unrelated toggle can never wipe a credential.
import { useEffect, useMemo, useState } from "react";
import { useMutation, useQuery, useQueryClient } from "@tanstack/react-query";
import { toast } from "sonner";
import {
Bell,
Check,
Hash,
Mail,
Plus,
Save,
Send,
Trash2,
Webhook,
} from "lucide-react";
import { PageHeader } from "@/components/layout/PageHeader";
import { ErrorState } from "@/components/ErrorState";
import { Button } from "@/components/ui/button";
import {
Card,
CardContent,
CardDescription,
CardHeader,
CardTitle,
} from "@/components/ui/card";
import { Badge } from "@/components/ui/badge";
import { Checkbox } from "@/components/ui/checkbox";
import { Input } from "@/components/ui/input";
import { Label } from "@/components/ui/label";
import { Skeleton } from "@/components/ui/skeleton";
import { Switch } from "@/components/ui/switch";
import {
getInstanceSettings,
getNotificationEvents,
putInstanceSettings,
testNotificationChannel,
type InstanceSettings,
type NotifyChannel,
type NotifyChannelType,
type NotifyEventDef,
} from "@/lib/api/client/admin/instance";
const SETTINGS_KEY = ["admin", "instance", "settings"];
const EVENTS_KEY = ["admin", "instance", "notification-events"];
const TYPES: {
value: NotifyChannelType;
label: string;
icon: typeof Hash;
placeholder: string;
help: string;
}[] = [
{
value: "discord",
label: "Discord",
icon: Hash,
placeholder: "https://discord.com/api/webhooks/…",
help: "Server settings, then Integrations, then New Webhook. Copy the webhook URL.",
},
{
value: "slack",
label: "Slack",
icon: Hash,
placeholder: "https://hooks.slack.com/services/…",
help: "Create a Slack app with an incoming webhook and copy its URL.",
},
{
value: "webhook",
label: "Webhook",
icon: Webhook,
placeholder: "https://example.com/hooks/warmbly",
help: "Receives the event as JSON. Set a secret to have it signed with HMAC-SHA256.",
},
{
value: "email",
label: "Email",
icon: Mail,
placeholder: "ops@example.com",
help: "Needs a working platform mail transport on this deployment.",
},
];
function typeDef(t: NotifyChannelType) {
return TYPES.find((x) => x.value === t) ?? TYPES[0];
}
function newChannel(): NotifyChannel {
return {
// The server assigns a real id on save; this one only has to be unique
// within the unsaved list so React can key it.
id: `new-${Math.random().toString(36).slice(2, 10)}`,
name: "",
type: "discord",
target: "",
secret: "",
events: [],
enabled: true,
};
}
export default function NotificationsPage() {
const queryClient = useQueryClient();
const settings = useQuery({ queryKey: SETTINGS_KEY, queryFn: getInstanceSettings });
const catalog = useQuery({ queryKey: EVENTS_KEY, queryFn: getNotificationEvents });
const [channels, setChannels] = useState<NotifyChannel[] | null>(null);
const [testing, setTesting] = useState<string | null>(null);
useEffect(() => {
if (settings.data) {
setChannels(settings.data.notifications?.channels ?? []);
}
}, [settings.data]);
const save = useMutation({
mutationFn: (next: NotifyChannel[]) => {
const base = settings.data as InstanceSettings;
return putInstanceSettings({
...base,
notifications: { channels: next },
});
},
onSuccess: (doc) => {
queryClient.setQueryData(SETTINGS_KEY, doc);
setChannels(doc.notifications?.channels ?? []);
toast.success("Notification channels saved");
},
onError: (e: Error) => toast.error(e.message || "Could not save channels"),
});
const groups = useMemo(() => {
const events = catalog.data?.events ?? [];
const order: string[] = [];
const byGroup = new Map<string, NotifyEventDef[]>();
for (const e of events) {
if (!byGroup.has(e.group)) {
byGroup.set(e.group, []);
order.push(e.group);
}
byGroup.get(e.group)!.push(e);
}
return order.map((g) => ({ group: g, events: byGroup.get(g)! }));
}, [catalog.data]);
if (settings.isError) {
return <ErrorState error={settings.error as Error} onRetry={() => settings.refetch()} />;
}
function update(id: string, patch: Partial<NotifyChannel>) {
setChannels((prev) => (prev ?? []).map((c) => (c.id === id ? { ...c, ...patch } : c)));
}
function remove(id: string) {
setChannels((prev) => (prev ?? []).filter((c) => c.id !== id));
}
function toggleEvent(id: string, key: string, on: boolean) {
setChannels((prev) =>
(prev ?? []).map((c) => {
if (c.id !== id) return c;
const next = on ? [...c.events, key] : c.events.filter((e) => e !== key);
return { ...c, events: next };
}),
);
}
async function sendTest(ch: NotifyChannel) {
setTesting(ch.id);
try {
// A saved channel is tested by id so the server uses its stored
// credential; an unsaved one carries its fields inline.
const saved = !ch.id.startsWith("new-");
await testNotificationChannel(
saved
? { id: ch.id }
: { type: ch.type, name: ch.name, target: ch.target, secret: ch.secret },
);
toast.success("Test alert delivered");
} catch (e) {
toast.error((e as Error).message || "Delivery failed");
} finally {
setTesting(null);
}
}
const list = channels ?? [];
const dirty =
!!settings.data &&
JSON.stringify(list) !== JSON.stringify(settings.data.notifications?.channels ?? []);
// Changing a channel's type clears its target on purpose, so a save with
// one still empty would drop the channel server-side. Block it here and
// say which one needs attention.
const incomplete = list.filter((c) => !c.target.trim());
return (
<div className="space-y-6">
<PageHeader
title="Notifications"
description="Where this instance tells you something happened. Add a Discord or Slack webhook, a signed endpoint, or an address."
>
<Button
variant="outline"
size="sm"
onClick={() => setChannels([...(channels ?? []), newChannel()])}
>
<Plus className="h-4 w-4" />
Add channel
</Button>
<Button
size="sm"
disabled={!dirty || incomplete.length > 0 || save.isPending}
title={
incomplete.length > 0
? "Every channel needs a destination before you can save"
: undefined
}
onClick={() => save.mutate(list)}
>
<Save className="h-4 w-4" />
{save.isPending ? "Saving…" : "Save"}
</Button>
</PageHeader>
{settings.isLoading ? (
<Skeleton className="h-64 w-full" />
) : list.length === 0 ? (
<Card>
<CardContent className="py-10 text-center">
<Bell className="mx-auto mb-3 h-8 w-8 text-muted-foreground" />
<p className="text-sm font-medium">No channels yet</p>
<p className="mx-auto mt-1 max-w-md text-sm text-muted-foreground">
Nothing is being sent anywhere. Add a channel and pick the events it
should receive; leave every event unchecked to receive all of them.
</p>
<Button
className="mt-4"
size="sm"
onClick={() => setChannels([newChannel()])}
>
<Plus className="h-4 w-4" />
Add a channel
</Button>
</CardContent>
</Card>
) : (
<div className="space-y-4">
{list.map((ch) => {
const def = typeDef(ch.type);
const Icon = def.icon;
const saved = !ch.id.startsWith("new-");
return (
<Card key={ch.id}>
<CardHeader className="pb-3">
<div className="flex flex-wrap items-start justify-between gap-3">
<div className="flex items-start gap-3">
<span className="mt-0.5 flex h-8 w-8 items-center justify-center rounded-md bg-muted">
<Icon className="h-4 w-4" />
</span>
<div>
<CardTitle className="text-base">
{ch.name || def.label}
</CardTitle>
<CardDescription>
{ch.events.length === 0
? "Receives every event"
: `Receives ${ch.events.length} event${ch.events.length === 1 ? "" : "s"}`}
{!saved && " · unsaved"}
</CardDescription>
</div>
</div>
<div className="flex items-center gap-2">
{!ch.enabled && <Badge variant="outline">Off</Badge>}
<Switch
checked={ch.enabled}
onCheckedChange={(v) => update(ch.id, { enabled: v })}
/>
<Button
variant="outline"
size="sm"
disabled={testing === ch.id || !ch.target}
onClick={() => sendTest(ch)}
>
<Send className="h-4 w-4" />
{testing === ch.id ? "Sending…" : "Test"}
</Button>
<Button
variant="ghost"
size="sm"
onClick={() => remove(ch.id)}
>
<Trash2 className="h-4 w-4" />
</Button>
</div>
</div>
</CardHeader>
<CardContent className="space-y-4">
<div className="grid gap-4 md:grid-cols-3">
<div className="space-y-1.5">
<Label>Type</Label>
<div className="flex flex-wrap gap-1.5">
{TYPES.map((t) => (
<Button
key={t.value}
type="button"
variant={ch.type === t.value ? "default" : "outline"}
size="sm"
onClick={() =>
update(ch.id, { type: t.value, target: "" })
}
>
{t.label}
</Button>
))}
</div>
</div>
<div className="space-y-1.5">
<Label htmlFor={`name-${ch.id}`}>Name</Label>
<Input
id={`name-${ch.id}`}
value={ch.name}
placeholder={def.label}
onChange={(e) => update(ch.id, { name: e.target.value })}
/>
</div>
<div className="space-y-1.5">
<Label htmlFor={`target-${ch.id}`}>
{ch.type === "email" ? "Address" : "Webhook URL"}
</Label>
<Input
id={`target-${ch.id}`}
value={ch.target}
placeholder={def.placeholder}
onChange={(e) => update(ch.id, { target: e.target.value })}
/>
{ch.target.trim() ? (
<p className="text-xs text-muted-foreground">{def.help}</p>
) : (
<p className="text-xs text-destructive">
{ch.type === "email"
? "Enter an address before saving."
: "Enter the webhook URL for this transport before saving."}
</p>
)}
</div>
</div>
{ch.type === "webhook" && (
<div className="space-y-1.5 md:max-w-sm">
<Label htmlFor={`secret-${ch.id}`}>Signing secret</Label>
<Input
id={`secret-${ch.id}`}
value={ch.secret ?? ""}
placeholder="Optional"
onChange={(e) => update(ch.id, { secret: e.target.value })}
/>
<p className="text-xs text-muted-foreground">
Signs the body as{" "}
<code>X-Warmbly-Signature: t=&lt;unix&gt;,v1=&lt;hex&gt;</code>, the
same scheme customer webhooks use.
</p>
</div>
)}
<div className="space-y-2">
<div className="flex items-center justify-between">
<Label>Events</Label>
<button
type="button"
className="text-xs text-muted-foreground hover:text-foreground"
onClick={() => update(ch.id, { events: [] })}
>
Receive everything
</button>
</div>
{catalog.isLoading ? (
<Skeleton className="h-24 w-full" />
) : (
<div className="grid gap-4 md:grid-cols-2">
{groups.map(({ group, events }) => (
<div key={group} className="space-y-2">
<p className="text-xs font-medium uppercase tracking-wide text-muted-foreground">
{group}
</p>
{events.map((e) => (
<label
key={e.key}
className="flex cursor-pointer items-start gap-2"
>
<Checkbox
checked={ch.events.includes(e.key)}
onCheckedChange={(v) =>
toggleEvent(ch.id, e.key, v === true)
}
/>
<span className="text-sm leading-tight">
{e.label}
<span className="block text-xs text-muted-foreground">
{e.description}
</span>
</span>
</label>
))}
</div>
))}
</div>
)}
{ch.events.length === 0 && (
<p className="flex items-center gap-1.5 text-xs text-muted-foreground">
<Check className="h-3 w-3" />
Nothing selected, so this channel receives every event.
</p>
)}
</div>
</CardContent>
</Card>
);
})}
</div>
)}
</div>
);
}
+7
View File
@@ -5,6 +5,7 @@
import { NavLink } from "react-router-dom";
import {
Bell,
Activity,
BarChart3,
Building2,
@@ -102,6 +103,12 @@ const GROUPS: NavGroup[] = [
icon: Settings2,
perm: AdminPerm.ManageSettings,
},
{
to: "/configuration/notifications",
label: "Notifications",
icon: Bell,
perm: AdminPerm.ManageSettings,
},
{
to: "/limits",
label: "Effective limits",
+5 -2
View File
@@ -1,9 +1,11 @@
// Sticky top bar. Holds the env pill, a search slot (unused for now —
// reserved for the cmd-K palette we'll add later), and the user menu.
// Sticky top bar. Holds the env pill, the version pill (which becomes the
// update indicator), a search slot (unused for now, reserved for the cmd-K
// palette we'll add later), and the user menu.
// The 3px admin stripe sits *above* this bar in AppShell so it's the
// first thing the eye lands on.
import { EnvPill } from "./EnvPill";
import { UpdatePill } from "./UpdatePill";
import { UserMenu } from "./UserMenu";
export function Topbar() {
@@ -12,6 +14,7 @@ export function Topbar() {
<div className="h-full flex items-center gap-3 px-4">
<div className="flex items-center gap-2">
<EnvPill />
<UpdatePill />
<span className="text-xs text-muted-foreground hidden sm:inline">
Connected to <code className="text-foreground font-mono">/admin/*</code>
</span>
@@ -0,0 +1,569 @@
// The update dialog behind the version pill: what is running, what is newest,
// and the one button that pulls, rebuilds and restarts through the updater.
// While a job runs it shows the steps and the live log; when the backend goes
// away for the restart it says so and keeps polling until it is back.
import { useEffect, useMemo, useRef, useState } from "react";
import { useMutation, useQueryClient } from "@tanstack/react-query";
import { toast } from "sonner";
import {
AlertTriangle,
Check,
CheckCircle2,
ExternalLink,
GitBranch,
Loader2,
Package,
RefreshCw,
RotateCw,
XCircle,
} from "lucide-react";
import {
Dialog,
DialogContent,
DialogDescription,
DialogFooter,
DialogHeader,
DialogTitle,
} from "@/components/ui/dialog";
import { Button } from "@/components/ui/button";
import { Badge } from "@/components/ui/badge";
import { docsUrl } from "@/lib/docs";
import { cn } from "@/lib/utils";
import { useAdminPerm } from "@/hooks/useAdminPerm";
import { AdminPerm } from "@/lib/auth/permissions";
import {
UPDATE_JOB_KEY,
UPDATE_STATE_KEY,
buildLabel,
isUpdating,
useUpdateJob,
useUpdateState,
} from "@/hooks/useUpdateState";
import {
applyUpdate,
checkForUpdates,
type UpdateJob,
type UpdateState,
} from "@/lib/api/client/admin/updates";
import { markUpdateStarted, readUpdateStarted } from "@/lib/updateSession";
const DOCS_UPDATES = "/development/updates/";
const STEPS: Record<string, string[]> = {
compose: ["fetch", "checkout", "build", "restart", "prune", "wait"],
image: ["resolve", "pull", "restart", "prune", "wait"],
command: ["fetch", "checkout", "command", "wait"],
};
const STEP_LABELS: Record<string, string> = {
fetch: "Fetch",
checkout: "Pull",
build: "Build",
resolve: "Pin release",
pull: "Pull images",
restart: "Restart",
prune: "Clean up",
command: "Run script",
wait: "Wait for backend",
starting: "Starting",
};
type Phase = "idle" | "running" | "restarting" | "done" | "failed";
interface Props {
open: boolean;
onOpenChange: (open: boolean) => void;
}
export function UpdateDialog({ open, onOpenChange }: Props) {
const qc = useQueryClient();
const canManage = useAdminPerm(AdminPerm.ManageSettings);
const stateQ = useUpdateState();
const jobQ = useUpdateJob(open);
const state: UpdateState | undefined = jobQ.data ?? stateQ.data;
const [confirming, setConfirming] = useState(false);
const started = readUpdateStarted();
const phase: Phase = useMemo(() => {
if (isUpdating(state)) return "running";
if (started && (jobQ.isError || stateQ.isError)) return "restarting";
const last = state?.updater.last_job;
if (started && last && last.status !== "running") {
return last.status === "succeeded" ? "done" : "failed";
}
return "idle";
}, [state, started, jobQ.isError, stateQ.isError]);
const canApply =
canManage &&
state?.updater.status === "ok" &&
!!state?.update_available &&
!state?.updater.checkout?.dirty &&
phase === "idle";
// Leaving the confirmation open once the update can no longer start
// (a phase change, or the checkout turning dirty) would let a stale
// click start a job that fails.
useEffect(() => {
if (!canApply) setConfirming(false);
}, [canApply]);
const checkMut = useMutation({
mutationFn: checkForUpdates,
onSuccess: (data) => {
qc.setQueryData(UPDATE_STATE_KEY, data);
qc.setQueryData(UPDATE_JOB_KEY, data);
toast.success(
data.update_available ? "A newer version is available" : "This instance is up to date",
);
},
onError: (err: Error) => toast.error(err.message || "Could not check for updates"),
});
const applyMut = useMutation({
mutationFn: () => {
// The state keeps refreshing while the confirmation is open; a
// checkout that turned dirty meanwhile must not start a job.
if (!canApply) return Promise.reject(new Error("The update can no longer start; check the status above."));
return applyUpdate("latest");
},
onSuccess: (job: UpdateJob) => {
markUpdateStarted(state?.running.version ?? "", state?.running.commit ?? job.from_commit);
setConfirming(false);
toast.success("Update started");
void qc.invalidateQueries({ queryKey: UPDATE_STATE_KEY });
void qc.invalidateQueries({ queryKey: UPDATE_JOB_KEY });
},
onError: (err: Error) => toast.error(err.message || "Could not start the update"),
});
const updater = state?.updater;
const checkout = updater?.checkout;
const job = updater?.job ?? updater?.last_job;
// An image install never builds, so the confirmation must not promise a
// rebuild it will not do.
const imageMode = updater?.mode === "image";
return (
<Dialog open={open} onOpenChange={onOpenChange}>
<DialogContent className="sm:max-w-xl">
<DialogHeader>
<DialogTitle>Updates</DialogTitle>
<DialogDescription>
{state
? `Running ${buildLabel(state)}${state.running.commit ? ` (${state.running.commit.slice(0, 7)})` : ""}.`
: "Reading the running version."}
</DialogDescription>
</DialogHeader>
{phase === "idle" && state && (
<div className="space-y-3">
<Overview state={state} />
{updater?.status !== "ok" && <UpdaterNotice state={state} />}
{checkout?.dirty && (
<Notice tone="warning">
The checkout has local modifications. The updater refuses to move it
until they are committed or stashed, or `UPDATER_ALLOW_DIRTY=true`.
</Notice>
)}
{job && job.status === "failed" && !started && (
<Notice tone="error">
The last update failed at step {STEP_LABELS[job.step] ?? job.step}:{" "}
{job.error}
</Notice>
)}
{confirming && (
<div className="animate-in fade-in slide-in-from-bottom-1 duration-200">
<Notice tone="warning">
<div className="font-medium text-foreground">
{imageMode
? "This pulls the release images and restarts every service."
: "This pulls the checkout, rebuilds the images and restarts every service."}
</div>
<div className="mt-1">
Sending and syncing pause for a few minutes and resume on their own;
nothing in flight is lost. Migrations apply when the backend comes
back. This panel reconnects by itself.
</div>
</Notice>
</div>
)}
</div>
)}
{(phase === "running" || phase === "restarting") && (
<Progress state={state} job={job} restarting={phase === "restarting"} />
)}
{phase === "done" && state && (
<div className="space-y-3 animate-in fade-in zoom-in-95 duration-300">
<div className="flex items-start gap-3 rounded-lg border border-emerald-200 bg-emerald-50 p-3">
<CheckCircle2 className="mt-0.5 size-4 shrink-0 text-emerald-600 animate-in zoom-in-50 duration-300" />
<div className="text-[13px] leading-relaxed text-emerald-800">
<div className="font-semibold">Updated to {buildLabel(state)}</div>
Every service is back and sending has resumed. Reload to pick up the
new admin panel.
</div>
</div>
{job?.log && <LogPanel lines={job.log} />}
</div>
)}
{phase === "failed" && (
<div className="space-y-3 animate-in fade-in duration-200">
<Notice tone="error">
<div className="font-semibold text-foreground">The update failed</div>
{job?.error ?? "See the log below."} The previous version is still running
unless the restart step had already begun.
</Notice>
{job?.log && <LogPanel lines={job.log} />}
</div>
)}
<DialogFooter className="gap-2 sm:justify-between">
<a
href={docsUrl(DOCS_UPDATES)}
target="_blank"
rel="noreferrer"
className="inline-flex items-center gap-1 text-xs font-medium text-[var(--admin-accent-strong)] hover:underline"
>
How updates work
<ExternalLink className="size-3" />
</a>
<div className="flex flex-wrap items-center gap-2">
{phase === "idle" && canManage && (
<Button
size="sm"
variant="outline"
onClick={() => checkMut.mutate()}
disabled={checkMut.isPending}
>
<RefreshCw className={cn("size-4", checkMut.isPending && "animate-spin")} />
{checkMut.isPending ? "Checking..." : "Check now"}
</Button>
)}
{phase === "idle" && canApply && !confirming && (
<Button size="sm" onClick={() => setConfirming(true)}>
<RotateCw className="size-4" />
Update and restart
</Button>
)}
{phase === "idle" && confirming && (
<>
<Button size="sm" variant="ghost" onClick={() => setConfirming(false)}>
Cancel
</Button>
<Button
size="sm"
onClick={() => applyMut.mutate()}
disabled={applyMut.isPending}
>
{applyMut.isPending ? (
<Loader2 className="size-4 animate-spin" />
) : (
<RotateCw className="size-4" />
)}
Update now
</Button>
</>
)}
{(phase === "done" || phase === "failed") && (
<Button size="sm" onClick={() => window.location.reload()}>
Reload
</Button>
)}
{(phase === "running" || phase === "restarting") && (
<Button size="sm" variant="ghost" onClick={() => onOpenChange(false)}>
Keep running in the background
</Button>
)}
</div>
</DialogFooter>
</DialogContent>
</Dialog>
);
}
function Overview({ state }: { state: UpdateState }) {
const { latest, updater } = state;
const checkout = updater.checkout;
const release = updater.release;
return (
<dl className="grid grid-cols-[8rem_1fr] gap-x-3 gap-y-2 text-[13px]">
<dt className="text-muted-foreground">Latest release</dt>
<dd>
{latest ? (
<span className="inline-flex flex-wrap items-center gap-2">
<span className="font-medium">{latest.tag}</span>
{latest.published_at && (
<span className="text-muted-foreground">
{new Date(latest.published_at).toLocaleDateString()}
</span>
)}
{latest.html_url && (
<a
href={latest.html_url}
target="_blank"
rel="noreferrer"
className="inline-flex items-center gap-1 text-xs font-medium text-[var(--admin-accent-strong)] hover:underline"
>
Release notes
<ExternalLink className="size-3" />
</a>
)}
</span>
) : state.check_error ? (
<span className="text-amber-700">Could not read releases: {state.check_error}</span>
) : state.enabled ? (
<span className="text-muted-foreground">No release found for {state.repo}</span>
) : (
<span className="text-muted-foreground">Release check is off</span>
)}
</dd>
<dt className="text-muted-foreground">Status</dt>
<dd>
{state.update_available ? (
<Badge variant="outline" className="border-amber-300 bg-amber-50 text-amber-700">
Update available
</Badge>
) : (
<Badge variant="outline" className="border-emerald-300 bg-emerald-50 text-emerald-700">
Up to date
</Badge>
)}
{state.checked_at && (
<span className="ml-2 text-xs text-muted-foreground">
checked {new Date(state.checked_at).toLocaleTimeString()}, every{" "}
{state.interval}
</span>
)}
</dd>
{release && (
<>
<dt className="text-muted-foreground">Installed</dt>
<dd className="flex flex-wrap items-center gap-2">
<span className="inline-flex items-center gap-1 font-mono text-xs">
<Package className="size-3.5 text-muted-foreground" />
{release.prefix}/*:{release.tag}
</span>
<span className="text-muted-foreground">
{release.pinned
? "pinned to this release"
: "following the channel tag"}
</span>
</dd>
</>
)}
{checkout && (
<>
<dt className="text-muted-foreground">Checkout</dt>
<dd className="flex flex-wrap items-center gap-2">
<span className="inline-flex items-center gap-1 font-mono text-xs">
<GitBranch className="size-3.5 text-muted-foreground" />
{checkout.detached ? "pinned" : checkout.branch}@{checkout.commit.slice(0, 7)}
</span>
{!checkout.detached && (
<span className="text-muted-foreground">
{checkout.behind > 0
? `${checkout.behind} commit${checkout.behind === 1 ? "" : "s"} behind`
: "matches the remote"}
</span>
)}
{checkout.fetch_error && (
<span className="text-amber-700">fetch failed: {checkout.fetch_error}</span>
)}
</dd>
</>
)}
<dt className="text-muted-foreground">Updater</dt>
<dd>
{updater.status === "ok" && (
<span>
ready
<span className="text-muted-foreground"> ({updater.mode} mode)</span>
</span>
)}
{updater.status === "off" && <span className="text-muted-foreground">not configured</span>}
{updater.status === "unreachable" && <span className="text-amber-700">unreachable</span>}
</dd>
</dl>
);
}
function UpdaterNotice({ state }: { state: UpdateState }) {
const u = state.updater;
// A clone-free install has no checkout to pull, so the by-hand command is
// the compose one. Which install this is comes from the release block, and
// an unreachable updater reports neither block, so that case names both
// rather than printing one command that may not work.
//
// A pinned install needs the tag edited first: pulling again resolves the
// same fixed version and updates nothing, which is the kind of instruction
// that looks like it worked.
const pinnedTag = u.release?.pinned ? (state.latest?.tag ?? "vX.Y.Z") : null;
const byHand = u.release
? "docker compose pull && docker compose up -d"
: u.checkout
? "git pull && make up"
: null;
if (u.status === "unreachable") {
return (
<Notice tone="warning">
<div className="font-medium text-foreground">The updater is not answering</div>
{u.error} Until it does, update by hand from the install directory:
{pinnedTag && <SetTagFirst tag={pinnedTag} />}
{byHand ? <Cmd>{byHand}</Cmd> : <BothCommands />}
</Notice>
);
}
return (
<Notice tone="info">
<div className="font-medium text-foreground">This panel can only report</div>
No updater is configured, so apply updates from a shell on the host:
{pinnedTag && <SetTagFirst tag={pinnedTag} />}
{byHand ? <Cmd>{byHand}</Cmd> : <BothCommands />}
To get the button, enable the updater compose profile (`make up` and the installer
both do) or run the updater unit on a bare-metal host.
</Notice>
);
}
// An image install pinned to a fixed version resolves the same images on every
// pull, so moving it is an edit to .env and then the pull. Without this the
// command below reports success and changes nothing.
function SetTagFirst({ tag }: { tag: string }) {
return (
<>
<div className="mt-1.5 text-xs">
This install is pinned, so set the release in <code>.env</code> first:
</div>
<Cmd>{`WARMBLY_TAG=${tag}`}</Cmd>
<div className="text-xs">then:</div>
</>
);
}
// Shown when the updater is unreachable, so nothing says which shape of install
// this is. Naming both beats guessing: the wrong one fails confusingly on an
// install that has no checkout, or no images to pull.
function BothCommands() {
return (
<>
<div className="mt-1.5 text-xs">From an install.sh install:</div>
<Cmd>docker compose pull && docker compose up -d</Cmd>
<div className="text-xs">From a git checkout:</div>
<Cmd>git pull && make up</Cmd>
</>
);
}
function Progress({
state,
job,
restarting,
}: {
state: UpdateState | undefined;
job: UpdateJob | undefined;
restarting: boolean;
}) {
const mode = state?.updater.mode ?? "compose";
const steps = STEPS[mode] ?? STEPS.compose;
const current = restarting ? "wait" : (job?.step ?? "starting");
const currentIdx = Math.max(0, steps.indexOf(current));
const percent = Math.round(((currentIdx + 0.5) / steps.length) * 100);
return (
<div className="space-y-3 animate-in fade-in duration-200">
<div className="flex items-start gap-3 rounded-lg border border-sky-200 bg-sky-50/60 p-3">
<Loader2 className="mt-0.5 size-4 shrink-0 animate-spin text-sky-600" />
<div className="min-w-0 flex-1 text-[13px] leading-relaxed text-sky-900">
<div className="flex items-center justify-between gap-3">
<span className="font-semibold">
{restarting ? "Restarting services" : `Updating: ${STEP_LABELS[current] ?? current}`}
</span>
<span className="text-xs tabular-nums text-sky-700">{percent}%</span>
</div>
<div className="mt-1.5 h-1.5 overflow-hidden rounded-full bg-sky-100">
<div
className="h-full rounded-full bg-sky-500 transition-[width] duration-500 ease-out"
style={{ width: `${percent}%` }}
/>
</div>
<div className="mt-1.5 text-xs text-sky-800/80">
{restarting
? "The backend is coming back up. This panel reconnects on its own; keep the tab open or come back later, the result is kept."
: "You can close this dialog; the pill in the top bar keeps following the job."}
</div>
</div>
</div>
<ol className="flex flex-wrap gap-1.5">
{steps.map((s, i) => {
const done = currentIdx > i || (restarting && s !== "wait");
const active = s === current;
return (
<li
key={s}
className={cn(
"inline-flex items-center gap-1 rounded-full border px-2 py-0.5 text-[11px]",
done && "border-emerald-200 bg-emerald-50 text-emerald-700",
active && "border-sky-300 bg-sky-50 text-sky-700",
!done && !active && "border-border text-muted-foreground",
)}
>
{done ? (
<Check className="size-3" />
) : active ? (
<Loader2 className="size-3 animate-spin" />
) : null}
{STEP_LABELS[s] ?? s}
</li>
);
})}
</ol>
{job?.log && job.log.length > 0 && <LogPanel lines={job.log} />}
</div>
);
}
function LogPanel({ lines }: { lines: string[] }) {
const ref = useRef<HTMLPreElement>(null);
useEffect(() => {
const el = ref.current;
if (el) el.scrollTop = el.scrollHeight;
}, [lines.length]);
return (
<pre
ref={ref}
className="max-h-64 overflow-auto rounded-md border border-border bg-zinc-950 p-3 text-[11px] leading-relaxed text-zinc-200"
>
{lines.join("\n")}
</pre>
);
}
function Notice({ tone, children }: { tone: "info" | "warning" | "error"; children: React.ReactNode }) {
const styles = {
info: "border-sky-200 bg-sky-50/60 text-sky-900",
warning: "border-amber-200 bg-amber-50/60 text-amber-900",
error: "border-red-200 bg-red-50/60 text-red-900",
}[tone];
const Icon = tone === "error" ? XCircle : AlertTriangle;
return (
<div className={cn("flex items-start gap-3 rounded-lg border p-3 text-[13px] leading-relaxed", styles)}>
<Icon className="mt-0.5 size-4 shrink-0" />
<div className="min-w-0 flex-1">{children}</div>
</div>
);
}
function Cmd({ children }: { children: string }) {
return (
<code className="mt-1.5 mb-1.5 block rounded bg-white/70 px-2 py-1 font-mono text-[12px] text-foreground">
{children}
</code>
);
}
@@ -0,0 +1,98 @@
// The version pill in the top bar. Quiet when the instance is current, amber
// when a newer version exists, a spinner while an update runs or the backend
// restarts. It also closes the loop after a restart: once the backend answers
// with a new build it says "Updated to vX" and refreshes every query.
import { useEffect, useState } from "react";
import { useQueryClient } from "@tanstack/react-query";
import { toast } from "sonner";
import { ArrowUpCircle, Loader2 } from "lucide-react";
import { cn } from "@/lib/utils";
import { useAdminPerm } from "@/hooks/useAdminPerm";
import { AdminPerm } from "@/lib/auth/permissions";
import { buildLabel, isUpdating, useUpdateState } from "@/hooks/useUpdateState";
import { clearUpdateStarted, readUpdateStarted } from "@/lib/updateSession";
import { UpdateDialog } from "./UpdateDialog";
export function UpdatePill() {
const canRead = useAdminPerm(AdminPerm.ViewAnalytics);
const stateQ = useUpdateState({ enabled: canRead });
const qc = useQueryClient();
const [open, setOpen] = useState(false);
const state = stateQ.data;
const updating = isUpdating(state);
const started = readUpdateStarted();
// The poll fails while the backend restarts; an update this browser
// started is the only reason to read that as "restarting" rather than down.
const restarting = !!started && stateQ.isError;
// The backend is back after an update this browser started: report the
// outcome once, then refresh everything so no page shows stale data.
useEffect(() => {
if (!started || !state || updating) return;
const last = state.updater.last_job;
const moved =
(state.running.commit && state.running.commit !== started.fromCommit) ||
(state.running.version && state.running.version !== started.fromVersion);
if (last?.status === "failed") {
clearUpdateStarted();
toast.error(`The update failed: ${last.error ?? "see the update dialog"}`);
return;
}
if (moved || last?.status === "succeeded") {
clearUpdateStarted();
toast.success(`Updated to ${buildLabel(state)}`);
void qc.invalidateQueries();
}
}, [started, state, updating, qc]);
if (!canRead || (!state && !restarting)) return null;
let tone = "border-border bg-white text-muted-foreground hover:text-foreground";
let label = buildLabel(state);
let icon: React.ReactNode = null;
let title = "Up to date";
if (restarting) {
tone = "border-sky-200 bg-sky-50 text-sky-700";
label = "Restarting";
icon = <Loader2 className="size-3 animate-spin" />;
title = "The backend is restarting after an update";
} else if (updating) {
tone = "border-sky-200 bg-sky-50 text-sky-700";
label = "Updating";
icon = <Loader2 className="size-3 animate-spin" />;
title = `Update in progress: ${state?.updater.job?.step ?? ""}`;
} else if (state?.update_available) {
tone = "border-amber-300 bg-amber-50 text-amber-800 hover:bg-amber-100";
label = state.latest?.tag && state.reason === "release" ? `Update to ${state.latest.tag}` : "Update available";
icon = (
<span className="relative flex size-3.5 items-center justify-center">
<span className="absolute inline-flex size-full rounded-full bg-amber-400 opacity-60 animate-ping" />
<ArrowUpCircle className="relative size-3.5" />
</span>
);
title = `A newer version is available; running ${buildLabel(state)}`;
} else if (state?.updater.status === "unreachable") {
tone = "border-amber-200 bg-white text-amber-700";
title = "The updater is not answering";
}
return (
<>
<button
type="button"
onClick={() => setOpen(true)}
title={title}
className={cn(
"inline-flex items-center gap-1.5 rounded-full border px-2 py-0.5 text-[11px] font-medium transition-colors",
tone,
)}
>
{icon}
{label}
</button>
<UpdateDialog open={open} onOpenChange={setOpen} />
</>
);
}
+45
View File
@@ -0,0 +1,45 @@
import { useQuery } from "@tanstack/react-query";
import { getUpdateState, type UpdateState } from "@/lib/api/client/admin/updates";
export const UPDATE_STATE_KEY = ["admin", "instance", "update"] as const;
export const UPDATE_JOB_KEY = ["admin", "instance", "update", "job"] as const;
// One cache entry feeds the top-bar pill, the Setup and health card and the
// dialog. It polls every minute, and every few seconds while a job runs, so
// the pill follows an update started from another tab or before a reload.
export function useUpdateState(options?: { enabled?: boolean }) {
return useQuery({
queryKey: UPDATE_STATE_KEY,
queryFn: () => getUpdateState(false),
refetchInterval: (query) => (isUpdating(query.state.data) ? 3_000 : 60_000),
// While the backend restarts every request fails; keep the last state
// on screen instead of flashing an error.
retry: false,
enabled: options?.enabled ?? true,
});
}
// The dialog's view of a running job, with the log. Polled only while open.
export function useUpdateJob(enabled: boolean) {
return useQuery({
queryKey: UPDATE_JOB_KEY,
queryFn: () => getUpdateState(true),
refetchInterval: (query) => (isUpdating(query.state.data) ? 2_000 : 15_000),
retry: false,
enabled,
});
}
export function isUpdating(state: UpdateState | undefined): boolean {
return state?.updater.job?.status === "running";
}
// A short, stable label for the running build: the tag when there is one,
// otherwise the commit.
export function buildLabel(state: UpdateState | undefined): string {
if (!state) return "";
const v = state.running.version;
if (v && v !== "dev") return v;
const c = state.running.commit ?? state.updater.checkout?.commit ?? "";
return c ? `dev ${c.slice(0, 7)}` : "dev";
}
@@ -111,6 +111,16 @@ export interface InstanceSettings {
daily_messages_per_mailbox: number;
daily_messages_per_org: number;
};
// How long event-level history is kept. Every window bounds personal data:
// opens and clicks carry a client, a device and a location, funnel events
// carry a visitor's path, and the audit trail carries IP addresses, user
// agents and change payloads. None of them affect a count or a routing
// decision; campaign progress keeps its own summary.
retention: {
engagement_event_days: number;
form_event_days: number;
audit_log_days: number;
};
// The sending-domain authentication gate. A mailbox whose domain has been
// failing SPF or DMARC for longer than the grace window stops sending cold
// mail and warmup mail until the records are fixed.
@@ -118,6 +128,71 @@ export interface InstanceSettings {
enforce_domain_auth: boolean;
auth_grace_hours: number;
};
// Operator notification channels. Targets and secrets are redacted on
// read: a chat webhook URL is a bearer credential, so the server returns a
// recognisable preview and treats the preview (or an empty string) as
// "keep what is stored" on write.
// Optional so a client that does not manage channels (the instance
// settings page) can PUT without them; an absent section keeps the
// stored channels untouched.
notifications?: {
channels: NotifyChannel[];
};
}
export type NotifyChannelType = "discord" | "slack" | "webhook" | "email";
export interface NotifyChannel {
id: string;
name: string;
type: NotifyChannelType;
/** Webhook URL, or the address for an email channel. Redacted on read. */
target: string;
/** HMAC secret for the generic webhook transport. Redacted on read. */
secret?: string;
/** Subscribed event keys. Empty means every event. */
events: string[];
enabled: boolean;
}
export type NotifySeverity = "info" | "warning" | "urgent";
export interface NotifyEventDef {
key: string;
label: string;
description: string;
group: string;
severity: NotifySeverity;
self_host_relevant: boolean;
}
export interface NotifyEventsResult {
events: NotifyEventDef[];
self_hosted: boolean;
}
export function getNotificationEvents(): Promise<NotifyEventsResult> {
return Request({
method: "GET",
url: "/admin/instance/notifications/events",
authorization: true,
});
}
/** Send a test alert. Pass `id` for a saved channel, or the unsaved fields. */
export function testNotificationChannel(body: {
id?: string;
type?: NotifyChannelType;
name?: string;
target?: string;
secret?: string;
}): Promise<{ delivered: boolean }> {
return Request({
method: "POST",
url: "/admin/instance/notifications/test",
data: body,
authorization: true,
});
}
export function getInstanceSettings(): Promise<InstanceSettings> {
+111
View File
@@ -0,0 +1,111 @@
// /admin/instance/update : what is running, what is newest, and the button
// that applies it through the host-side updater.
import { Request } from "@/lib/api/client";
export interface RunningBuild {
version: string;
commit?: string;
built_at?: string;
}
export interface LatestRelease {
tag: string;
name?: string;
html_url?: string;
published_at?: string;
channel: string;
}
export interface UpdaterCheckout {
branch: string;
detached: boolean;
commit: string;
describe: string;
remote_commit: string;
behind: number;
dirty: boolean;
fetched_at: string;
fetch_error?: string;
}
// An image install has no checkout: what it runs is the tag pinned in its
// .env, which the updater reports instead.
export interface UpdaterRelease {
tag: string;
prefix: string;
// False for a moving channel tag (prod, dev), where re-pulling the same
// name is itself an update.
pinned: boolean;
}
export type UpdateJobStatus = "running" | "succeeded" | "failed";
export interface UpdateJob {
id: string;
status: UpdateJobStatus;
target: string;
step: string;
started_at: string;
finished_at?: string;
error?: string;
from_commit: string;
to_commit?: string;
// Absent on the top-bar poll; present when fetched with log=1.
log?: string[] | null;
}
export type UpdaterStatus = "off" | "ok" | "unreachable";
export interface UpdaterView {
configured: boolean;
status: UpdaterStatus;
error?: string;
mode?: "compose" | "image" | "command";
repo_dir?: string;
// Exactly one of these: a checkout in compose and command mode, a release
// in image mode.
checkout?: UpdaterCheckout;
release?: UpdaterRelease;
job?: UpdateJob;
last_job?: UpdateJob;
}
export interface UpdateState {
running: RunningBuild;
latest?: LatestRelease;
update_available: boolean;
reason?: "release" | "commits";
checked_at?: string;
check_error?: string;
enabled: boolean;
interval: string;
channel: string;
repo: string;
updater: UpdaterView;
}
export function getUpdateState(withLog = false): Promise<UpdateState> {
return Request({
method: "GET",
url: withLog ? "/admin/instance/update?log=1" : "/admin/instance/update",
authorization: true,
});
}
export function checkForUpdates(): Promise<UpdateState> {
return Request({
method: "POST",
url: "/admin/instance/update/check",
authorization: true,
});
}
export function applyUpdate(target = "latest"): Promise<UpdateJob> {
return Request({
method: "POST",
url: "/admin/instance/update/apply",
data: { target },
authorization: true,
});
}
@@ -151,12 +151,16 @@ export function isJobActive(job: WarmupGenerationJob): boolean {
return !TERMINAL_JOB_STATUS.has(job.status);
}
/** Only batch jobs that are still in flight can be cancelled. */
/**
* Only batch jobs that are still in flight can be cancelled. A batch already
* cancelling stays active until the poller ingests what it finished, but a
* second cancel has nothing left to ask for.
*/
export function isJobCancellable(job: WarmupGenerationJob): boolean {
if (job.mode !== "batch") return false;
const bs = job.batch_status ?? "";
if (!bs) return !TERMINAL_JOB_STATUS.has(job.status);
return !TERMINAL_BATCH_STATUS.has(bs);
return bs !== "cancelling" && !TERMINAL_BATCH_STATUS.has(bs);
}
export interface WarmupAbRow {
+44
View File
@@ -0,0 +1,44 @@
// Remembers that this browser started an update, across the backend restart
// and a page reload, so the top bar can say "Updated to vX" (or that it
// failed) once the backend answers again instead of silently going quiet.
const KEY = "warmbly.admin.update.started";
export interface StartedUpdate {
fromVersion: string;
fromCommit: string;
startedAt: number;
}
export function markUpdateStarted(fromVersion: string, fromCommit: string) {
try {
const v: StartedUpdate = { fromVersion, fromCommit, startedAt: Date.now() };
sessionStorage.setItem(KEY, JSON.stringify(v));
} catch {
/* storage unavailable: the dialog still tracks the job while open */
}
}
export function readUpdateStarted(): StartedUpdate | null {
try {
const raw = sessionStorage.getItem(KEY);
if (!raw) return null;
const v = JSON.parse(raw) as StartedUpdate;
// An entry older than an hour is a job nobody is waiting on any more.
if (Date.now() - v.startedAt > 60 * 60_000) {
sessionStorage.removeItem(KEY);
return null;
}
return v;
} catch {
return null;
}
}
export function clearUpdateStarted() {
try {
sessionStorage.removeItem(KEY);
} catch {
/* ignore */
}
}
+12
View File
@@ -45,6 +45,7 @@ import SystemStatusPage from "@/app/dashboard/SystemStatusPage";
import HealthPage from "@/app/dashboard/HealthPage";
import ConfigurationPage from "@/app/dashboard/ConfigurationPage";
import InstanceSettingsPage from "@/app/dashboard/InstanceSettingsPage";
import NotificationsPage from "@/app/dashboard/NotificationsPage";
import LimitsPage from "@/app/dashboard/LimitsPage";
import RealtimeManager from "@/lib/realtime/RealtimeManager";
import { RequirePermission } from "@/components/layout/RequirePermission";
@@ -151,6 +152,17 @@ const router = createBrowserRouter([
</RequirePermission>
),
},
{
path: "configuration/notifications",
element: (
<RequirePermission
perm={AdminPerm.ManageSettings}
permissionLabel="Manage settings"
>
<NotificationsPage />
</RequirePermission>
),
},
{
path: "limits",
element: (
+131 -15
View File
@@ -2,6 +2,7 @@ package main
import (
"context"
"encoding/json"
"errors"
"fmt"
"log"
@@ -25,6 +26,7 @@ import (
"github.com/warmbly/warmbly/internal/app/admin"
"github.com/warmbly/warmbly/internal/app/adminoutreach"
"github.com/warmbly/warmbly/internal/app/advanced"
"github.com/warmbly/warmbly/internal/app/unsublink"
"github.com/jackc/pgx/v5/pgxpool"
"github.com/warmbly/warmbly/internal/app/advisor"
@@ -38,6 +40,7 @@ import (
"github.com/warmbly/warmbly/internal/app/bootstrap"
"github.com/warmbly/warmbly/internal/app/campaign"
"github.com/warmbly/warmbly/internal/app/cipher"
"github.com/warmbly/warmbly/internal/app/cliauth"
"github.com/warmbly/warmbly/internal/app/cloudlink"
"github.com/warmbly/warmbly/internal/app/compose"
"github.com/warmbly/warmbly/internal/app/contact"
@@ -67,6 +70,7 @@ import (
"github.com/warmbly/warmbly/internal/app/notification"
"github.com/warmbly/warmbly/internal/app/oauth"
"github.com/warmbly/warmbly/internal/app/oidcauth"
"github.com/warmbly/warmbly/internal/app/opsnotify"
"github.com/warmbly/warmbly/internal/app/organization"
orgrisk "github.com/warmbly/warmbly/internal/app/orgrisk"
"github.com/warmbly/warmbly/internal/app/orgtransfer"
@@ -95,6 +99,7 @@ import (
"github.com/warmbly/warmbly/internal/app/twofa"
"github.com/warmbly/warmbly/internal/app/tz"
"github.com/warmbly/warmbly/internal/app/unibox"
"github.com/warmbly/warmbly/internal/app/updates"
"github.com/warmbly/warmbly/internal/app/user"
warmupapp "github.com/warmbly/warmbly/internal/app/warmup"
"github.com/warmbly/warmbly/internal/app/warmupcontent"
@@ -159,6 +164,7 @@ func main() {
var emailService email.EmailService
var poolLinkService poollink.Service
var cloudLinkService cloudlink.Service
var cliAuthService cliauth.Service
var campaignService campaign.CampaignService
var analyticsService analytics.AnalyticsService
var rateLimitService ratelimit.RateLimitService
@@ -179,6 +185,7 @@ func main() {
var provisioningPolicyRepo repository.ProvisioningPolicyRepository
var tasksService tasks.TasksService
var advancedService advanced.Service
var unsubSigner *unsublink.Signer
var warmupContentRepo repository.WarmupContentRepository
var warmupContentService warmupcontent.Service
var creditRepository repository.CreditRepository
@@ -232,6 +239,7 @@ func main() {
var workerRepoForHandler repository.WorkerRepository
var credentialsRepository repository.CredentialsRepository
var releasesService *releases.Service
var updatesService *updates.Service
// Notifications
var emailNotificationService notify.EmailNotificationService
@@ -280,6 +288,10 @@ func main() {
// instanceSettings and the health registry are built after the handler
// dependencies, so the pool is hoisted out of the connection block.
var instanceSettings instancesettings.Service
// opsNotifier fans instance-wide operator alerts out to the Discord/Slack/
// webhook/email channels an admin configured. Nil-safe everywhere: a
// deployment with no channels simply never delivers anything.
var opsNotifier opsnotify.Notifier
var instanceChecksDB *pgxpool.Pool
var userRepoForHandler repository.UserRepository
var organizationRepoForHandler repository.OrganizationRepository
@@ -585,6 +597,7 @@ func main() {
trackedLinkRepository = repository.NewTrackedLinkRepository(primaryDB.Pool)
instanceChecksDB = primaryDB.Pool
instanceSettings = instancesettings.NewService(instancesettings.NewStore(primaryDB.Pool))
bootstrapInstanceSettings(ctx, instanceSettings)
if err != nil {
sentry.CaptureException(err)
log.Fatal(err)
@@ -711,6 +724,13 @@ func main() {
// trial start (planRepo + creditService, both already constructed above).
trialService = trial.NewService(subscriptionRepository, userRepostory, planRepository, creditService)
featureGateService = feature.NewService(subscriptionRepository, planRepository)
// An approved daily-send increase must raise what is enforced, not
// only what the dashboard shows.
if g, ok := featureGateService.(interface {
WireLimitOverrides(feature.LimitOverrideReader)
}); ok {
g.WireLimitOverrides(organizationRepository)
}
workerAssignmentService = worker.NewAssignmentService(workerRepository, subscriptionRepository, planRepository)
subscriptionService = subscription.NewService(subscriptionRepository, planRepository)
// dailyThrottleService needs the cache that's constructed
@@ -814,6 +834,29 @@ func main() {
if organizationService != nil {
organizationService.WireInstanceSettings(instanceSettings)
}
// Operator alerts. The channel list lives in the same settings
// document, so this needs nothing else configured to work.
opsNotifier = opsnotify.NewService(instanceSettings, emailNotificationService, config.AppBaseURL())
authService.WireOperatorNotifier(opsNotifier)
if organizationService != nil {
organizationService.WireOperatorNotifier(opsNotifier)
}
if aware, ok := warmupService.(interface {
WireOperatorNotifier(warmupapp.OperatorNotifier)
}); ok && warmupService != nil {
aware.WireOperatorNotifier(opsNotifier)
}
if aware, ok := orgRiskService.(interface {
WireOperatorNotifier(orgrisk.OperatorNotifier)
}); ok && orgRiskService != nil {
aware.WireOperatorNotifier(opsNotifier)
}
if aware, ok := stripeService.(interface {
WireOperatorNotifier(stripe.OperatorNotifier)
}); ok && stripeService != nil {
aware.WireOperatorNotifier(opsNotifier)
}
}
log.Printf("Auth policy: login_code=%s registration=%s (DISABLE_REGISTRATION) email_verification=%t sso_auto_provision=%t",
authPolicy.LoginCode, authPolicy.Registration, authPolicy.RequireEmailVerification, authPolicy.SSOAutoProvision)
@@ -1122,12 +1165,31 @@ func main() {
)
releasesService.RunBootCheck(ctx)
// Update indicator and one-click update. The release check is on by
// default (one GitHub API read per interval); applying an update needs
// the host-side updater (UPDATER_URL), which the compose stack ships as
// the "updater" profile.
updateInterval, _ := time.ParseDuration(getenvDefault("UPDATE_CHECK_INTERVAL", "30m"))
updatesService = updates.New(updates.Config{
Enabled: getenvDefault("UPDATE_CHECK_ENABLED", "true") != "false",
Interval: updateInterval,
Channel: getenvDefault("UPDATE_CHANNEL", "stable"),
GithubRepo: getenvDefault("RELEASES_GITHUB_REPO", "warmbly/warmbly"),
GithubToken: os.Getenv("RELEASES_GITHUB_TOKEN"),
UpdaterURL: os.Getenv("UPDATER_URL"),
UpdaterToken: getenvDefault("UPDATER_TOKEN", os.Getenv("INTERNAL_API_TOKEN")),
})
updatesService.Start(ctx)
eventsPublisher := events.NewPublisher(bus, s3, codecImpl, cipherService)
// apiCfg.Hostname is the bind address, not a reachable base. Building
// the mailbox-connect redirect_uri from it sends the provider
// "0.0.0.0:8080/addresses/google/callback".
oauth2Cfg := config.LoadOauth2(oauthPublicBaseURL(apiCfg.Hostname))
// Recipient unsubscribe links live on the API origin, signed under the
// auth secret; the same base the OAuth callbacks are built on.
unsubSigner = unsublink.New(authCfg.AuthSecret, oauthPublicBaseURL(apiCfg.Hostname))
emailService = email.NewServiceWithWorker(
emailRepostory,
cipherService,
@@ -1141,10 +1203,9 @@ func main() {
)
// Fan out email-account lifecycle events to customer webhooks.
emailService.WireWebhooks(webhookService)
// Same wire-after-construct pattern for the daily throttle —
// only the prod backend has a real cache; jobs / tests build
// emailService without one.
emailService.WireThrottle(dailyThrottleService)
// Every connect path checks the workspace's mailbox allowance
// (fair use for paid plans, the free cap otherwise).
emailService.WireMailboxAllowance(organizationService)
// Seed Graph delta cursors when the reconciler reloads mailboxes.
emailService.WireGraphDelta(repository.NewEmailGraphDeltaRepository(primaryDB))
// The Gmail equivalent: without it a reloaded mailbox re-bootstraps its
@@ -1204,6 +1265,11 @@ func main() {
if aware, ok := contactService.(contact.SegmentAware); ok {
aware.WireSegments(segmentRepository, segmentService)
}
// A new contact is an event: customer webhooks and "contact created"
// automations hear about it from the one write path every creator uses.
if aware, ok := contactService.(contact.WebhookAware); ok {
aware.WireWebhooks(webhookServiceForHandler)
}
formRepository := repository.NewFormRepository(primaryDB)
formEventRepository := repository.NewFormEventRepository(primaryDB)
formService = form.NewService(formRepository)
@@ -1216,7 +1282,7 @@ func main() {
formService.SetLinks(repository.NewFormLinkRepository(primaryDB))
formService.SetEvents(formEventRepository)
formService.SetDomains(organizationRepoForHandler)
go jobs.NewFormEventsRetentionJob(formEventRepository).Start(ctx, 12*time.Hour)
go jobs.NewFormEventsRetentionJob(formEventRepository).WireRetention(instanceSettings).Start(ctx, 12*time.Hour)
go jobs.NewFormsDomainSweep(organizationRepoForHandler).Start(ctx, time.Hour)
// A visibly bad import is filed on the workspace's posture. On its own
// it can only reach `watch`, which changes nothing.
@@ -1231,6 +1297,9 @@ func main() {
leadSyncServiceForHandler = leadsync.NewService(leadSyncRepository, integrationServiceForHandler, contactService)
apiKeyService = apikey.NewService(cache, apiKeyRepository)
// `warmbly auth login`: the browser approval mints an ordinary API key
// through the service above, so it has to be built after it.
cliAuthService = cliauth.NewService(repository.NewCLIAuthRepository(primaryDB.Pool), apiKeyService, organizationService, userService, organizationRepository)
crmService = crm.NewService(crmRepository)
teamRepository := repository.NewTeamRepository(primaryDB.Pool)
teamService = team.NewService(teamRepository)
@@ -1300,20 +1369,29 @@ func main() {
if aware, ok := campaignService.(campaign.ProgressAware); ok {
aware.WireProgress(campaignProgressRepository)
}
// The wizard's audience-versus-pool estimate counts segment members.
if aware, ok := campaignService.(campaign.SegmentAware); ok {
aware.WireSegments(segmentService)
}
// Delete drops attachment objects and duplicate copies them, so the
// campaign service needs the store the attachment handler writes to.
if aware, ok := campaignService.(campaign.AttachmentAware); ok {
aware.WireAttachments(attachmentRepoForHandler, s3ForHandler)
}
// Deleting a step cascades its attachment rows away, so the sequence
// service needs the same store to drop the objects behind them.
if aware, ok := sequenceService.(sequence.AttachmentAware); ok {
aware.WireAttachments(attachmentRepoForHandler, s3ForHandler)
}
// Attaching a lead to a running campaign has to wake that campaign's
// parked send chain, or the lead sits queued until the chain's next
// tick. Wired here because contactService is built before the scheduler
// and Cloud Tasks client exist.
if segmentService != nil {
segmentService.SetCampaignWaker(campaignService)
// A completed campaign whose linked segments grow is restarted
// through the full launch checks, never by a raw status flip.
segmentService.SetCampaignStarter(campaignService)
// Sweep enrolments are audited as campaign updates so teammates'
// Leads tabs refresh through the audit spine.
segmentService.SetEnrolmentAuditor(auditService)
}
if contactService != nil {
contactService.SetCampaignWaker(campaignService)
@@ -1430,9 +1508,10 @@ func main() {
// the advanced/contact/org services exist (the integration service was
// constructed earlier).
integrationServiceForHandler.SetNativeActions(nativeactions.Adapter{
Adv: advancedService,
Contacts: contactRepostory,
Orgs: organizationRepository,
Adv: advancedService,
Contacts: contactRepostory,
Orgs: organizationRepository,
ContactSvc: contactService,
})
integrationServiceForHandler.SetPublisher(streamingPublisher)
// AI automation nodes (ai_step / ai_switch) run over the same provider +
@@ -1517,6 +1596,7 @@ func main() {
trackedLinkRepository,
integrationServiceForHandler, // AutomationRunner for campaign run_automation steps
)
tasksService.SetUnsubscribeLinks(unsubSigner)
// Sequence action nodes that pin a contact into or out of a segment,
// both on the scheduled path (tasks) and the instant reply path (advanced).
if aware, ok := tasksService.(tasks.SegmentAware); ok {
@@ -1658,10 +1738,11 @@ func main() {
orgTransferScheduler := jobs.NewOrgTransferScheduler(orgTransferJob, 1*time.Hour)
go orgTransferScheduler.Start(ctx)
// Prune audit entries past the retention window (90 days). Bounding the
// trail's age also bounds how long PII is retained. auditRepository is
// Prune audit entries past the retention window. Bounding the trail's
// age also bounds how long PII is retained, so the window is an
// instance setting and is read on every pass. auditRepository is
// constructed earlier (before authService).
auditRetentionJob := jobs.NewAuditRetentionJob(auditRepository, 90*24*time.Hour)
auditRetentionJob := jobs.NewAuditRetentionJob(auditRepository).WireRetention(instanceSettings)
auditRetentionScheduler := jobs.NewAuditRetentionScheduler(auditRetentionJob, 6*time.Hour)
go auditRetentionScheduler.Start(ctx)
@@ -1838,6 +1919,7 @@ func main() {
Policy: authPolicy,
DB: instanceChecksDB,
Cache: authCache,
Updates: updatesService,
})
h := &handler.Handler{
@@ -1853,9 +1935,11 @@ func main() {
InstanceRuntime: instanceRuntime,
InstanceChecks: instanceChecks,
InstanceSettings: instanceSettings,
OpsNotifier: opsNotifier,
PoolLinkService: poolLinkService,
CloudLinkService: cloudLinkService,
CLIAuthService: cliAuthService,
TokenService: tokenService,
PasskeyService: passkeyService,
@@ -1918,12 +2002,14 @@ func main() {
WorkerRepo: workerRepoForHandler,
CredentialsRepo: credentialsRepository,
ReleasesService: releasesService,
UpdatesService: updatesService,
// Notifications
EmailNotificationService: emailNotificationService,
// Advanced outreach controls
AdvancedService: advancedService,
AdvancedService: advancedService,
UnsubscribeLinks: unsubSigner,
// Warmup health
WarmupService: warmupService,
@@ -2137,3 +2223,33 @@ func (m advisorMembers) MemberPermissions(ctx context.Context, orgID, userID uui
}
return member.Permissions, nil
}
// bootstrapInstanceSettings applies WARMBLY_SETTINGS_BOOTSTRAP once, on an
// instance whose settings document has never been written. It is how an
// unattended install ships its data-control answers (what is imported, what is
// kept and for how long) with the rest of the environment, so the wizard's
// choices are in place before the first mailbox is connected instead of being
// something the operator has to redo in the panel.
//
// The body is the same partial document PUT /admin/instance/settings takes.
// From the first write onwards the panel is authoritative and this is a no-op,
// so the variable can stay in .env without ever undoing a later edit.
func bootstrapInstanceSettings(ctx context.Context, svc instancesettings.Service) {
raw := strings.TrimSpace(os.Getenv("WARMBLY_SETTINGS_BOOTSTRAP"))
if raw == "" || svc == nil {
return
}
var patch instancesettings.Patch
if err := json.Unmarshal([]byte(raw), &patch); err != nil {
log.Printf("WARMBLY_SETTINGS_BOOTSTRAP is not valid JSON and was ignored: %v", err)
return
}
applied, err := svc.Bootstrap(ctx, patch)
if err != nil {
log.Printf("WARMBLY_SETTINGS_BOOTSTRAP could not be applied: %v", err)
return
}
if applied {
log.Printf("instance settings seeded from WARMBLY_SETTINGS_BOOTSTRAP")
}
}
+112
View File
@@ -0,0 +1,112 @@
package main
import (
"fmt"
"sort"
"strings"
"github.com/spf13/cobra"
)
// Aliases are plain command lines stored in config.yml. They are expanded
// before cobra sees the arguments, so an alias can carry flags and the user
// can still add more on the end.
func newAliasCmd(f *Factory) *cobra.Command {
cmd := &cobra.Command{
Use: "alias <command>",
Short: "Shortcuts for command lines you type often",
GroupID: groupSetup,
Long: `Save a command line under a shorter name.
Anything after the alias on the command line is appended, so an alias can be a
starting point rather than a fixed command.`,
Example: ` $ warmbly alias set hot "campaign list --status active"
$ warmbly hot --json`,
}
set := &cobra.Command{
Use: "set <name> <expansion>",
Short: "Create or replace an alias",
Args: cobra.ExactArgs(2),
RunE: func(c *cobra.Command, args []string) error {
name, expansion := args[0], args[1]
if strings.ContainsAny(name, " \t") {
return fmt.Errorf("an alias name cannot contain spaces")
}
// Shadowing a real command would make it unreachable, and the
// person who did it would have no way to tell why.
for _, existing := range c.Root().Commands() {
if existing.Name() == name {
return fmt.Errorf("%q is already a warmbly command, so an alias for it would hide it", name)
}
}
if _, err := splitArgs(expansion); err != nil {
return err
}
cfg, err := f.Config()
if err != nil {
return err
}
if cfg.Aliases == nil {
cfg.Aliases = map[string]string{}
}
cfg.Aliases[name] = expansion
if err := cfg.Save(); err != nil {
return err
}
f.IO.Errorf("%s %s = %s\n", f.IO.Tick(), name, expansion)
return nil
},
}
list := &cobra.Command{
Use: "list",
Aliases: []string{"ls"},
Short: "Show every alias",
Args: cobra.NoArgs,
RunE: func(*cobra.Command, []string) error {
cfg, err := f.Config()
if err != nil {
return err
}
if len(cfg.Aliases) == 0 {
f.IO.Println(f.IO.Gray("No aliases yet. Try: warmbly alias set hot \"campaign list --status active\""))
return nil
}
names := make([]string, 0, len(cfg.Aliases))
for name := range cfg.Aliases {
names = append(names, name)
}
sort.Strings(names)
for _, name := range names {
f.IO.Printf("%-12s %s\n", name, cfg.Aliases[name])
}
return nil
},
}
del := &cobra.Command{
Use: "delete <name>",
Aliases: []string{"rm"},
Short: "Remove an alias",
Args: cobra.ExactArgs(1),
RunE: func(c *cobra.Command, args []string) error {
cfg, err := f.Config()
if err != nil {
return err
}
if _, ok := cfg.Aliases[args[0]]; !ok {
return fmt.Errorf("no alias called %q", args[0])
}
delete(cfg.Aliases, args[0])
if err := cfg.Save(); err != nil {
return err
}
f.IO.Errorf("%s removed %s\n", f.IO.Tick(), args[0])
return nil
},
}
cmd.AddCommand(set, list, del)
return cmd
}
+251
View File
@@ -0,0 +1,251 @@
package main
import (
"encoding/json"
"fmt"
"io"
"net/http"
"net/url"
"strconv"
"strings"
"github.com/spf13/cobra"
"github.com/warmbly/warmbly/internal/cli/api"
"github.com/warmbly/warmbly/internal/cli/output"
)
// newAPICmd is the escape hatch that makes the typed commands optional: every
// endpoint is reachable on day one, whether or not a noun-verb command for it
// exists yet.
func newAPICmd(f *Factory) *cobra.Command {
var (
method string
rawField []string
field []string
headers []string
input string
paginate bool
maxPages int
include bool
silent bool
idemKey string
)
cmd := &cobra.Command{
Use: "api <endpoint>",
Short: "Call any Warmbly API endpoint",
GroupID: groupDevelop,
Long: `Make an authenticated request to the Warmbly REST API.
The endpoint is relative to /v1 unless it already names a version, so
"/campaigns" and "/v1/campaigns" are the same call. The method defaults to GET,
or POST when any field is supplied.
Fields build a JSON body. -f keeps the value a string; -F guesses the type, so
true, false, null and numbers arrive as themselves, and @file or @- reads a
value from a file or stdin. Nested keys use key[sub]=value and repeated key[]
builds an array.`,
Example: ` $ warmbly api /me
$ warmbly api "/campaigns?limit=10" --paginate
$ warmbly api /contacts -f email=jane@example.com -f first_name=Jane
$ warmbly api /campaigns/CAMPAIGN_ID -X PATCH -F daily_limit=40
$ warmbly api /contacts/search -X POST --input filter.json
$ warmbly api /webhooks/WEBHOOK_ID -X DELETE`,
Args: cobra.ExactArgs(1),
RunE: func(c *cobra.Command, args []string) error {
client, err := f.Client()
if err != nil {
return err
}
parsed, err := url.Parse(args[0])
if err != nil {
return fmt.Errorf("%q is not a usable endpoint: %w", args[0], err)
}
body, err := bodyFromArg(f.IO.In, input)
if err != nil {
return err
}
fields, err := buildFields(rawField, field, f.IO.In)
if err != nil {
return err
}
if len(fields) > 0 {
if body != nil {
return fmt.Errorf("pass fields or --input, not both")
}
body, err = json.Marshal(fields)
if err != nil {
return err
}
}
if body != nil && !json.Valid(body) {
return fmt.Errorf("the request body is not valid JSON")
}
if method == "" {
method = http.MethodGet
if body != nil {
method = http.MethodPost
}
}
method = strings.ToUpper(method)
if method == http.MethodGet && body != nil {
return fmt.Errorf("a GET request carries no body. Put parameters in the query string, or pass -X POST.")
}
req := api.Request{
Method: method,
Path: parsed.Path,
Query: parsed.Query(),
Body: body,
IdempotencyKey: idemKey,
Headers: map[string]string{},
}
for _, h := range headers {
name, value, ok := strings.Cut(h, ":")
if !ok {
return fmt.Errorf("headers are name:value, not %q", h)
}
req.Headers[strings.TrimSpace(name)] = strings.TrimSpace(value)
}
if paginate {
if method != http.MethodGet {
return fmt.Errorf("--paginate only makes sense on a GET")
}
merged, perr := client.Paginate(c.Context(), req, maxPages)
if perr != nil {
return perr
}
if silent {
return nil
}
return (&output.Printer{IO: f.IO, JSON: true}).Print(merged, output.Table{})
}
resp, err := client.Do(c.Context(), req)
if resp != nil && include {
f.IO.Printf("HTTP %d\n", resp.Status)
for name, values := range resp.Header {
for _, v := range values {
f.IO.Printf("%s: %s\n", name, v)
}
}
f.IO.Println()
}
if err != nil {
return err
}
if silent {
return nil
}
return (&output.Printer{IO: f.IO, JSON: true, Template: f.Template}).Print(resp.Body, output.Table{})
},
}
cmd.Flags().StringVarP(&method, "method", "X", "", "HTTP method (default GET, or POST when fields are given)")
cmd.Flags().StringArrayVarP(&rawField, "raw-field", "f", nil, "Body field as a string: key=value")
cmd.Flags().StringArrayVarP(&field, "field", "F", nil, "Body field with a guessed type: key=value, key=@file")
cmd.Flags().StringArrayVarP(&headers, "header", "H", nil, "Extra request header: name:value")
cmd.Flags().StringVar(&input, "input", "", "Request body: JSON, @file, or - for stdin")
cmd.Flags().BoolVar(&paginate, "paginate", false, "Follow the cursor and merge every page")
cmd.Flags().IntVar(&maxPages, "max-pages", 100, "Stop after this many pages")
cmd.Flags().BoolVarP(&include, "include", "i", false, "Print the status and response headers too")
cmd.Flags().BoolVar(&silent, "silent", false, "Do not print the response body")
cmd.Flags().StringVar(&idemKey, "idempotency-key", "", "Idempotency-Key header for a safely retryable write")
return cmd
}
// buildFields turns -f and -F into one JSON object. Order matters only for
// duplicate keys, where the last one wins, as in curl.
func buildFields(raw, typed []string, stdin io.Reader) (map[string]any, error) {
out := map[string]any{}
for _, kv := range raw {
key, value, ok := strings.Cut(kv, "=")
if !ok {
return nil, fmt.Errorf("fields are key=value, not %q", kv)
}
if err := assign(out, key, value); err != nil {
return nil, err
}
}
for _, kv := range typed {
key, value, ok := strings.Cut(kv, "=")
if !ok {
return nil, fmt.Errorf("fields are key=value, not %q", kv)
}
if strings.HasPrefix(value, "@") {
source := value
if value == "@-" {
source = "-"
}
data, err := bodyFromArg(stdin, source)
if err != nil {
return nil, err
}
if err := assign(out, key, strings.TrimRight(string(data), "\n")); err != nil {
return nil, err
}
continue
}
if err := assign(out, key, guessType(value)); err != nil {
return nil, err
}
}
if len(out) == 0 {
return nil, nil
}
return out, nil
}
// guessType is the -F conversion: JSON literals become themselves, everything
// else stays a string.
func guessType(v string) any {
switch v {
case "true":
return true
case "false":
return false
case "null":
return nil
}
if n, err := strconv.ParseInt(v, 10, 64); err == nil {
return n
}
if fl, err := strconv.ParseFloat(v, 64); err == nil {
return fl
}
return v
}
// assign writes one field, understanding key[sub] for nesting and key[] for
// appending to an array.
func assign(obj map[string]any, key string, value any) error {
open := strings.Index(key, "[")
if open < 0 {
obj[key] = value
return nil
}
if !strings.HasSuffix(key, "]") {
return fmt.Errorf("unbalanced brackets in field %q", key)
}
head := key[:open]
inner := key[open+1 : len(key)-1]
if head == "" {
return fmt.Errorf("field %q has no name", key)
}
if inner == "" {
existing, _ := obj[head].([]any)
obj[head] = append(existing, value)
return nil
}
nested, ok := obj[head].(map[string]any)
if !ok {
nested = map[string]any{}
obj[head] = nested
}
return assign(nested, inner, value)
}
+747
View File
@@ -0,0 +1,747 @@
package main
import (
"context"
"encoding/json"
"fmt"
"net/http"
"os"
"sort"
"strings"
"time"
"github.com/spf13/cobra"
"github.com/warmbly/warmbly/internal/app/apikey"
"github.com/warmbly/warmbly/internal/app/oauth"
"github.com/warmbly/warmbly/internal/cli/api"
"github.com/warmbly/warmbly/internal/cli/config"
"github.com/warmbly/warmbly/internal/models"
)
func newAuthCmd(f *Factory) *cobra.Command {
cmd := &cobra.Command{
Use: "auth <command>",
Short: "Sign in, sign out, and see who you are",
GroupID: groupCore,
Long: `Sign the CLI in to a Warmbly instance.
Signing in through the browser creates one API key named for this machine. It
appears under Settings > API keys and can be revoked there or with
` + "`warmbly auth logout`" + `. Credentials are written to ` + config.HostsPath() + `
at 0600, and WARMBLY_TOKEN overrides the file without ever being written to it,
which is how CI authenticates with no login step.`,
}
cmd.AddCommand(
newAuthLoginCmd(f),
newAuthStatusCmd(f),
newAuthTokenCmd(f),
newAuthSwitchCmd(f),
newAuthRefreshCmd(f),
newAuthLogoutCmd(f),
)
return cmd
}
// parseScopes accepts the two presets plus any comma or space separated list
// of scope names, in either case, so `--scopes read_campaigns,READ_CONTACTS`
// works and a typo is named rather than silently dropped.
func parseScopes(raw string) (uint64, error) {
raw = strings.TrimSpace(raw)
switch strings.ToLower(raw) {
case "":
return models.APIPermFullAccess, nil
case "full", "full-access", "all":
return models.APIPermFullAccess, nil
case "read", "read-only", "readonly":
return models.APIPermReadOnly, nil
}
mask, unknown := oauth.ParseScopes(strings.NewReplacer(",", " ", "+", " ").Replace(raw))
if len(unknown) > 0 {
return 0, fmt.Errorf("unknown scope %s.\nRun `warmbly key permissions` for the full list, or use the presets: full, read-only.", strings.Join(unknown, ", "))
}
if mask == 0 {
return 0, fmt.Errorf("--scopes granted nothing. Name at least one scope, or use full or read-only.")
}
return mask, nil
}
func newAuthLoginCmd(f *Factory) *cobra.Command {
var (
hostname string
apiURL string
withToken bool
web bool
scopeStr string
force bool
)
cmd := &cobra.Command{
Use: "login",
Short: "Sign in to a Warmbly instance",
Long: `Sign in to a Warmbly instance.
With no flags this asks which instance, then how: a browser approval that
creates a key for this machine, or pasting a key you already have.`,
Example: ` $ warmbly auth login
$ warmbly auth login --hostname warmbly.acme.com
$ warmbly auth login --scopes read-only
$ echo $KEY | warmbly auth login --with-token`,
Args: cobra.NoArgs,
RunE: func(c *cobra.Command, _ []string) error {
return runAuthLogin(c.Context(), f, hostname, apiURL, withToken, web, scopeStr, force)
},
}
cmd.Flags().StringVar(&hostname, "hostname", "", "Instance to sign in to (default: warmbly.com)")
cmd.Flags().StringVar(&apiURL, "api-url", "", "API base URL, when it is not derivable from the hostname")
cmd.Flags().BoolVar(&withToken, "with-token", false, "Read an API key from stdin instead of using the browser")
cmd.Flags().BoolVarP(&web, "web", "w", false, "Go straight to the browser approval")
cmd.Flags().StringVarP(&scopeStr, "scopes", "s", "", "Scopes to request: full, read-only, or a list of names")
cmd.Flags().BoolVar(&force, "force", false, "Replace an existing sign-in for this host without asking")
return cmd
}
func runAuthLogin(ctx context.Context, f *Factory, hostname, apiURL string, withToken, web bool, scopeStr string, force bool) error {
io := f.IO
cfg, err := f.Config()
if err != nil {
return err
}
hosts, err := f.Hosts()
if err != nil {
return err
}
// 1. Which instance.
if hostname == "" && apiURL == "" {
if !io.IsStdinTTY() {
hostname = config.DefaultHost
} else {
idx, serr := io.Select("Where do you want to sign in?", []string{
"warmbly.com (the hosted service)",
"A self-hosted instance",
})
if serr != nil {
return serr
}
if idx == 0 {
hostname = config.DefaultHost
} else {
answer, ierr := io.Input("Instance hostname (for example warmbly.acme.com)", "")
if ierr != nil {
return ierr
}
if strings.TrimSpace(answer) == "" {
return fmt.Errorf("a hostname is required to sign in to a self-hosted instance")
}
hostname = answer
}
}
}
if hostname == "" {
hostname = apiURL
}
host := config.NormalizeHost(hostname)
if existing := hosts[host]; existing != nil && !force {
if !io.IsStdinTTY() {
return fmt.Errorf("already signed in to %s as %s. Pass --force to replace that sign-in.", host, existing.User)
}
ok, cerr := io.Confirm(fmt.Sprintf("Already signed in to %s as %s. Sign in again?", host, existing.User), false)
if cerr != nil {
return cerr
}
if !ok {
return errCancelled
}
}
// 2. Where its API is.
base, err := resolveAPIBase(ctx, f, host, apiURL)
if err != nil {
return err
}
// 3. How to authenticate.
useToken := withToken
if !withToken && !web && io.IsStdinTTY() {
idx, serr := io.Select("How do you want to sign in?", []string{
"Approve in a browser (creates a key for this machine)",
"Paste an API key you already have",
})
if serr != nil {
return serr
}
useToken = idx == 1
}
scopes, err := parseScopes(scopeStr)
if err != nil {
return err
}
var entry *config.Host
if useToken {
entry, err = loginWithToken(ctx, f, base)
} else {
entry, err = loginWithBrowser(ctx, f, cfg, base, scopes)
}
if err != nil {
return err
}
hosts[host] = entry
if err := hosts.Save(); err != nil {
return err
}
// Signing in makes that host the active one. Nobody expects to sign in and
// still have the next command talk to somewhere else.
if cfg.ActiveHost != host {
cfg.ActiveHost = host
if err := cfg.Save(); err != nil {
return err
}
}
io.Errorf("%s Signed in to %s as %s\n", io.Tick(), io.Bold(host), io.Bold(entry.User))
if entry.Organization != "" {
io.Errorf(" Workspace %s\n", entry.Organization)
}
io.Errorf(" Credentials written to %s\n", config.HostsPath())
return nil
}
// resolveAPIBase finds the API for a host, probing the layouts the installer
// writes. Guessing wrong here produces a confusing 404 on every later command,
// so it is settled once, at sign-in, and stored.
func resolveAPIBase(ctx context.Context, f *Factory, host, explicit string) (string, error) {
if explicit != "" {
return strings.TrimRight(explicit, "/"), nil
}
if v := strings.TrimSpace(os.Getenv(config.APIURLEnv)); v != "" {
return strings.TrimRight(v, "/"), nil
}
if host == config.DefaultHost {
return config.DefaultAPIURL(host), nil
}
candidates := config.CandidateAPIURLs(host)
for _, base := range candidates {
if reachable(ctx, f, base) {
if f.Debug {
f.IO.Errorf("* resolved API base %s\n", base)
}
return base, nil
}
}
return "", fmt.Errorf("could not find a Warmbly API for %s. Tried:\n %s\nPass --api-url with the instance's API base URL (the API_PUBLIC_URL it is configured with).", host, strings.Join(candidates, "\n "))
}
// deploymentConfig is the slice of GET /auth/config the CLI uses: enough to
// tell a Warmbly API from any other 200, plus the two URLs a client cannot
// derive on a self-hosted instance.
type deploymentConfig struct {
Registration string `json:"registration"`
AppURL string `json:"app_url"`
WebsocketURL string `json:"websocket_url"`
}
// fetchDeploymentConfig reads the public deployment facts, or nil when the
// base is not a Warmbly API. /health alone would accept any 200.
func fetchDeploymentConfig(ctx context.Context, f *Factory, base string) *deploymentConfig {
client := api.New(base, "", UserAgent())
client.HTTP.Timeout = 8 * time.Second
if f.Debug {
client.Debug = f.IO.ErrOut
}
probeCtx, cancel := context.WithTimeout(ctx, 10*time.Second)
defer cancel()
resp, err := client.Do(probeCtx, api.Request{Method: http.MethodGet, Path: "/auth/config", Anonymous: true})
if err != nil || resp == nil || resp.Status != http.StatusOK {
return nil
}
var probe deploymentConfig
if json.Unmarshal(resp.Body, &probe) != nil || probe.Registration == "" {
return nil
}
return &probe
}
func reachable(ctx context.Context, f *Factory, base string) bool {
return fetchDeploymentConfig(ctx, f, base) != nil
}
func loginWithToken(ctx context.Context, f *Factory, base string) (*config.Host, error) {
io := f.IO
if io.IsStdinTTY() {
io.Errorln(io.Gray("Create a key under Settings > API keys, then paste it here. It is not echoed."))
}
token, err := io.Secret("API key")
if err != nil {
return nil, err
}
token = strings.TrimSpace(token)
if token == "" {
return nil, fmt.Errorf("no key was given")
}
if !strings.HasPrefix(token, apikey.KeyPrefix) {
return nil, fmt.Errorf("that does not look like a Warmbly API key: it should start with %q", apikey.KeyPrefix)
}
entry := &config.Host{APIURL: base, Token: token, AddedAt: time.Now().UTC()}
if err := fillIdentity(ctx, f, entry); err != nil {
return nil, err
}
if cfg := fetchDeploymentConfig(ctx, f, base); cfg != nil {
entry.AppURL = cfg.AppURL
}
return entry, nil
}
func loginWithBrowser(ctx context.Context, f *Factory, cfg *config.Config, base string, scopes uint64) (*config.Host, error) {
io := f.IO
client := api.New(base, "", UserAgent())
if f.Debug {
client.Debug = io.ErrOut
}
machine, _ := os.Hostname()
start, err := startDeviceFlow(ctx, client, machine, scopes)
if err != nil {
return nil, err
}
target := start.VerificationURLComplete
if target == "" {
target = start.VerificationURL
}
io.Errorf("\n %s %s\n", io.Gray("Your code:"), io.Bold(start.UserCode))
io.Errorf(" %s %s\n\n", io.Gray("Approve at:"), target)
if io.IsStdinTTY() {
if ok, cerr := io.Confirm("Open that in your browser now?", true); cerr == nil && ok {
if berr := openBrowser(cfg, target); berr != nil {
io.Errorln(io.Gray("Could not open a browser. Use the link above."))
}
}
}
io.Errorf("%s Waiting for approval (the code expires in %d minutes)\n", io.Gray("…"), start.ExpiresIn/60)
result, err := pollDeviceFlow(ctx, client, start)
if err != nil {
return nil, err
}
entry := &config.Host{
APIURL: base,
AppURL: appURLFromVerification(start.VerificationURL),
Token: result.Token,
User: result.UserEmail,
UserID: result.UserID,
Organization: result.OrganizationName,
OrganizationID: result.OrganizationID,
Scopes: result.ScopeNames,
APIKeyID: result.APIKeyID,
AddedAt: time.Now().UTC(),
}
if entry.User == "" {
// The approval did not carry an identity; ask the API who we are.
if err := fillIdentity(ctx, f, entry); err != nil {
return nil, err
}
}
return entry, nil
}
// appURLFromVerification recovers the dashboard origin from the approval link
// the instance just handed us, which is the instance's own APP_URL and so is
// exact where a hostname guess is not.
func appURLFromVerification(raw string) string {
return strings.TrimSuffix(strings.TrimSpace(raw), "/cli")
}
// fillIdentity calls /me, which both validates the credential and gives the
// host entry the labels `auth status` prints.
func fillIdentity(ctx context.Context, f *Factory, entry *config.Host) error {
client := api.New(entry.APIURL, entry.Token, UserAgent())
if f.Debug {
client.Debug = f.IO.ErrOut
}
var id models.Identity
if err := client.JSON(ctx, api.Request{Method: http.MethodGet, Path: "/me"}, &id); err != nil {
if api.StatusOf(err) == http.StatusUnauthorized {
return fmt.Errorf("that key was rejected by %s. Check it is a key for this instance and has not been revoked.", entry.APIURL)
}
return err
}
entry.User = id.Email
entry.UserID = id.UserID.String()
entry.Scopes = id.Scopes
entry.Organization = id.OrganizationName
if id.OrganizationID != nil {
entry.OrganizationID = id.OrganizationID.String()
}
return nil
}
func newAuthStatusCmd(f *Factory) *cobra.Command {
var showToken bool
cmd := &cobra.Command{
Use: "status",
Short: "Show which hosts you are signed in to",
Long: `Show every signed-in host, who you are on it, and where the credential
came from. Run this first when a command fails with a credential error: an
environment variable nobody remembers exporting is the usual answer.`,
Args: cobra.NoArgs,
RunE: func(c *cobra.Command, _ []string) error {
return runAuthStatus(c.Context(), f, showToken)
},
}
cmd.Flags().BoolVarP(&showToken, "show-token", "t", false, "Print the token itself")
return cmd
}
func runAuthStatus(ctx context.Context, f *Factory, showToken bool) error {
io := f.IO
cfg, err := f.Config()
if err != nil {
return err
}
hosts, err := f.Hosts()
if err != nil {
return err
}
resolved, resolveErr := f.Resolved()
names := hosts.Names()
// A token from the environment points at a host that may not be in the
// file at all; it still deserves a line.
if resolveErr == nil && hosts[resolved.Host] == nil {
names = append(names, resolved.Host)
}
if len(names) == 0 {
io.Errorf("%s Not signed in anywhere.\n", io.Cross())
io.Errorln(io.Gray("Run `warmbly auth login` to sign in."))
return errSilent
}
sort.Strings(names)
failed := false
for _, name := range names {
active := resolveErr == nil && name == resolved.Host
marker := " "
if active {
marker = io.Green("* ")
}
io.Printf("%s%s\n", marker, io.Bold(name))
entry := hosts[name]
token := ""
source := config.HostsPath()
base := ""
if entry != nil {
token, base = entry.Token, entry.APIURL
}
if active {
token, source, base = resolved.Token, resolved.Source, resolved.APIURL
}
if token == "" {
io.Printf(" %s no credential\n", io.Cross())
failed = true
continue
}
probe := &config.Host{APIURL: base, Token: token}
if err := fillIdentity(ctx, f, probe); err != nil {
io.Printf(" %s %s\n", io.Cross(), err.Error())
failed = true
} else {
io.Printf(" %s signed in as %s\n", io.Tick(), io.Bold(probe.User))
if probe.Organization != "" {
io.Printf(" - workspace: %s\n", probe.Organization)
}
io.Printf(" - scopes: %s\n", scopeSummary(probe.Scopes))
}
io.Printf(" - api: %s\n", base)
io.Printf(" - token from: %s\n", source)
if showToken {
io.Printf(" - token: %s\n", token)
} else {
io.Printf(" - token: %s\n", maskToken(token))
}
}
if cfg.ActiveHost != "" && len(names) > 1 {
io.Println()
io.Println(io.Gray("The * host is the one commands use. `warmbly auth switch` changes it."))
}
if failed {
return errSilent
}
return nil
}
// scopeSummary keeps a full-access key from printing twenty-four lines.
func scopeSummary(scopes []string) string {
if len(scopes) == 0 {
return "none reported"
}
if len(scopes) >= len(models.AllAPIPermissions) {
return fmt.Sprintf("all %d", len(scopes))
}
if len(scopes) > 6 {
return fmt.Sprintf("%s and %d more", strings.Join(scopes[:6], ", "), len(scopes)-6)
}
return strings.Join(scopes, ", ")
}
func maskToken(t string) string {
if len(t) <= 12 {
return strings.Repeat("*", len(t))
}
return t[:8] + strings.Repeat("*", 8) + t[len(t)-4:]
}
func newAuthTokenCmd(f *Factory) *cobra.Command {
return &cobra.Command{
Use: "token",
Short: "Print the token the CLI is using",
Long: `Print the active token on stdout and nothing else, so it can be piped
into another tool or exported into a CI environment.`,
Example: ` $ export WARMBLY_TOKEN=$(warmbly auth token)
$ curl -H "Authorization: Bearer $(warmbly auth token)" https://api.warmbly.com/v1/me`,
Args: cobra.NoArgs,
RunE: func(*cobra.Command, []string) error {
r, err := f.Resolved()
if err != nil {
return err
}
f.IO.Println(r.Token)
return nil
},
}
}
func newAuthSwitchCmd(f *Factory) *cobra.Command {
var hostname string
cmd := &cobra.Command{
Use: "switch",
Short: "Change which signed-in host commands use",
Args: cobra.MaximumNArgs(1),
RunE: func(c *cobra.Command, args []string) error {
if len(args) == 1 {
hostname = args[0]
}
cfg, err := f.Config()
if err != nil {
return err
}
hosts, err := f.Hosts()
if err != nil {
return err
}
names := hosts.Names()
if len(names) == 0 {
return fmt.Errorf("not signed in anywhere. Run `warmbly auth login` first.")
}
if hostname == "" {
if len(names) == 1 {
hostname = names[0]
} else {
labels := make([]string, len(names))
for i, n := range names {
labels[i] = n
if hosts[n].User != "" {
labels[i] += " " + f.IO.Gray(hosts[n].User)
}
}
idx, serr := f.IO.Select("Use which host?", labels)
if serr != nil {
return serr
}
hostname = names[idx]
}
}
host := config.NormalizeHost(hostname)
if hosts[host] == nil {
return fmt.Errorf("not signed in to %s. Signed in to: %s", host, strings.Join(names, ", "))
}
cfg.ActiveHost = host
if err := cfg.Save(); err != nil {
return err
}
f.IO.Errorf("%s Now using %s\n", f.IO.Tick(), f.IO.Bold(host))
return nil
},
}
cmd.Flags().StringVar(&hostname, "hostname", "", "Host to switch to")
return cmd
}
func newAuthRefreshCmd(f *Factory) *cobra.Command {
var scopeStr string
cmd := &cobra.Command{
Use: "refresh",
Short: "Sign in again, usually to add scopes",
Long: `Run the browser sign-in again for the active host.
A key's scopes are fixed when it is created, so widening what the CLI may do
means a new key. The old one is revoked once the new one works.`,
Example: ` $ warmbly auth refresh --scopes full
$ warmbly auth refresh --scopes read_campaigns,send_campaigns`,
Args: cobra.NoArgs,
RunE: func(c *cobra.Command, _ []string) error {
r, err := f.Resolved()
if err != nil {
return err
}
old := r.Entry
if err := runAuthLogin(c.Context(), f, r.Host, r.APIURL, false, true, scopeStr, true); err != nil {
return err
}
// Revoke the key we replaced, so refreshing does not accumulate a
// key per run under Settings > API keys.
if old != nil && old.APIKeyID != "" {
if err := revokeKey(c.Context(), f, old.APIKeyID); err != nil && f.Debug {
f.IO.Errorf("* could not revoke the previous key: %v\n", err)
}
}
return nil
},
}
cmd.Flags().StringVarP(&scopeStr, "scopes", "s", "", "Scopes to request: full, read-only, or a list of names")
return cmd
}
func newAuthLogoutCmd(f *Factory) *cobra.Command {
var hostname string
var keepKey bool
cmd := &cobra.Command{
Use: "logout",
Short: "Sign out and revoke this machine's key",
Long: `Forget a host's credential.
The key the sign-in created is revoked on the instance too, so signing out on
a machine you are handing back actually ends its access. --keep-key skips the
revocation, for a key you pasted in and use elsewhere.`,
Args: cobra.NoArgs,
RunE: func(c *cobra.Command, _ []string) error {
return runAuthLogout(c.Context(), f, hostname, keepKey)
},
}
cmd.Flags().StringVar(&hostname, "hostname", "", "Host to sign out of (default: the active one)")
cmd.Flags().BoolVar(&keepKey, "keep-key", false, "Forget the credential locally without revoking it")
return cmd
}
func runAuthLogout(ctx context.Context, f *Factory, hostname string, keepKey bool) error {
io := f.IO
cfg, err := f.Config()
if err != nil {
return err
}
hosts, err := f.Hosts()
if err != nil {
return err
}
host := config.NormalizeHost(hostname)
if hostname == "" {
r, rerr := f.Resolved()
if rerr != nil {
return rerr
}
host = r.Host
}
entry := hosts[host]
if entry == nil {
return fmt.Errorf("not signed in to %s", host)
}
if !f.AssumeYes && io.IsStdinTTY() {
ok, cerr := io.Confirm(fmt.Sprintf("Sign out of %s as %s?", host, entry.User), false)
if cerr != nil {
return cerr
}
if !ok {
return errCancelled
}
}
// Revoke first: if it fails, the credential is still in the file and the
// user can retry, which is better than a live key nobody can reach.
revoked := false
if !keepKey && entry.APIKeyID != "" {
f.hosts = hosts
if err := revokeKeyWith(ctx, f, entry, entry.APIKeyID); err != nil {
io.Errorf("%s Could not revoke the key on %s: %v\n", io.Yellow("!"), host, err)
io.Errorln(io.Gray("Revoke it by hand under Settings > API keys."))
} else {
revoked = true
}
}
delete(hosts, host)
if err := hosts.Save(); err != nil {
return err
}
if cfg.ActiveHost == host {
cfg.ActiveHost = ""
if names := hosts.Names(); len(names) == 1 {
cfg.ActiveHost = names[0]
}
if err := cfg.Save(); err != nil {
return err
}
}
io.Errorf("%s Signed out of %s\n", io.Tick(), io.Bold(host))
if revoked {
io.Errorln(io.Gray("The key this machine was using has been revoked."))
}
return nil
}
func revokeKey(ctx context.Context, f *Factory, keyID string) error {
r, err := f.Resolved()
if err != nil {
return err
}
return revokeKeyWith(ctx, f, r.Entry, keyID)
}
// revokeKeyWith ends a key using the credential in entry.
//
// It asks the instance to revoke the calling credential first: self-revocation
// needs no scope, so it works for a read-only sign-in, where deleting by id
// would be refused. The by-id path is the fallback for an instance that
// predates /api-keys/self, and for revoking a key that is not the caller.
func revokeKeyWith(ctx context.Context, f *Factory, entry *config.Host, keyID string) error {
if entry == nil {
return fmt.Errorf("no credential to revoke with")
}
client := api.New(entry.APIURL, entry.Token, UserAgent())
if f.Debug {
client.Debug = f.IO.ErrOut
}
if keyID == "" || keyID == entry.APIKeyID {
_, err := client.Do(ctx, api.Request{Method: http.MethodDelete, Path: "/api-keys/self"})
if err == nil {
return nil
}
if status := api.StatusOf(err); status != http.StatusNotFound && status != http.StatusMethodNotAllowed {
return err
}
if keyID == "" {
return err
}
}
_, err := client.Do(ctx, api.Request{Method: http.MethodDelete, Path: "/api-keys/" + keyID})
if err != nil && api.StatusOf(err) == http.StatusNotFound {
// Already gone is the outcome we wanted.
return nil
}
return err
}
+144
View File
@@ -0,0 +1,144 @@
package main
import (
"fmt"
"sort"
"strings"
"github.com/spf13/cobra"
"github.com/warmbly/warmbly/internal/cli/config"
)
// browseTargets maps a noun to its dashboard path, so `warmbly browse
// campaigns` does not require anyone to remember the URL layout.
var browseTargets = map[string]string{
"campaigns": "/app/campaigns",
"campaign": "/app/campaigns",
"contacts": "/app/contacts",
"contact": "/app/contacts",
"mailboxes": "/app/emails",
"mailbox": "/app/emails",
"emails": "/app/emails",
"inbox": "/app/unibox",
"unibox": "/app/unibox",
"analytics": "/app/analytics",
"automations": "/app/automations",
"forms": "/app/forms",
"templates": "/app/templates",
"crm": "/app/crm",
"audit": "/app/audit",
"keys": "/app/api-keys",
"api-keys": "/app/api-keys",
"settings": "/app/settings/profile",
"webhooks": "/app/settings/webhooks",
"billing": "/app/settings/billing",
"members": "/app/settings/members",
"deliverability": "/app/deliverability",
}
// browseDetail is the subset that has a per-record page, for `warmbly browse
// campaign <id>`.
var browseDetail = map[string]string{
"campaign": "/app/campaigns",
"campaigns": "/app/campaigns",
"contact": "/app/contacts",
"contacts": "/app/contacts",
"mailbox": "/app/emails",
"mailboxes": "/app/emails",
"automation": "/app/automations",
"form": "/app/forms",
"forms": "/app/forms",
}
func newBrowseCmd(f *Factory) *cobra.Command {
var printOnly bool
cmd := &cobra.Command{
Use: "browse [<section>] [<id>]",
Short: "Open the dashboard in a browser",
GroupID: groupCore,
Long: `Open the Warmbly dashboard for the host you are signed in to.
With a section it goes straight there; with a section and an id it opens that
record. --no-browser prints the URL instead, which is what you want over SSH.`,
Example: ` $ warmbly browse
$ warmbly browse campaigns
$ warmbly browse campaign 6f1c...
$ warmbly browse inbox --no-browser`,
Args: cobra.MaximumNArgs(2),
RunE: func(c *cobra.Command, args []string) error {
cfg, err := f.Config()
if err != nil {
return err
}
r, err := f.Resolved()
if err != nil {
return err
}
path := "/app/emails"
if len(args) > 0 {
section := strings.ToLower(args[0])
if len(args) > 1 {
detail, ok := browseDetail[section]
if !ok {
return fmt.Errorf("%q has no detail page to open by id. Sections with one: %s", args[0], keysOf(browseDetail))
}
path = detail + "/" + args[1]
} else {
target, ok := browseTargets[section]
if !ok {
return fmt.Errorf("nothing to browse called %q. Try one of: %s", args[0], keysOf(browseTargets))
}
path = target
}
}
url := dashboardURL(r) + path
if printOnly || !f.IO.IsStdoutTTY() {
f.IO.Println(url)
return nil
}
f.IO.Errorf("%s Opening %s\n", f.IO.Gray("→"), url)
if err := openBrowser(cfg, url); err != nil {
f.IO.Println(url)
}
return nil
},
}
cmd.Flags().BoolVar(&printOnly, "no-browser", false, "Print the URL instead of opening it")
return cmd
}
// dashboardURL is where this host's dashboard lives. The instance reports its
// own APP_URL at sign-in, which is exact; the derivation below is the fallback
// for a credential that came from the environment and never signed in.
func dashboardURL(r *config.Resolved) string {
if r.Entry != nil && strings.TrimSpace(r.Entry.AppURL) != "" {
return strings.TrimRight(r.Entry.AppURL, "/")
}
return appBaseURL(r.Host)
}
// appBaseURL is the dashboard for a host, following the layout the installer
// writes: app.<host> for a real deployment, the host itself for a local one.
func appBaseURL(host string) string {
host = config.NormalizeHost(host)
if strings.HasPrefix(host, "localhost") || strings.HasPrefix(host, "127.0.0.1") {
return "http://" + host
}
if strings.Contains(host, ":") {
return "http://" + host
}
return "https://app." + host
}
// keysOf lists a target map's names, sorted, for an error message.
func keysOf(m map[string]string) string {
out := make([]string, 0, len(m))
for name := range m {
out = append(out, name)
}
sort.Strings(out)
return strings.Join(out, ", ")
}
+301
View File
@@ -0,0 +1,301 @@
package main
import (
"context"
"encoding/json"
"fmt"
"net/http"
"net/http/httptest"
"strings"
"testing"
"github.com/warmbly/warmbly/internal/cli/api"
"github.com/warmbly/warmbly/internal/cli/config"
"github.com/warmbly/warmbly/internal/models"
)
func TestFillPath(t *testing.T) {
specs := []argSpec{{Name: "id"}, {Name: "step"}}
got, err := fillPath("/campaigns/{id}/steps/{step}", specs, []string{"abc", "def"})
if err != nil {
t.Fatalf("fill: %v", err)
}
if got != "/campaigns/abc/steps/def" {
t.Errorf("path = %q", got)
}
if _, err := fillPath("/campaigns/{id}", specs[:1], []string{" "}); err == nil {
t.Error("an empty argument must be rejected rather than producing /campaigns/")
}
// A tool name can contain characters that need escaping in a path.
got, err = fillPath("/ai/tools/{name}/call", []argSpec{{Name: "name"}}, []string{"a b"})
if err != nil {
t.Fatalf("fill: %v", err)
}
if !strings.Contains(got, "a%20b") {
t.Errorf("path segment was not escaped: %q", got)
}
}
// Every spec has to produce a runnable command: one {} per positional argument
// and no leftovers, or the command is dead on arrival at runtime.
func TestEverySpecPathMatchesItsArguments(t *testing.T) {
seen := map[string]bool{}
for _, r := range resourceSpecs() {
if seen[r.Name] {
t.Errorf("two resources are called %q", r.Name)
}
seen[r.Name] = true
endpoints := map[string]bool{}
for _, e := range r.Endpoints {
if endpoints[e.Name] {
t.Errorf("%s has two %q commands", r.Name, e.Name)
}
endpoints[e.Name] = true
if e.Method == "" || e.Path == "" || e.Short == "" {
t.Errorf("%s %s is missing a method, path or summary", r.Name, e.Name)
}
if !strings.HasPrefix(e.Path, "/") {
t.Errorf("%s %s path %q must be /v1-relative", r.Name, e.Name, e.Path)
}
placeholders := strings.Count(e.Path, "{")
if placeholders != len(e.Args) {
t.Errorf("%s %s has %d placeholders and %d arguments", r.Name, e.Name, placeholders, len(e.Args))
}
args := make([]string, len(e.Args))
for i := range e.Args {
args[i] = "x"
if e.Args[i].Help == "" {
t.Errorf("%s %s argument %q has no help", r.Name, e.Name, e.Args[i].Name)
}
}
if _, err := fillPath(e.Path, e.Args, args); err != nil {
t.Errorf("%s %s: %v", r.Name, e.Name, err)
}
if e.Method == http.MethodGet && e.Body != bodyNone {
t.Errorf("%s %s is a GET with a body", r.Name, e.Name)
}
flags := map[string]bool{}
for _, fl := range e.Flag {
if flags[fl.Name] {
t.Errorf("%s %s declares --%s twice", r.Name, e.Name, fl.Name)
}
flags[fl.Name] = true
if fl.Help == "" {
t.Errorf("%s %s flag --%s has no help", r.Name, e.Name, fl.Name)
}
// -h is cobra's help shorthand; claiming it breaks the command.
if fl.Short == "h" {
t.Errorf("%s %s cannot use -h for --%s", r.Name, e.Name, fl.Name)
}
}
}
}
}
// Building the whole command tree catches the failures cobra reports by
// panicking: a duplicate shorthand, a bad group id.
func TestCommandTreeBuilds(t *testing.T) {
f := NewFactory()
root := newRootCmd(f)
if len(root.Commands()) == 0 {
t.Fatal("no commands registered")
}
for _, c := range root.Commands() {
if c.GroupID == "" && c.Name() != "help" && c.Name() != "completion" {
t.Errorf("%s has no group, so it falls out of the grouped help", c.Name())
}
for _, sub := range c.Commands() {
if sub.Short == "" {
t.Errorf("%s %s has no summary", c.Name(), sub.Name())
}
}
}
}
func TestBuildFields(t *testing.T) {
body, err := buildFields(
[]string{"name=Jane", "note=true"},
[]string{"limit=40", "active=true", "missing=null", "tags[]=a", "tags[]=b", "nested[key]=v"},
strings.NewReader(""),
)
if err != nil {
t.Fatalf("build: %v", err)
}
if body["name"] != "Jane" {
t.Errorf("-f name should stay a string, got %#v", body["name"])
}
if body["note"] != "true" {
t.Errorf("-f keeps values literal, got %#v", body["note"])
}
if body["limit"] != int64(40) {
t.Errorf("-F limit should be a number, got %#v", body["limit"])
}
if body["active"] != true {
t.Errorf("-F active should be a bool, got %#v", body["active"])
}
if body["missing"] != nil {
t.Errorf("-F null should be null, got %#v", body["missing"])
}
tags, _ := body["tags"].([]any)
if len(tags) != 2 {
t.Errorf("key[] should build an array, got %#v", body["tags"])
}
nested, _ := body["nested"].(map[string]any)
if nested["key"] != "v" {
t.Errorf("key[sub] should nest, got %#v", body["nested"])
}
if _, err := buildFields([]string{"broken"}, nil, strings.NewReader("")); err == nil {
t.Error("a field with no = must be rejected")
}
}
func TestSplitArgs(t *testing.T) {
got, err := splitArgs(`campaign list --status "in progress" --q 'x y'`)
if err != nil {
t.Fatalf("split: %v", err)
}
want := []string{"campaign", "list", "--status", "in progress", "--q", "x y"}
if len(got) != len(want) {
t.Fatalf("got %#v, want %#v", got, want)
}
for i := range want {
if got[i] != want[i] {
t.Fatalf("got %#v, want %#v", got, want)
}
}
if _, err := splitArgs(`unbalanced "`); err == nil {
t.Error("an unbalanced quote must be an error, not a silent truncation")
}
// An empty quoted argument is still an argument.
if got, _ := splitArgs(`a "" b`); len(got) != 3 {
t.Errorf("empty quoted argument was dropped: %#v", got)
}
}
func TestExpandAliases(t *testing.T) {
dir := t.TempDir()
t.Setenv(config.DirEnv, dir)
cfg := &config.Config{Aliases: map[string]string{"hot": "campaign list --status active"}}
if err := cfg.Save(); err != nil {
t.Fatalf("save: %v", err)
}
f := NewFactory()
got := expandAliases(f, []string{"hot", "--json"})
want := []string{"campaign", "list", "--status", "active", "--json"}
if strings.Join(got, " ") != strings.Join(want, " ") {
t.Errorf("expanded to %v, want %v", got, want)
}
// A non-alias is untouched.
if got := expandAliases(f, []string{"campaign", "list"}); got[0] != "campaign" {
t.Errorf("a real command was rewritten: %v", got)
}
}
func TestParseScopes(t *testing.T) {
if mask, err := parseScopes(""); err != nil || mask != models.APIPermFullAccess {
t.Errorf("empty should mean full access, got %d %v", mask, err)
}
if mask, err := parseScopes("read-only"); err != nil || mask != models.APIPermReadOnly {
t.Errorf("read-only preset = %d %v", mask, err)
}
mask, err := parseScopes("read_campaigns,SEND_CAMPAIGNS")
if err != nil {
t.Fatalf("named scopes: %v", err)
}
if mask&models.APIPermReadCampaigns == 0 || mask&models.APIPermSendCampaigns == 0 {
t.Errorf("named scopes did not resolve: %d", mask)
}
if _, err := parseScopes("not_a_scope"); err == nil {
t.Error("an unknown scope must be named, not dropped")
}
}
// The device flow is the whole sign-in, so it gets an end-to-end test against
// a server that behaves like the real one: pending, then approved once.
func TestDeviceFlow(t *testing.T) {
polls := 0
srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
switch r.URL.Path {
case "/v1/auth/cli/code":
var req models.CLIAuthStartRequest
_ = json.NewDecoder(r.Body).Decode(&req)
if req.CLIVersion == "" || req.Scopes == 0 {
t.Errorf("the CLI must identify itself and name its scopes: %+v", req)
}
w.WriteHeader(http.StatusCreated)
fmt.Fprint(w, `{"device_code":"dc","user_code":"ABCD-EFGH","verification_uri":"https://app.example/cli","verification_uri_complete":"https://app.example/cli?code=ABCD-EFGH","expires_in":600,"interval":1}`)
case "/v1/auth/cli/poll":
polls++
if polls < 2 {
fmt.Fprint(w, `{"status":"pending"}`)
return
}
fmt.Fprint(w, `{"status":"approved","token":"wmbly_minted","user_email":"jane@example.com","organization_name":"Acme","api_key_id":"key-1","scope_names":["READ_CAMPAIGNS"]}`)
default:
t.Errorf("unexpected path %s", r.URL.Path)
}
}))
defer srv.Close()
client := api.New(srv.URL, "", "test")
start, err := startDeviceFlow(context.Background(), client, "laptop", models.APIPermReadOnly)
if err != nil {
t.Fatalf("start: %v", err)
}
if start.UserCode != "ABCD-EFGH" {
t.Errorf("user code = %q", start.UserCode)
}
result, err := pollDeviceFlow(context.Background(), client, start)
if err != nil {
t.Fatalf("poll: %v", err)
}
if polls < 2 {
t.Errorf("the client stopped polling before approval")
}
if result.Token != "wmbly_minted" || result.UserEmail != "jane@example.com" {
t.Errorf("approval payload lost: %+v", result)
}
}
func TestDeviceFlowStopsOnDenial(t *testing.T) {
srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
fmt.Fprint(w, `{"status":"denied"}`)
}))
defer srv.Close()
client := api.New(srv.URL, "", "test")
_, err := pollDeviceFlow(context.Background(), client, &deviceStart{DeviceCode: "dc", UserCode: "X", Interval: 1, ExpiresIn: 60})
if err == nil || !strings.Contains(err.Error(), "declined") {
t.Errorf("a denial must end the wait, got %v", err)
}
}
func TestAppURLFromVerification(t *testing.T) {
// The instance hands back its own APP_URL with /cli on the end, which is
// exact where a hostname guess is not.
if got := appURLFromVerification("https://app.acme.dev/cli"); got != "https://app.acme.dev" {
t.Errorf("got %q", got)
}
if got := appURLFromVerification("http://localhost:5173/cli"); got != "http://localhost:5173" {
t.Errorf("got %q", got)
}
if got := appURLFromVerification(""); got != "" {
t.Errorf("got %q, want empty", got)
}
}
func TestDashboardURLPrefersWhatTheInstanceReported(t *testing.T) {
r := &config.Resolved{Host: "acme.dev", Entry: &config.Host{AppURL: "https://warmbly.acme.dev/"}}
if got := dashboardURL(r); got != "https://warmbly.acme.dev" {
t.Errorf("got %q, want the reported origin", got)
}
// With nothing reported, fall back to the layout the installer writes.
if got := dashboardURL(&config.Resolved{Host: "acme.dev"}); got != "https://app.acme.dev" {
t.Errorf("fallback = %q", got)
}
}
+142
View File
@@ -0,0 +1,142 @@
package main
import (
"fmt"
"sort"
"strings"
"github.com/spf13/cobra"
"github.com/warmbly/warmbly/internal/cli/config"
)
func newConfigCmd(f *Factory) *cobra.Command {
cmd := &cobra.Command{
Use: "config <command>",
Short: "Read and write the CLI's own settings",
GroupID: groupSetup,
Long: `Read and write ` + config.ConfigPath() + `.
These are preferences, not credentials: credentials live in hosts.yml and are
managed with ` + "`warmbly auth`" + `.`,
}
get := &cobra.Command{
Use: "get <key>",
Short: "Print one setting",
Args: cobra.ExactArgs(1),
RunE: func(*cobra.Command, []string) error { return nil },
}
get.RunE = func(c *cobra.Command, args []string) error {
cfg, err := f.Config()
if err != nil {
return err
}
if !knownKey(args[0]) {
return unknownKey(args[0])
}
f.IO.Println(cfg.Get(args[0]))
return nil
}
set := &cobra.Command{
Use: "set <key> <value>",
Short: "Change one setting",
Example: " $ warmbly config set output json\n $ warmbly config set confirm always",
Args: cobra.ExactArgs(2),
RunE: func(c *cobra.Command, args []string) error {
cfg, err := f.Config()
if err != nil {
return err
}
if err := cfg.Set(args[0], args[1]); err != nil {
return err
}
if err := cfg.Save(); err != nil {
return err
}
f.IO.Errorf("%s %s = %s\n", f.IO.Tick(), args[0], args[1])
return nil
},
}
list := &cobra.Command{
Use: "list",
Aliases: []string{"ls"},
Short: "Show every setting and what it does",
Args: cobra.NoArgs,
RunE: func(*cobra.Command, []string) error {
cfg, err := f.Config()
if err != nil {
return err
}
for _, k := range config.Keys {
value := cfg.Get(k.Name)
if value == "" {
value = f.IO.Gray("(unset, " + k.Default + ")")
}
f.IO.Printf("%-14s %s\n", k.Name, value)
f.IO.Printf("%-14s %s\n", "", f.IO.Gray(k.Help))
}
f.IO.Println()
f.IO.Printf("%s %s\n", f.IO.Gray("config:"), config.ConfigPath())
f.IO.Printf("%s %s\n", f.IO.Gray("hosts: "), config.HostsPath())
return nil
},
}
clear := &cobra.Command{
Use: "clear <key>",
Short: "Reset one setting to its default",
Args: cobra.ExactArgs(1),
RunE: func(c *cobra.Command, args []string) error {
cfg, err := f.Config()
if err != nil {
return err
}
if !knownKey(args[0]) {
return unknownKey(args[0])
}
// Set validates values, so clearing goes through the zero value
// directly rather than through a value it would reject.
switch args[0] {
case "active_host":
cfg.ActiveHost = ""
case "output":
cfg.Output = ""
case "confirm":
cfg.Confirm = ""
case "pager":
cfg.Pager = ""
case "browser":
cfg.Browser = ""
}
if err := cfg.Save(); err != nil {
return err
}
f.IO.Errorf("%s %s reset\n", f.IO.Tick(), args[0])
return nil
},
}
cmd.AddCommand(get, set, list, clear)
return cmd
}
func knownKey(name string) bool {
for _, k := range config.Keys {
if k.Name == name {
return true
}
}
return false
}
func unknownKey(name string) error {
names := make([]string, 0, len(config.Keys))
for _, k := range config.Keys {
names = append(names, k.Name)
}
sort.Strings(names)
return fmt.Errorf("unknown config key %q. Settable keys: %s", name, strings.Join(names, ", "))
}
+151
View File
@@ -0,0 +1,151 @@
package main
import (
"context"
"encoding/json"
"errors"
"fmt"
"net/http"
"os/exec"
"runtime"
"strings"
"time"
"github.com/warmbly/warmbly/internal/cli/api"
"github.com/warmbly/warmbly/internal/cli/config"
"github.com/warmbly/warmbly/internal/models"
"github.com/warmbly/warmbly/internal/version"
)
// The browser half of `warmbly auth login`, RFC 8628 shaped: ask for a code,
// show it, wait for a member to approve it in the browser, collect the key.
type deviceStart struct {
DeviceCode string `json:"device_code"`
UserCode string `json:"user_code"`
VerificationURL string `json:"verification_uri"`
VerificationURLComplete string `json:"verification_uri_complete"`
ExpiresIn int `json:"expires_in"`
Interval int `json:"interval"`
}
type devicePoll struct {
Status string `json:"status"`
Token string `json:"token"`
APIKeyID string `json:"api_key_id"`
ScopeNames []string `json:"scope_names"`
UserID string `json:"user_id"`
UserEmail string `json:"user_email"`
UserName string `json:"user_name"`
OrganizationID string `json:"organization_id"`
OrganizationName string `json:"organization_name"`
}
// startDeviceFlow opens the handshake. The client is anonymous: there is
// nothing to authenticate with yet.
func startDeviceFlow(ctx context.Context, client *api.Client, hostname string, scopes uint64) (*deviceStart, error) {
body, err := json.Marshal(models.CLIAuthStartRequest{
ClientName: "Warmbly CLI",
Hostname: hostname,
CLIVersion: version.String(),
Scopes: scopes,
})
if err != nil {
return nil, err
}
resp, err := client.Do(ctx, api.Request{Method: http.MethodPost, Path: "/auth/cli/code", Body: body, Anonymous: true})
if err != nil {
if api.StatusOf(err) == http.StatusNotFound || api.StatusOf(err) == http.StatusNotImplemented {
return nil, fmt.Errorf("%s does not support browser sign-in for the CLI.\nUse `warmbly auth login --with-token` with an API key from Settings > API keys instead.", client.BaseURL)
}
return nil, err
}
var out deviceStart
if err := json.Unmarshal(resp.Body, &out); err != nil {
return nil, fmt.Errorf("the sign-in handshake returned something unexpected: %w", err)
}
if out.DeviceCode == "" || out.UserCode == "" {
return nil, errors.New("the sign-in handshake returned no code")
}
if out.Interval <= 0 {
out.Interval = 3
}
if out.ExpiresIn <= 0 {
out.ExpiresIn = 600
}
return &out, nil
}
// pollDeviceFlow waits for the browser half. It stops on approval, denial,
// expiry, or the context being cancelled, and never faster than the interval
// the server asked for.
func pollDeviceFlow(ctx context.Context, client *api.Client, start *deviceStart) (*devicePoll, error) {
body, err := json.Marshal(map[string]string{"device_code": start.DeviceCode})
if err != nil {
return nil, err
}
interval := time.Duration(start.Interval) * time.Second
deadline := time.Now().Add(time.Duration(start.ExpiresIn) * time.Second)
for {
select {
case <-ctx.Done():
return nil, ctx.Err()
case <-time.After(interval):
}
if time.Now().After(deadline) {
return nil, fmt.Errorf("the code %s expired before it was approved. Run `warmbly auth login` again.", start.UserCode)
}
resp, err := client.Do(ctx, api.Request{Method: http.MethodPost, Path: "/auth/cli/poll", Body: body, Anonymous: true})
if err != nil {
// The code is gone: expired, or someone else claimed it.
if api.StatusOf(err) == http.StatusNotFound {
return nil, fmt.Errorf("the code %s is no longer valid. Run `warmbly auth login` again.", start.UserCode)
}
// A rate limit or a blip must not end a sign-in someone is
// standing at; back off and keep waiting.
if api.StatusOf(err) == http.StatusTooManyRequests || api.StatusOf(err) == 0 {
interval += time.Second
continue
}
return nil, err
}
var out devicePoll
if err := json.Unmarshal(resp.Body, &out); err != nil {
return nil, err
}
switch out.Status {
case string(models.CLIAuthCodeApproved):
if out.Token == "" {
return nil, errors.New("the approval returned no token. Run `warmbly auth login` again.")
}
return &out, nil
case string(models.CLIAuthCodeDenied):
return nil, errors.New("the request was declined in the browser. Nothing was created.")
case string(models.CLIAuthCodeClaimed):
return nil, errors.New("that code was already used. Run `warmbly auth login` again.")
}
}
}
// openBrowser opens a URL, honouring the browser config key and then BROWSER.
// A failure is never fatal: the URL has already been printed.
func openBrowser(cfg *config.Config, url string) error {
if custom := strings.TrimSpace(cfg.Browser); custom != "" {
parts, err := splitArgs(custom)
if err != nil || len(parts) == 0 {
return fmt.Errorf("the browser config key is not a usable command")
}
return exec.Command(parts[0], append(parts[1:], url)...).Start()
}
switch runtime.GOOS {
case "darwin":
return exec.Command("open", url).Start()
case "windows":
return exec.Command("rundll32", "url.dll,FileProtocolHandler", url).Start()
default:
return exec.Command("xdg-open", url).Start()
}
}
+336
View File
@@ -0,0 +1,336 @@
package main
import (
"context"
"encoding/json"
"fmt"
"net/http"
"net/url"
"strings"
"time"
"github.com/gorilla/websocket"
"github.com/spf13/cobra"
"github.com/warmbly/warmbly/internal/cli/api"
"github.com/warmbly/warmbly/internal/cli/config"
"github.com/warmbly/warmbly/internal/cli/iostreams"
)
// `warmbly events tail` is the terminal view of the developer WebSocket: the
// same stream the dashboard runs on, printed as it happens. It is the fastest
// way to see whether an integration is receiving what you think it is, without
// standing up a public webhook URL first.
//
// The socket speaks the Phoenix channel protocol, serializer 1.0.0: every
// frame is [join_ref, ref, topic, event, payload].
func newEventsCmd(f *Factory) *cobra.Command {
cmd := &cobra.Command{
Use: "events <command>",
Short: "Watch live events as they happen",
GroupID: groupDevelop,
Long: `Subscribe to the workspace's live event stream.
This needs a key with the REALTIME_SUBSCRIBE scope. Sign in again with
` + "`warmbly auth refresh --scopes full`" + ` if the key you have does not carry it.`,
}
cmd.AddCommand(newEventsTailCmd(f))
return cmd
}
func newEventsTailCmd(f *Factory) *cobra.Command {
var (
intents []string
wsURL string
orgID string
compact bool
maxCount int
)
cmd := &cobra.Command{
Use: "tail",
Short: "Stream live events to the terminal",
Long: `Print events as Warmbly publishes them: sends, opens, clicks, replies,
inbox arrivals, campaign state, and the custom events your automations fire.
Filter with --intent, which matches the event type as a case-insensitive
substring, so --intent EMAIL takes EMAIL_SENT, EMAIL_OPENED and EMAIL_RECEIVED.
Intents reduce traffic; they are not a permission boundary, and the key's
scopes still decide what reaches you at all.`,
Example: ` $ warmbly events tail
$ warmbly events tail --intent EMAIL --intent CAMPAIGN
$ warmbly events tail --json | jq 'select(.event_type == "EMAIL_REPLIED")'`,
Args: cobra.NoArgs,
RunE: func(c *cobra.Command, _ []string) error {
return runEventsTail(c.Context(), f, wsURL, orgID, intents, compact, maxCount)
},
}
cmd.Flags().StringArrayVar(&intents, "intent", nil, "Only these event families, for example EMAIL or CAMPAIGN")
cmd.Flags().StringVar(&wsURL, "url", "", "WebSocket URL, when the instance does not advertise one")
cmd.Flags().StringVar(&orgID, "org", "", "Organization to subscribe to (default: the signed-in one)")
cmd.Flags().BoolVar(&compact, "compact", false, "One line per event, even on a terminal")
cmd.Flags().IntVar(&maxCount, "count", 0, "Stop after this many events")
return cmd
}
func runEventsTail(ctx context.Context, f *Factory, wsURL, orgID string, intents []string, compact bool, maxCount int) error {
io := f.IO
r, err := f.Resolved()
if err != nil {
return err
}
if orgID == "" && r.Entry != nil {
orgID = r.Entry.OrganizationID
}
if orgID == "" {
// The key knows its own organization even when the config file does not.
client, cerr := f.Client()
if cerr != nil {
return cerr
}
var id struct {
OrganizationID string `json:"organization_id"`
}
if jerr := client.JSON(ctx, api.Request{Method: http.MethodGet, Path: "/me"}, &id); jerr != nil {
return jerr
}
orgID = id.OrganizationID
}
if orgID == "" {
return fmt.Errorf("this credential is not scoped to a workspace, so there is no org channel to join")
}
endpoint, err := resolveSocketURL(ctx, f, r, wsURL)
if err != nil {
return err
}
target := endpoint
if !strings.Contains(target, "?") {
target += "?"
} else {
target += "&"
}
target += "vsn=1.0.0&token=" + url.QueryEscape(r.Token)
if f.Debug {
io.Errorf("* connecting to %s\n", endpoint)
}
dialer := websocket.Dialer{HandshakeTimeout: 20 * time.Second}
conn, resp, err := dialer.DialContext(ctx, target, nil)
if resp != nil && resp.Body != nil {
// The handshake response body carries the rejection reason and nothing
// the stream needs; the socket itself is what stays open.
defer resp.Body.Close()
}
if err != nil {
if resp != nil && resp.StatusCode == http.StatusForbidden {
return fmt.Errorf("the realtime gateway refused this key.\nIt needs the REALTIME_SUBSCRIBE scope: `warmbly auth refresh --scopes full`.")
}
return fmt.Errorf("could not connect to the realtime gateway at %s: %w\nPass --url if this instance serves it somewhere else.", endpoint, err)
}
defer conn.Close()
topic := "org:" + orgID
payload := map[string]any{}
if len(intents) > 0 {
payload["intents"] = intents
}
if err := conn.WriteJSON([]any{"1", "1", topic, "phx_join", payload}); err != nil {
return err
}
io.Errorf("%s Listening on %s%s\n", io.Gray("…"), io.Bold(topic), intentSuffix(io, intents))
io.Errorln(io.Gray("Ctrl-C to stop."))
// Heartbeats are client-initiated. The join reply carries the cadence the
// server wants; until it arrives, the documented default is safe.
heartbeat := time.NewTicker(25 * time.Second)
defer heartbeat.Stop()
done := make(chan error, 1)
go func() {
seen := 0
for {
var frame []json.RawMessage
if err := conn.ReadJSON(&frame); err != nil {
done <- err
return
}
if len(frame) < 5 {
continue
}
var event string
var body json.RawMessage
_ = json.Unmarshal(frame[3], &event)
body = frame[4]
switch event {
case "phx_reply":
if hb := handleJoinReply(io, body, heartbeat); hb != nil {
done <- hb
return
}
continue
case "phx_close", "phx_error":
done <- fmt.Errorf("the channel closed. Rejoin with `warmbly events tail`.")
return
case "phx_join", "heartbeat":
continue
}
printEvent(f, event, body, compact)
seen++
if maxCount > 0 && seen >= maxCount {
done <- nil
return
}
}
}()
ref := 2
for {
select {
case <-ctx.Done():
_ = conn.WriteMessage(websocket.CloseMessage, websocket.FormatCloseMessage(websocket.CloseNormalClosure, ""))
return nil
case err := <-done:
if err != nil && !websocket.IsCloseError(err, websocket.CloseNormalClosure, websocket.CloseGoingAway) {
return err
}
return nil
case <-heartbeat.C:
ref++
if err := conn.WriteJSON([]any{nil, fmt.Sprint(ref), "phoenix", "heartbeat", map[string]any{}}); err != nil {
return err
}
}
}
}
// handleJoinReply reads the HELLO the org channel answers a join with: it
// carries the heartbeat cadence, so the client does not hardcode one, and it
// is where a rejected join is reported.
func handleJoinReply(io *iostreams.IOStreams, body json.RawMessage, heartbeat *time.Ticker) error {
var reply struct {
Status string `json:"status"`
Response struct {
Role string `json:"role"`
HeartbeatIntervalMS int `json:"heartbeat_interval_ms"`
Seq int64 `json:"seq"`
Reason string `json:"reason"`
} `json:"response"`
}
if err := json.Unmarshal(body, &reply); err != nil {
return nil
}
if reply.Status == "error" {
reason := reply.Response.Reason
if reason == "" {
reason = "the gateway refused the join"
}
return fmt.Errorf("could not join the channel: %s", reason)
}
if reply.Response.HeartbeatIntervalMS > 1000 {
heartbeat.Reset(time.Duration(reply.Response.HeartbeatIntervalMS) * time.Millisecond)
}
if reply.Response.Role != "" {
io.Errorf("%s\n", io.Gray("Joined as "+reply.Response.Role+"."))
}
return nil
}
func printEvent(f *Factory, event string, body json.RawMessage, compact bool) {
io := f.IO
if f.JSONOut || !io.IsStdoutTTY() {
io.Println(strings.TrimSpace(string(body)))
return
}
var fields map[string]any
_ = json.Unmarshal(body, &fields)
stamp := time.Now().Format("15:04:05")
io.Printf("%s %s %s\n", io.Gray(stamp), eventColour(io, event), io.Gray(summarize(fields)))
if compact {
return
}
// The interesting ids, on one indented line, so a terminal stays readable
// while still carrying enough to look something up.
var parts []string
for _, key := range []string{"campaign_id", "contact_id", "email_id", "thread_id", "email_account_id", "name", "entity_type", "action"} {
if v, ok := fields[key]; ok && v != nil && fmt.Sprint(v) != "" {
parts = append(parts, fmt.Sprintf("%s=%v", key, v))
}
}
if len(parts) > 0 {
io.Printf(" %s\n", io.Gray(strings.Join(parts, " ")))
}
}
func eventColour(io *iostreams.IOStreams, event string) string {
switch {
case strings.Contains(event, "FAILED"), strings.Contains(event, "BOUNCE"), strings.Contains(event, "ERROR"):
return io.Red(event)
case strings.Contains(event, "REPLIED"), strings.Contains(event, "BOOKED"):
return io.Green(event)
case strings.Contains(event, "OPENED"), strings.Contains(event, "CLICKED"):
return io.Cyan(event)
default:
return io.Bold(event)
}
}
// summarize is the short human tail of an event line: whichever descriptive
// field the event happens to carry.
func summarize(fields map[string]any) string {
for _, key := range []string{"subject", "email", "to", "contact_email", "campaign_name", "message", "status", "name"} {
if v, ok := fields[key]; ok {
if s := strings.TrimSpace(fmt.Sprint(v)); s != "" && s != "<nil>" {
if len(s) > 70 {
s = s[:69] + "…"
}
return s
}
}
}
return ""
}
// intentSuffix names the filter in the "listening" line, so a stream that goes
// quiet does not look broken when it is only filtered.
func intentSuffix(io *iostreams.IOStreams, intents []string) string {
if len(intents) == 0 {
return ""
}
return io.Gray(" (" + strings.Join(intents, ", ") + " only)")
}
// resolveSocketURL finds the realtime gateway: the flag, then what the
// instance advertises on /auth/config, then the layouts the installer writes.
func resolveSocketURL(ctx context.Context, f *Factory, r *config.Resolved, explicit string) (string, error) {
if explicit != "" {
return explicit, nil
}
client := api.New(r.APIURL, "", UserAgent())
if f.Debug {
client.Debug = f.IO.ErrOut
}
var cfg struct {
WebsocketURL string `json:"websocket_url"`
}
if err := client.JSON(ctx, api.Request{Method: http.MethodGet, Path: "/auth/config", Anonymous: true}, &cfg); err == nil && cfg.WebsocketURL != "" {
return cfg.WebsocketURL, nil
}
host := config.NormalizeHost(r.Host)
if host == config.DefaultHost {
return "wss://realtime." + config.DefaultHost + "/socket/websocket", nil
}
if strings.HasPrefix(host, "localhost") || strings.HasPrefix(host, "127.0.0.1") {
return "ws://" + strings.Split(host, ":")[0] + ":4000/socket/websocket", nil
}
// The installer's proxy and Caddy layouts both put it on ws.<host>.
return "wss://ws." + host + "/socket/websocket", nil
}
+168
View File
@@ -0,0 +1,168 @@
package main
import (
"fmt"
"io"
"os"
"runtime"
"strings"
"github.com/warmbly/warmbly/internal/cli/api"
"github.com/warmbly/warmbly/internal/cli/config"
"github.com/warmbly/warmbly/internal/cli/iostreams"
"github.com/warmbly/warmbly/internal/cli/output"
"github.com/warmbly/warmbly/internal/version"
)
// Factory is what every command is handed: the terminal, the two config files,
// and a way to build an authenticated client. Building it lazily matters,
// because `warmbly auth login` and `warmbly version` must work before there is
// anything to authenticate with.
type Factory struct {
IO *iostreams.IOStreams
// Global flags, bound once on the root command.
HostFlag string
JSONOut bool
Template string
Fields []string
AssumeYes bool
NoColor bool
Debug bool
cfg *config.Config
hosts config.Hosts
}
func NewFactory() *Factory {
return &Factory{IO: iostreams.System()}
}
// UserAgent identifies the CLI in API usage logs, which is how an operator
// tells a CLI call from a script's.
func UserAgent() string {
return fmt.Sprintf("warmbly-cli/%s (%s/%s)", version.String(), runtime.GOOS, runtime.GOARCH)
}
func (f *Factory) Config() (*config.Config, error) {
if f.cfg != nil {
return f.cfg, nil
}
cfg, err := config.Load()
if err != nil {
return nil, err
}
f.cfg = cfg
return cfg, nil
}
func (f *Factory) Hosts() (config.Hosts, error) {
if f.hosts != nil {
return f.hosts, nil
}
hosts, err := config.LoadHosts()
if err != nil {
return nil, err
}
f.hosts = hosts
return hosts, nil
}
// Resolved answers which host and token this invocation uses.
func (f *Factory) Resolved() (*config.Resolved, error) {
cfg, err := f.Config()
if err != nil {
return nil, err
}
hosts, err := f.Hosts()
if err != nil {
return nil, err
}
return config.Resolve(cfg, hosts, f.HostFlag)
}
// Client builds an authenticated client, or explains how to get one.
func (f *Factory) Client() (*api.Client, error) {
r, err := f.Resolved()
if err != nil {
return nil, err
}
c := api.New(r.APIURL, r.Token, UserAgent())
if f.Debug {
c.Debug = f.IO.ErrOut
}
return c, nil
}
// Printer is the renderer for this invocation, honouring the config default
// and then the flags.
func (f *Factory) Printer() *output.Printer {
jsonOut := f.JSONOut
if !jsonOut {
if cfg, err := f.Config(); err == nil && cfg.Get("output") == "json" {
jsonOut = true
}
}
return &output.Printer{IO: f.IO, JSON: jsonOut, Template: f.Template, Fields: f.Fields}
}
// ConfirmSend is the gate in front of anything that puts real mail on the
// wire. Without a terminal it refuses rather than sending, because a script
// that forgot --yes must not discover the omission by mailing strangers.
func (f *Factory) ConfirmSend(what string) error {
if f.AssumeYes {
return nil
}
if !f.IO.IsStdinTTY() {
return fmt.Errorf("%s sends real mail, and there is no terminal to confirm on.\nPass --yes to go ahead.", what)
}
ok, err := f.IO.Confirm(f.IO.Yellow("! ")+what+" sends real mail. Continue?", false)
if err != nil {
return err
}
if !ok {
return errCancelled
}
return nil
}
// ConfirmMutation guards a destructive but non-sending change. It only asks
// when the user opted in with `warmbly config set confirm always`, because
// prompting on every write makes a CLI tiring to use.
func (f *Factory) ConfirmMutation(what string) error {
if f.AssumeYes {
return nil
}
cfg, err := f.Config()
if err != nil || cfg.Get("confirm") != "always" {
return nil
}
if !f.IO.IsStdinTTY() {
return nil
}
ok, cerr := f.IO.Confirm(what+"?", false)
if cerr != nil {
return cerr
}
if !ok {
return errCancelled
}
return nil
}
// bodyFromArg turns a --input value into a request body: a JSON literal, `-`
// for stdin, or @path for a file, the conventions curl taught everyone.
func bodyFromArg(in io.Reader, data string) ([]byte, error) {
data = strings.TrimSpace(data)
if data == "" {
return nil, nil
}
switch {
case data == "-":
return io.ReadAll(in)
case strings.HasPrefix(data, "@"):
return os.ReadFile(strings.TrimPrefix(data, "@"))
default:
return []byte(data), nil
}
}
+101
View File
@@ -0,0 +1,101 @@
// warmbly is the command line interface to Warmbly.
//
// It is the customer's CLI, not the operator's: it signs in as a person, holds
// one credential per host in ~/.config/warmbly, and speaks only the public
// REST API, so it drives the hosted service and any self-hosted instance the
// caller can reach. Everything it can do is bounded by the scopes the sign-in
// approved, and it never serves HTTP.
//
// The other CLI, warmblyctl, is the operator's: it talks to Postgres directly,
// runs inside the backend container, and exists for recovery and accounts. If
// you are asking "what is wrong with this install", that is the one you want.
//
// warmbly auth login
// warmbly campaign list
// warmbly api "/campaigns?limit=10"
package main
import (
"context"
"errors"
"fmt"
"os"
"os/signal"
"github.com/warmbly/warmbly/internal/cli/api"
"github.com/warmbly/warmbly/internal/cli/config"
"github.com/warmbly/warmbly/internal/cli/iostreams"
)
// errCancelled is a user saying no at a prompt. It exits 1 with no error line,
// because the person already knows what happened.
var errCancelled = errors.New("cancelled")
// errSilent lets a command print its own failure and still exit non-zero.
var errSilent = errors.New("")
func main() {
ctx, stop := signal.NotifyContext(context.Background(), os.Interrupt)
defer stop()
f := NewFactory()
root := newRootCmd(f)
root.SetArgs(expandAliases(f, os.Args[1:]))
err := root.ExecuteContext(ctx)
// After the command, never before: the reminder is not worth delaying a
// result for, and it must not appear instead of an error.
if err == nil {
nudgeAboutUpdates(ctx, f)
}
if err != nil {
os.Exit(reportError(f.IO, err))
}
}
// reportError turns whatever came back into one line a person can act on, and
// the exit code a script can branch on:
//
// 1 the command failed
// 2 usage was wrong
// 4 not signed in, or the credential was rejected
func reportError(io *iostreams.IOStreams, err error) int {
if errors.Is(err, errCancelled) {
fmt.Fprintln(io.ErrOut, io.Gray("Cancelled."))
return 1
}
if errors.Is(err, errSilent) {
return 1
}
var noToken *config.ErrNoToken
if errors.As(err, &noToken) {
fmt.Fprintf(io.ErrOut, "%s %s\n", io.Cross(), noToken.Error())
return 4
}
var apiErr *api.Error
if errors.As(err, &apiErr) {
fmt.Fprintf(io.ErrOut, "%s %s\n", io.Cross(), apiErr.Error())
switch {
case apiErr.IsUnauthorized():
fmt.Fprintln(io.ErrOut, io.Gray("The credential was rejected. Run `warmbly auth status` to see which one was used, or `warmbly auth login` to replace it."))
return 4
case apiErr.Status == 403:
fmt.Fprintln(io.ErrOut, io.Gray("The key is missing a scope for this call. `warmbly auth refresh` re-runs the sign-in and can ask for more."))
return 4
}
return 1
}
var noTTY *iostreams.ErrNoTTY
if errors.As(err, &noTTY) {
fmt.Fprintf(io.ErrOut, "%s %s\n", io.Cross(), noTTY.Error())
return 2
}
fmt.Fprintf(io.ErrOut, "%s %s\n", io.Cross(), err.Error())
return 1
}
+338
View File
@@ -0,0 +1,338 @@
package main
import (
"encoding/json"
"fmt"
"net/http"
"net/url"
"strings"
"github.com/spf13/cobra"
"github.com/warmbly/warmbly/internal/cli/api"
"github.com/warmbly/warmbly/internal/cli/output"
)
// The typed commands are a table. Dispatch, flags, help, the request and the
// rendering all come from one row, so covering a new endpoint is adding a row
// rather than writing a command. `warmbly api` covers anything the table does
// not, which is what keeps the table from having to be exhaustive to be useful.
type bodyMode int
const (
bodyNone bodyMode = iota // the endpoint takes no body
bodyOptional // a body may be sent
bodyRequired // a body must be sent
)
type flagKind int
const (
flagString flagKind = iota
flagInt
flagBool
flagStrings
)
// argSpec is one positional argument, filling the next {} in the path.
type argSpec struct {
Name string
Help string
}
// flagSpec is one flag. Query flags become query parameters; the rest become
// body fields, so `campaign create --name X` needs no JSON.
type flagSpec struct {
Name string
Short string
Help string
Kind flagKind
Query bool
// Key overrides the body field or query parameter name, which otherwise
// is the flag name with dashes turned into underscores.
Key string
}
func (f flagSpec) key() string {
if f.Key != "" {
return f.Key
}
return strings.ReplaceAll(f.Name, "-", "_")
}
type endpoint struct {
Name string
Aliases []string
Short string
Long string
Example string
Method string
// Path is /v1-relative and carries one {name} per positional argument.
Path string
Args []argSpec
Flag []flagSpec
Body bodyMode
// Sends marks a command that puts real mail on the wire. Those confirm.
Sends bool
// Paginate offers --all, which walks the cursor.
Paginate bool
// Idempotent offers --idempotency-key.
Idempotent bool
Table output.Table
// Success is what to say on a terminal when there is nothing to tabulate.
Success string
}
type resource struct {
Name string
Aliases []string
Short string
Long string
Group string
Endpoints []endpoint
}
func resourceCommands(f *Factory) []*cobra.Command {
specs := resourceSpecs()
out := make([]*cobra.Command, 0, len(specs))
for _, r := range specs {
out = append(out, buildResource(f, r))
}
return out
}
func buildResource(f *Factory, r resource) *cobra.Command {
cmd := &cobra.Command{
Use: r.Name + " <command>",
Aliases: r.Aliases,
Short: r.Short,
Long: r.Long,
GroupID: r.Group,
}
for _, e := range r.Endpoints {
cmd.AddCommand(buildEndpoint(f, r, e))
}
return cmd
}
func buildEndpoint(f *Factory, r resource, e endpoint) *cobra.Command {
use := e.Name
for _, a := range e.Args {
use += " <" + a.Name + ">"
}
long := e.Long
if long == "" {
long = e.Short + "."
}
if len(e.Args) > 0 {
var lines []string
for _, a := range e.Args {
lines = append(lines, fmt.Sprintf(" <%s> %s", a.Name, a.Help))
}
long += "\n\nArguments:\n" + strings.Join(lines, "\n")
}
if e.Sends {
long += "\n\nThis command sends real mail. It asks before doing so; --yes skips the question."
}
cmd := &cobra.Command{
Use: use,
Aliases: e.Aliases,
Short: e.Short,
Long: long,
Example: e.Example,
Args: cobra.ExactArgs(len(e.Args)),
}
// Flag values are held here so the runner reads whatever cobra parsed.
strs := map[string]*string{}
ints := map[string]*int{}
bools := map[string]*bool{}
slices := map[string]*[]string{}
for _, fl := range e.Flag {
switch fl.Kind {
case flagInt:
ints[fl.Name] = cmd.Flags().IntP(fl.Name, fl.Short, 0, fl.Help)
case flagBool:
bools[fl.Name] = cmd.Flags().BoolP(fl.Name, fl.Short, false, fl.Help)
case flagStrings:
slices[fl.Name] = cmd.Flags().StringSliceP(fl.Name, fl.Short, nil, fl.Help)
default:
strs[fl.Name] = cmd.Flags().StringP(fl.Name, fl.Short, "", fl.Help)
}
}
var (
input string
rawField []string
typField []string
all bool
maxPages int
idemKey string
)
if e.Body != bodyNone {
cmd.Flags().StringVar(&input, "input", "", "Request body: JSON, @file, or - for stdin")
cmd.Flags().StringArrayVarP(&rawField, "raw-field", "f", nil, "Body field as a string: key=value")
cmd.Flags().StringArrayVarP(&typField, "field", "F", nil, "Body field with a guessed type: key=value")
}
if e.Paginate {
cmd.Flags().BoolVar(&all, "all", false, "Fetch every page, not just the first")
cmd.Flags().IntVar(&maxPages, "max-pages", 100, "Stop after this many pages when --all is set")
}
if e.Idempotent {
cmd.Flags().StringVar(&idemKey, "idempotency-key", "", "Idempotency-Key header for a safely retryable write")
}
cmd.RunE = func(c *cobra.Command, args []string) error {
path, err := fillPath(e.Path, e.Args, args)
if err != nil {
return err
}
query := url.Values{}
body := map[string]any{}
for _, fl := range e.Flag {
if !c.Flags().Changed(fl.Name) {
continue
}
var value any
switch fl.Kind {
case flagInt:
value = *ints[fl.Name]
case flagBool:
value = *bools[fl.Name]
case flagStrings:
value = *slices[fl.Name]
default:
value = *strs[fl.Name]
}
if fl.Query {
query.Set(fl.key(), queryString(value))
continue
}
if err := assign(body, fl.key(), value); err != nil {
return err
}
}
raw, err := bodyFromArg(f.IO.In, input)
if err != nil {
return err
}
fields, err := buildFields(rawField, typField, f.IO.In)
if err != nil {
return err
}
for k, v := range fields {
body[k] = v
}
var payload []byte
switch {
case raw != nil && len(body) > 0:
// Merging a literal body with flags would silently pick a winner.
return fmt.Errorf("pass --input or the field flags, not both")
case raw != nil:
if !json.Valid(raw) {
return fmt.Errorf("the request body is not valid JSON")
}
payload = raw
case len(body) > 0:
payload, err = json.Marshal(body)
if err != nil {
return err
}
case e.Body == bodyRequired:
return fmt.Errorf("%s %s needs a body.\nSupply one with the flags above, with -f key=value, or with --input @file.json", r.Name, e.Name)
case e.Body == bodyOptional && e.Method != http.MethodGet:
payload = []byte("{}")
}
if e.Sends {
if err := f.ConfirmSend(fmt.Sprintf("`warmbly %s %s`", r.Name, e.Name)); err != nil {
return err
}
} else if e.Method == http.MethodDelete {
if err := f.ConfirmMutation(fmt.Sprintf("Run `warmbly %s %s`", r.Name, e.Name)); err != nil {
return err
}
}
client, err := f.Client()
if err != nil {
return err
}
req := api.Request{
Method: e.Method,
Path: path,
Query: query,
Body: payload,
IdempotencyKey: idemKey,
}
printer := f.Printer()
if all {
merged, perr := client.Paginate(c.Context(), req, maxPages)
if perr != nil {
return perr
}
return printer.Print(merged, e.Table)
}
resp, err := client.Do(c.Context(), req)
if err != nil {
return err
}
// Nothing to tabulate and a terminal to talk to: say what happened
// rather than printing an empty object.
if len(e.Table.Columns) == 0 && e.Success != "" && !printer.JSON && printer.Template == "" && f.IO.IsStdoutTTY() {
f.IO.Printf("%s %s\n", f.IO.Tick(), e.Success)
return nil
}
return printer.Print(resp.Body, e.Table)
}
return cmd
}
// fillPath substitutes positional arguments into the {} markers, in order.
func fillPath(path string, specs []argSpec, args []string) (string, error) {
for i, spec := range specs {
marker := "{" + spec.Name + "}"
if !strings.Contains(path, marker) {
return "", fmt.Errorf("internal: path %q has no %s", path, marker)
}
value := strings.TrimSpace(args[i])
if value == "" {
return "", fmt.Errorf("<%s> cannot be empty", spec.Name)
}
path = strings.ReplaceAll(path, marker, url.PathEscape(value))
}
if strings.Contains(path, "{") {
return "", fmt.Errorf("internal: path %q still has an unfilled placeholder", path)
}
return path, nil
}
func queryString(v any) string {
switch t := v.(type) {
case string:
return t
case bool:
if t {
return "true"
}
return "false"
case int:
return fmt.Sprint(t)
case []string:
return strings.Join(t, ",")
default:
return fmt.Sprint(t)
}
}
+147
View File
@@ -0,0 +1,147 @@
package main
import (
"fmt"
"strings"
"github.com/spf13/cobra"
)
// Command groups, so `warmbly --help` reads as a product rather than an
// alphabetical dump of forty nouns.
const (
groupCore = "core"
groupWork = "work"
groupData = "data"
groupDevelop = "develop"
groupSetup = "setup"
)
func newRootCmd(f *Factory) *cobra.Command {
cmd := &cobra.Command{
Use: "warmbly <command> <subcommand> [flags]",
Short: "Warmbly from the command line",
Long: `Work with Warmbly from your terminal.
Sign in once with ` + "`warmbly auth login`" + `, then drive campaigns, contacts,
mailboxes and the inbox as yourself. Everything the CLI can do is bounded by
the scopes you approved, on the hosted service or on your own instance.`,
Example: ` $ warmbly auth login
$ warmbly campaign list
$ warmbly mailbox list --json
$ warmbly inbox list --unseen
$ warmbly api "/campaigns?limit=10"`,
SilenceUsage: true,
SilenceErrors: true,
// A bare `warmbly` is a request for the help, not an error.
RunE: func(c *cobra.Command, args []string) error {
if len(args) == 0 {
return c.Help()
}
return fmt.Errorf("unknown command %q. Run `warmbly --help` for the full list.", args[0])
},
}
p := cmd.PersistentFlags()
p.StringVar(&f.HostFlag, "host", "", "Signed-in host to use (default: the active one)")
p.BoolVar(&f.JSONOut, "json", false, "Print the API response as JSON")
p.StringVar(&f.Template, "template", "", "Format the response with a Go template")
p.StringSliceVar(&f.Fields, "fields", nil, "Table columns to keep, comma separated")
p.BoolVar(&f.AssumeYes, "yes", false, "Answer every prompt with yes, including sends")
p.BoolVar(&f.NoColor, "no-color", false, "Never colourise output")
p.BoolVar(&f.Debug, "debug", false, "Print each request to stderr")
cmd.PersistentPreRun = func(*cobra.Command, []string) {
if f.NoColor {
f.IO.SetColor(false)
}
}
cmd.AddGroup(
&cobra.Group{ID: groupCore, Title: "Getting started"},
&cobra.Group{ID: groupWork, Title: "Doing the work"},
&cobra.Group{ID: groupData, Title: "Your data"},
&cobra.Group{ID: groupDevelop, Title: "Building on Warmbly"},
&cobra.Group{ID: groupSetup, Title: "Setting up the CLI"},
)
cmd.AddCommand(newAuthCmd(f))
cmd.AddCommand(newStatusCmd(f))
cmd.AddCommand(newBrowseCmd(f))
cmd.AddCommand(newAPICmd(f))
cmd.AddCommand(newEventsCmd(f))
cmd.AddCommand(newConfigCmd(f))
cmd.AddCommand(newAliasCmd(f))
cmd.AddCommand(newVersionCmd(f))
cmd.AddCommand(newUpgradeCmd(f))
for _, rc := range resourceCommands(f) {
cmd.AddCommand(rc)
}
cmd.SetOut(f.IO.Out)
cmd.SetErr(f.IO.ErrOut)
// The generated completion command has its own group so it does not sit
// under "Getting started" pretending to be a first step.
cmd.SetHelpCommandGroupID(groupSetup)
cmd.SetCompletionCommandGroupID(groupSetup)
return cmd
}
// expandAliases rewrites the argument list when the first word is a user
// alias. Aliases are plain command lines, so `warmbly alias set hot 'campaign
// list --status active'` makes `warmbly hot --json` work.
func expandAliases(f *Factory, args []string) []string {
if len(args) == 0 || strings.HasPrefix(args[0], "-") {
return args
}
cfg, err := f.Config()
if err != nil || len(cfg.Aliases) == 0 {
return args
}
expansion, ok := cfg.Aliases[args[0]]
if !ok {
return args
}
parts, err := splitArgs(expansion)
if err != nil || len(parts) == 0 {
return args
}
return append(parts, args[1:]...)
}
// splitArgs is shell-ish word splitting: quotes group, nothing else is
// special. An alias is a command line, not a shell script.
func splitArgs(in string) ([]string, error) {
var out []string
var cur strings.Builder
var quote rune
started := false
for _, r := range in {
switch {
case quote != 0:
if r == quote {
quote = 0
continue
}
cur.WriteRune(r)
case r == '\'' || r == '"':
quote = r
started = true
case r == ' ' || r == '\t':
if started || cur.Len() > 0 {
out = append(out, cur.String())
cur.Reset()
started = false
}
default:
cur.WriteRune(r)
}
}
if quote != 0 {
return nil, fmt.Errorf("unbalanced quote in %q", in)
}
if started || cur.Len() > 0 {
out = append(out, cur.String())
}
return out, nil
}
+1925
View File
File diff suppressed because it is too large Load Diff
+241
View File
@@ -0,0 +1,241 @@
package main
import (
"context"
"encoding/json"
"fmt"
"net/http"
"net/url"
"strings"
"github.com/spf13/cobra"
"github.com/warmbly/warmbly/internal/cli/api"
"github.com/warmbly/warmbly/internal/cli/iostreams"
"github.com/warmbly/warmbly/internal/models"
)
// `warmbly status` is the one screen answer to "what is happening in my
// workspace right now". It is several calls composed into one view, and every
// section degrades on its own: a key without analytics scope still gets the
// mailbox and inbox lines rather than one failure for the whole command.
func newStatusCmd(f *Factory) *cobra.Command {
return &cobra.Command{
Use: "status",
Short: "What is happening in your workspace right now",
GroupID: groupCore,
Long: `A single screen: who you are, which mailboxes need attention, what is
sending today, and what is waiting for a reply.
Sections you have no scope for are skipped rather than failing the command.`,
Args: cobra.NoArgs,
RunE: func(c *cobra.Command, _ []string) error {
return runStatus(c.Context(), f)
},
}
}
func runStatus(ctx context.Context, f *Factory) error {
io := f.IO
client, err := f.Client()
if err != nil {
return err
}
// --json is one document rather than a rendered screen, so a script gets
// the same information without parsing prose.
bundle := map[string]any{}
collect := func(key, path string, query url.Values) json.RawMessage {
resp, derr := client.Do(ctx, api.Request{Method: http.MethodGet, Path: path, Query: query})
if derr != nil {
if f.Debug {
io.Errorf("* %s: %v\n", path, derr)
}
return nil
}
var parsed any
if json.Unmarshal(resp.Body, &parsed) == nil {
bundle[key] = parsed
}
return resp.Body
}
identity := collect("me", "/me", nil)
mailboxes := collect("mailboxes", "/emails", url.Values{"limit": []string{"100"}})
campaigns := collect("campaigns", "/campaigns", url.Values{"limit": []string{"100"}})
unread := collect("inbox", "/unibox/count", nil)
dashboard := collect("analytics", "/analytics/dashboard", nil)
if f.JSONOut || !io.IsStdoutTTY() {
raw, merr := json.MarshalIndent(bundle, "", " ")
if merr != nil {
return merr
}
io.Println(string(raw))
return nil
}
printIdentity(io, identity)
printMailboxes(io, mailboxes)
printCampaigns(io, campaigns)
printInbox(io, unread, dashboard)
if len(bundle) == 0 {
return fmt.Errorf("nothing could be read with this credential. Run `warmbly auth status` to check it.")
}
return nil
}
func printIdentity(io *iostreams.IOStreams, raw json.RawMessage) {
if raw == nil {
return
}
var id struct {
Email string `json:"email"`
OrganizationName string `json:"organization_name"`
}
if json.Unmarshal(raw, &id) != nil {
return
}
line := io.Bold(id.Email)
if id.OrganizationName != "" {
line += io.Gray(" in ") + io.Bold(id.OrganizationName)
}
io.Printf("%s\n\n", line)
}
func printMailboxes(io *iostreams.IOStreams, raw json.RawMessage) {
if raw == nil {
return
}
var list struct {
Data []struct {
Email string `json:"email"`
Status string `json:"status"`
AuthState string `json:"auth_state"`
CampaignLimit int `json:"campaign_limit"`
Warmup *string `json:"warmup"`
} `json:"data"`
}
if json.Unmarshal(raw, &list) != nil {
return
}
total, warming, capacity, unchecked := len(list.Data), 0, 0, 0
var trouble []string
for _, m := range list.Data {
capacity += m.CampaignLimit
if m.Warmup != nil {
warming++
}
// "unknown" means never checked and never gates sending, so it is not
// a problem to report; only a failing check is.
switch {
case m.Status != "" && m.Status != "active":
trouble = append(trouble, fmt.Sprintf("%s %s", m.Email, io.Red(m.Status)))
case m.AuthState == models.AuthStateFailing:
trouble = append(trouble, fmt.Sprintf("%s %s", m.Email, io.Yellow("SPF, DKIM or DMARC failing")))
case m.AuthState == models.AuthStateUnknown:
unchecked++
}
}
io.Printf("%s\n", io.Gray("MAILBOXES"))
if total == 0 {
io.Printf(" none connected. `warmbly browse mailboxes` opens the dashboard.\n\n")
return
}
io.Printf(" %d connected, %d warming, %d emails/day of campaign capacity\n", total, warming, capacity)
for _, t := range trouble {
io.Printf(" %s %s\n", io.Cross(), t)
}
if len(trouble) == 0 {
io.Printf(" %s all healthy\n", io.Tick())
}
if unchecked > 0 {
io.Printf(" %s\n", io.Gray(fmt.Sprintf("%d never had their authentication checked: `warmbly mailbox recheck <id>`", unchecked)))
}
io.Println()
}
func printCampaigns(io *iostreams.IOStreams, raw json.RawMessage) {
if raw == nil {
return
}
var list struct {
Data []struct {
Name string `json:"name"`
Status string `json:"status"`
} `json:"data"`
}
if json.Unmarshal(raw, &list) != nil {
return
}
counts := map[string]int{}
var active []string
for _, c := range list.Data {
counts[c.Status]++
if c.Status == "active" || c.Status == "running" {
active = append(active, c.Name)
}
}
io.Printf("%s\n", io.Gray("CAMPAIGNS"))
if len(list.Data) == 0 {
io.Printf(" none yet. `warmbly campaign create --name \"My campaign\"` starts one.\n\n")
return
}
var parts []string
for _, status := range []string{"active", "paused", "draft", "completed"} {
if n := counts[status]; n > 0 {
parts = append(parts, fmt.Sprintf("%d %s", n, status))
}
}
io.Printf(" %s\n", strings.Join(parts, ", "))
for i, name := range active {
if i == 5 {
io.Printf(" %s\n", io.Gray(fmt.Sprintf("and %d more sending", len(active)-5)))
break
}
io.Printf(" %s %s\n", io.Green("▸"), name)
}
io.Println()
}
func printInbox(io *iostreams.IOStreams, unread, dashboard json.RawMessage) {
io.Printf("%s\n", io.Gray("INBOX"))
if unread != nil {
var count struct {
Count int `json:"count"`
Total int `json:"total"`
}
if json.Unmarshal(unread, &count) == nil {
n := count.Count
if n == 0 {
n = count.Total
}
if n > 0 {
io.Printf(" %s unread\n", io.Bold(fmt.Sprint(n)))
} else {
io.Printf(" %s nothing unread\n", io.Tick())
}
}
}
if dashboard != nil {
var stats map[string]any
if json.Unmarshal(dashboard, &stats) == nil {
var parts []string
for _, key := range []string{"sent", "opened", "clicked", "replied", "bounced"} {
if v, ok := stats[key]; ok {
parts = append(parts, fmt.Sprintf("%v %s", v, key))
}
}
if len(parts) > 0 {
io.Printf("\n%s\n %s\n", io.Gray("RECENT ACTIVITY"), strings.Join(parts, ", "))
}
}
}
io.Println()
}
+133
View File
@@ -0,0 +1,133 @@
package main
import (
"context"
"fmt"
"os"
"time"
"github.com/spf13/cobra"
"github.com/warmbly/warmbly/internal/cli/config"
"github.com/warmbly/warmbly/internal/cli/update"
"github.com/warmbly/warmbly/internal/version"
)
func newUpgradeCmd(f *Factory) *cobra.Command {
var check bool
cmd := &cobra.Command{
Use: "upgrade",
Aliases: []string{"update", "self-update"},
Short: "Update the CLI to the newest release",
GroupID: groupSetup,
Long: `Replace this binary with the newest published release.
When the CLI came from a package manager it says which command to run instead,
because overwriting a file Homebrew or Scoop owns produces a version that
reverts on their next upgrade.`,
Example: ` $ warmbly upgrade
$ warmbly upgrade --check`,
Args: cobra.NoArgs,
RunE: func(c *cobra.Command, _ []string) error {
return runUpgrade(c.Context(), f, check)
},
}
cmd.Flags().BoolVar(&check, "check", false, "Only report whether a newer release exists")
return cmd
}
func runUpgrade(ctx context.Context, f *Factory, checkOnly bool) error {
io := f.IO
current := version.String()
io.Errorf("%s\n", io.Gray("Checking for a newer release"))
latest, err := update.LatestVersion(ctx, 15*time.Second)
if err != nil {
return fmt.Errorf("could not reach the release feed: %w", err)
}
// The state file is refreshed here too, so an explicit check silences the
// automatic reminder for the next day.
state := config.LoadState()
state.LastUpdateCheck = time.Now().UTC()
state.LatestVersion = latest
_ = state.Save()
if !update.IsNewer(current, latest) {
if current == latest {
io.Printf("%s warmbly %s is the newest release\n", io.Tick(), io.Bold(current))
} else {
// A dev build is not behind, it is simply not one of ours.
io.Printf("%s running %s; the newest release is %s\n", io.Tick(), io.Bold(current), io.Bold(latest))
}
return nil
}
io.Printf("%s %s is available (you have %s)\n", io.Yellow("↑"), io.Bold(latest), current)
if checkOnly {
return nil
}
executable, err := os.Executable()
if err != nil {
return fmt.Errorf("could not find this binary on disk: %w", err)
}
if cmd := update.DetectMethod(executable).UpgradeCommand(); cmd != "" {
io.Println()
io.Printf("This CLI was installed with a package manager. Upgrade it with:\n\n %s\n", io.Bold(cmd))
return nil
}
if !f.AssumeYes && io.IsStdinTTY() {
ok, cerr := io.Confirm(fmt.Sprintf("Replace %s with %s?", executable, latest), true)
if cerr != nil {
return cerr
}
if !ok {
return errCancelled
}
}
if err := update.Replace(ctx, executable, func(step string) {
io.Errorf("%s %s\n", io.Gray("…"), step)
}); err != nil {
return err
}
io.Printf("%s Upgraded to %s\n", io.Tick(), io.Bold(latest))
return nil
}
// nudgeAboutUpdates prints a one-line reminder after a command has finished,
// at most once a day, and only when someone is watching.
//
// It runs after the command's own output so it never delays a result, and
// every failure is swallowed: a version check is not worth an error message,
// and an air-gapped machine must not pay for one on every run.
func nudgeAboutUpdates(ctx context.Context, f *Factory) {
if !f.IO.IsStdoutTTY() || !f.IO.IsStdinTTY() {
return
}
if f.JSONOut || os.Getenv("WARMBLY_NO_UPDATE_CHECK") != "" || os.Getenv("CI") != "" {
return
}
state := config.LoadState()
if latest := state.LatestVersion; latest != "" && update.IsNewer(version.String(), latest) {
f.IO.Errorf("\n%s %s is available. Run %s\n",
f.IO.Yellow("↑"), f.IO.Bold(latest), f.IO.Bold("warmbly upgrade"))
}
if time.Since(state.LastUpdateCheck) < update.CheckInterval {
return
}
// Two seconds is the whole budget: this happens after the user already has
// what they asked for, and a slow network must not make the CLI feel slow.
latest, err := update.LatestVersion(ctx, 2*time.Second)
if err != nil {
return
}
state.LastUpdateCheck = time.Now().UTC()
state.LatestVersion = latest
_ = state.Save()
}
+45
View File
@@ -0,0 +1,45 @@
package main
import (
"encoding/json"
"runtime"
"github.com/spf13/cobra"
"github.com/warmbly/warmbly/internal/version"
)
func newVersionCmd(f *Factory) *cobra.Command {
return &cobra.Command{
Use: "version",
Short: "Print the CLI's version",
GroupID: groupSetup,
Args: cobra.NoArgs,
RunE: func(*cobra.Command, []string) error {
info := version.Current()
if f.JSONOut {
payload, err := json.MarshalIndent(map[string]string{
"version": info.Version,
"commit": info.Commit,
"built": info.BuiltAt,
"go": runtime.Version(),
"os": runtime.GOOS,
"arch": runtime.GOARCH,
}, "", " ")
if err != nil {
return err
}
f.IO.Println(string(payload))
return nil
}
f.IO.Printf("warmbly %s (%s/%s)\n", info.Version, runtime.GOOS, runtime.GOARCH)
if c := version.ShortCommit(); c != "" {
f.IO.Printf("commit %s\n", c)
}
if info.BuiltAt != "" {
f.IO.Printf("built %s\n", info.BuiltAt)
}
return nil
},
}
}
+47 -3
View File
@@ -19,13 +19,16 @@ import (
"github.com/warmbly/warmbly/internal/app/advanced"
"github.com/warmbly/warmbly/internal/app/cipher"
jobs "github.com/warmbly/warmbly/internal/app/consumer"
"github.com/warmbly/warmbly/internal/app/contact"
"github.com/warmbly/warmbly/internal/app/credits"
"github.com/warmbly/warmbly/internal/app/creditwatch"
"github.com/warmbly/warmbly/internal/app/feature"
"github.com/warmbly/warmbly/internal/app/inboxagent"
"github.com/warmbly/warmbly/internal/app/instancesettings"
"github.com/warmbly/warmbly/internal/app/integration"
"github.com/warmbly/warmbly/internal/app/nativeactions"
"github.com/warmbly/warmbly/internal/app/notification"
"github.com/warmbly/warmbly/internal/app/opsnotify"
"github.com/warmbly/warmbly/internal/app/replyclassify"
warmupapp "github.com/warmbly/warmbly/internal/app/warmup"
"github.com/warmbly/warmbly/internal/app/webhook"
@@ -47,6 +50,7 @@ import (
"github.com/warmbly/warmbly/internal/observability"
"github.com/warmbly/warmbly/internal/pkg/encrypt"
"github.com/warmbly/warmbly/internal/pkg/generation"
"github.com/warmbly/warmbly/internal/pkg/geo"
"github.com/warmbly/warmbly/internal/repository"
)
@@ -313,10 +317,18 @@ func main() {
// wiring native actions here a reply-triggered automation's add_tag /
// create_deal / label_email node would fail with "native actions are not
// available". Mirrors the backend wiring.
// The lead-intake actions (create or update contact, add to campaign) write
// through a contact service so a reply-triggered flow in this process gets
// the same plan check, campaign wake and contact.created as the backend.
contactServiceC := contact.NewService(contactRepo, subscriptionRepoConsumer, planRepoConsumer, streamingPublisher)
if aware, ok := contactServiceC.(contact.WebhookAware); ok {
aware.WireWebhooks(webhookService)
}
integrationServiceC.SetNativeActions(nativeactions.Adapter{
Adv: advancedService,
Contacts: contactRepo,
Orgs: orgRepoConsumer,
Adv: advancedService,
Contacts: contactRepo,
Orgs: orgRepoConsumer,
ContactSvc: contactServiceC,
})
// In-app notifications: the reply/bounce/complaint gate fires in THIS
// process (inbox ingest + deliverability ingest run in the consumer), so the
@@ -339,6 +351,22 @@ func main() {
log.Printf("Warning: notification email disabled, EMAIL_NAME/EMAIL_ADDRESS not set: %v", ecErr)
}
notificationService.WireDelivery(notifEmail, integrationServiceC, repository.NewUserRepostory(primaryDB, kmsClient), orgRepoConsumer)
// Operator alerts. The dead-worker detector runs in this process, and a
// stranded fleet is the operator's problem, not a tenant's. Reads the same
// channel list the admin panel writes; a mail transport is optional (the
// chat and webhook transports do not need one).
var opsMailer opsnotify.Mailer
if notifEmail != nil {
if m, ok := notifEmail.(opsnotify.Mailer); ok {
opsMailer = m
}
}
opsNotifierC := opsnotify.NewService(
instancesettings.NewService(instancesettings.NewStore(primaryDB.Pool)),
opsMailer,
config.AppBaseURL(),
)
// Mobile push (APNs) fires from THIS process too: reply/bounce/complaint
// notifications are created here. Redis backs the immediate-then-digest
// window shared with the backend. The sender stays a nil interface (not a
@@ -398,6 +426,7 @@ func main() {
AdminRepo: repository.NewAdminRepository(primaryDB.Pool),
AssignmentService: workerAssignmentSvc,
Notifier: notificationService,
OpsNotifier: opsNotifierC,
TaskRepo: taskRepo,
CampaignRepo: campaignRepo,
CampaignProgressRepo: campaignProgressRepo,
@@ -468,6 +497,14 @@ func main() {
// open/click action chains (advancedService), the open/click analog of the
// reply path. Decodes with the same codec the Rust tracking service writes
// (Avro on Kafka, JSON on NATS).
// GeoIP is optional here as on the backend: it only turns an open or
// click's source network into a country and city on the logs.
geoPath, _ := cfg.LoadGeoDBPath(ctx)
geoloc, gerr := geo.New(geoPath)
if gerr != nil {
log.Printf("GeoIP database not found at %s; engagement locations are disabled.", geoPath)
geoloc, _ = geo.New("")
}
if trackingCfg, terr := cfg.LoadTrackingConsumerConfig(ctx); terr != nil {
log.Println("tracking consumer config unavailable; opens/clicks not consumed:", terr)
} else if trackingConsumer, terr := jobs.NewTrackingConsumer(
@@ -481,11 +518,18 @@ func main() {
contactRepo,
streamingPublisher,
repository.NewTrackingDedupeRepository(primaryDB.Pool),
repository.NewTrackedLinkRepository(primaryDB.Pool),
repository.NewLinkClickRepository(primaryDB.Pool),
advancedService,
verificationEvidence,
repository.NewEmailOpenRepository(primaryDB.Pool),
geoloc,
); terr != nil {
log.Println("tracking consumer unavailable; opens/clicks not consumed:", terr)
} else {
// The engagement prune reads its window from the instance settings on
// every pass, so shortening it in the admin panel needs no restart.
trackingConsumer.WireRetention(instancesettings.NewService(instancesettings.NewStore(primaryDB.Pool)))
defer trackingConsumer.Close()
go func() {
if err := trackingConsumer.Start(ctx); err != nil {
+114
View File
@@ -0,0 +1,114 @@
// The updater is the host-side agent behind "Update and restart" in the admin
// panel. It moves the install forward and restarts the stack, then reports
// progress to the backend. UPDATER_MODE picks how: compose pulls a git
// checkout and rebuilds it, image pins a release tag in the install's .env and
// pulls the published images (the clone-free install.sh install), command
// hands the rebuild to a script of your own.
// See docs/content/docs/development/updates.mdx.
package main
import (
"context"
"errors"
"log"
"net/http"
"os"
"os/signal"
"strconv"
"strings"
"syscall"
"time"
"github.com/warmbly/warmbly/internal/updater"
"github.com/warmbly/warmbly/internal/version"
)
func main() {
log.SetFlags(log.LstdFlags | log.LUTC)
repoDir := getenv("UPDATER_REPO_DIR", "")
if repoDir == "" {
if wd, err := os.Getwd(); err == nil {
repoDir = wd
}
}
cfg := updater.Config{
Mode: updater.Mode(getenv("UPDATER_MODE", string(updater.ModeCompose))),
RepoDir: repoDir,
Remote: getenv("UPDATER_REMOTE", "origin"),
Command: os.Getenv("UPDATER_COMMAND"),
ComposeProject: getenv("UPDATER_COMPOSE_PROJECT", "warmbly"),
ComposeProfiles: getenv("UPDATER_COMPOSE_PROFILES", "updater"),
BackendHealthURL: getenv("UPDATER_BACKEND_HEALTH_URL", "http://backend:8080/health"),
StateDir: getenv("UPDATER_STATE_DIR", "/var/lib/warmbly-updater"),
FetchInterval: duration("UPDATER_FETCH_INTERVAL", 30*time.Minute),
Prune: boolean("UPDATER_PRUNE", true),
AllowDirty: boolean("UPDATER_ALLOW_DIRTY", false),
Version: version.String(),
}
switch cfg.Mode {
case updater.ModeCompose, updater.ModeCommand, updater.ModeImage:
default:
log.Fatalf("updater: UPDATER_MODE must be compose, image or command, got %q", cfg.Mode)
}
runner, err := updater.NewRunner(cfg)
if err != nil {
log.Fatalf("updater: %v", err)
}
// The updater shares the backend's internal token unless given its own.
token := getenv("UPDATER_TOKEN", os.Getenv("INTERNAL_API_TOKEN"))
server, err := updater.NewServer(runner, token)
if err != nil {
log.Fatalf("updater: %v", err)
}
ctx, stop := signal.NotifyContext(context.Background(), os.Interrupt, syscall.SIGTERM)
defer stop()
runner.Start(ctx)
addr := getenv("UPDATER_ADDR", ":8095")
srv := &http.Server{Addr: addr, Handler: server.Handler(), ReadHeaderTimeout: 10 * time.Second}
go func() {
<-ctx.Done()
runner.Stop()
shutdownCtx, cancel := context.WithTimeout(context.Background(), 5*time.Second)
defer cancel()
_ = srv.Shutdown(shutdownCtx)
}()
log.Printf("updater: %s mode, install %s, listening on %s (version %s)", cfg.Mode, cfg.RepoDir, addr, cfg.Version)
if err := srv.ListenAndServe(); err != nil && !errors.Is(err, http.ErrServerClosed) {
log.Fatalf("updater: %v", err)
}
}
func getenv(key, def string) string {
if v := strings.TrimSpace(os.Getenv(key)); v != "" {
return v
}
return def
}
func duration(key string, def time.Duration) time.Duration {
v := strings.TrimSpace(os.Getenv(key))
if v == "" {
return def
}
d, err := time.ParseDuration(v)
if err != nil || d < time.Minute {
return def
}
return d
}
func boolean(key string, def bool) bool {
v := strings.TrimSpace(os.Getenv(key))
if v == "" {
return def
}
b, err := strconv.ParseBool(v)
if err != nil {
return def
}
return b
}
+19 -13
View File
@@ -82,6 +82,11 @@ var apiSpecs = []apiSpec{
{name: "contact import-commit", summary: "Commit a previewed bulk import", method: "POST", path: "/contacts/import/commit", body: bodyRequired},
{name: "contact export", summary: "Export contacts", method: "POST", path: "/contacts/export", body: bodyOptional},
// Suppression list.
{name: "suppression list", summary: "The workspace suppression list: addresses and domains that get no campaign mail", method: "GET", path: "/suppressions", query: []string{"limit", "cursor", "q"}},
{name: "suppression add", summary: "Add addresses or domains to the suppression list", method: "POST", path: "/suppressions", body: bodyRequired},
{name: "suppression remove", summary: "Lift a suppression entry so the address can be emailed again", method: "DELETE", path: "/suppressions/{id}"},
// Mailboxes (email accounts).
{name: "mailbox list", summary: "List connected mailboxes", method: "GET", path: "/emails", query: []string{"limit", "cursor", "q"}},
{name: "mailbox get", summary: "Get one mailbox", method: "GET", path: "/emails/{id}"},
@@ -169,24 +174,25 @@ var apiSpecs = []apiSpec{
// apiFamilyOrder keeps the top-level help stable; maps iterate randomly.
var apiFamilyOrder = []string{
"me", "campaign", "contact", "mailbox", "inbox", "analytics",
"me", "campaign", "contact", "suppression", "mailbox", "inbox", "analytics",
"settings", "webhook", "apikey", "template", "crm", "tool",
}
// apiFamilies drives top-level dispatch and help for the typed commands.
var apiFamilies = map[string]string{
"me": "Who the API key is",
"campaign": "Campaigns and their sequences",
"contact": "Contacts, notes, and imports",
"mailbox": "Connected mailboxes and warmup",
"inbox": "The unified inbox",
"analytics": "Analytics and the audit trail",
"settings": "Organization outreach settings",
"webhook": "Webhook endpoints",
"apikey": "API keys",
"template": "Reply templates",
"crm": "Pipelines, deals, and CRM tasks",
"tool": "AI agent tools (list and call the registry)",
"me": "Who the API key is",
"campaign": "Campaigns and their sequences",
"contact": "Contacts, notes, and imports",
"suppression": "The suppression list: addresses and domains that get no campaign mail",
"mailbox": "Connected mailboxes and warmup",
"inbox": "The unified inbox",
"analytics": "Analytics and the audit trail",
"settings": "Organization outreach settings",
"webhook": "Webhook endpoints",
"apikey": "API keys",
"template": "Reply templates",
"crm": "Pipelines, deals, and CRM tasks",
"tool": "AI agent tools (list and call the registry)",
}
// runAPIResource runs one typed command: `warmblyctl campaign start --id ...`.
+922
View File
@@ -0,0 +1,922 @@
package main
import (
"archive/tar"
"bufio"
"compress/gzip"
"context"
"crypto/sha256"
"encoding/hex"
"encoding/json"
"errors"
"fmt"
"io"
"net/url"
"os"
"os/exec"
"path/filepath"
"sort"
"strings"
"time"
"github.com/warmbly/warmbly/internal/version"
)
// Instance-level backup and restore: one bundle holding everything a Warmbly
// install is, restorable onto a fresh host.
//
// This is deliberately not `warmblyctl org export`. That one moves ONE
// workspace between two running instances and re-seals its secrets for the
// destination's keys. This one moves THE INSTANCE: every workspace, every
// user, the platform admins, the API keys, and the keys those secrets are
// sealed with, onto a host that has nothing yet.
//
// The reason it is one command rather than a documented list of steps is that
// the parts are only a backup together. A pg_dump alone restores an instance
// whose every mailbox credential decrypts to nothing, because the ciphertext
// is in the database and the key that opens it is in .env; the blob root alone
// restores bodies nothing points at. The bundle carries all three and the
// restore refuses to run when the keys do not match, which is the failure this
// command exists to make impossible.
// bundleVersion is the archive layout. The restore refuses a version it does
// not know rather than half-applying an archive it cannot read.
const bundleVersion = 1
// Paths inside the bundle.
const (
manifestPath = "manifest.json"
databasePath = "database.sql"
keysPath = "keys.env"
blobsPrefix = "blobs/"
)
// backupKeys are the environment values that belong in the bundle. The first
// two are unrecoverable: every sealed mailbox credential and every per-org DEK
// is opened with them, so a database without them is not a backup. The rest
// are here because restoring without them signs out every session and breaks
// every worker and websocket connection, which reads as a broken restore.
var backupKeys = []string{
"CREDENTIALS_ENCRYPTION_KEY",
"KMS_LOCAL_MASTER_KEY",
"AUTH_SECRET",
"INTERNAL_API_TOKEN",
"SECRET_KEY_BASE",
}
// unrecoverableKeys are the two the restore checks. A mismatch on either is
// refused, because it produces an instance that looks restored and whose
// mailboxes authenticate against nothing.
var unrecoverableKeys = []string{"CREDENTIALS_ENCRYPTION_KEY", "KMS_LOCAL_MASTER_KEY"}
// manifest describes what a bundle holds, so the restore can say what it is
// about to do before it does it.
type manifest struct {
Version int `json:"version"`
CreatedAt time.Time `json:"created_at"`
AppVersion string `json:"app_version"`
AppURL string `json:"app_url,omitempty"`
Database string `json:"database"`
DatabaseSHA string `json:"database_sha256"`
// Counts are for the confirmation line, not for logic.
Organizations int `json:"organizations"`
Users int `json:"users"`
Mailboxes int `json:"mailboxes"`
// BlobProvider records why blobs are or are not in the bundle. An s3
// install keeps its bodies in the bucket, and the bucket is not ours to
// copy, so the archive says so instead of pretending to be complete.
BlobProvider string `json:"blob_provider"`
BlobRoot string `json:"blob_root,omitempty"`
BlobFiles int `json:"blob_files"`
BlobBytes int64 `json:"blob_bytes"`
// Keys is the list of key names in keys.env, never their values.
Keys []string `json:"keys"`
}
// ---------- backup ----------
func runBackup(ctx context.Context, args []string) error {
fs := newFlagSet("backup")
out := fs.String("out", "", "Path to write the bundle to. Defaults to ./warmbly-backup-<timestamp>.tar.gz")
noKeys := fs.Bool("no-keys", false, "Leave the encryption keys out. The bundle is then not restorable on its own.")
noBlobs := fs.Bool("no-blobs", false, "Leave message bodies, attachments and avatars out. Database only.")
force := fs.Bool("force", false, "Overwrite the output file if it already exists")
if err := fs.Parse(args); err != nil {
return err
}
if err := noExtraArgs(fs); err != nil {
return err
}
target := strings.TrimSpace(*out)
if target == "" {
target = fmt.Sprintf("warmbly-backup-%s.tar.gz", time.Now().UTC().Format("20060102-150405"))
}
if _, err := os.Stat(target); err == nil && !*force {
return fmt.Errorf("%s already exists. Pass --force to overwrite it, or --out with another path.", target)
}
dsn, err := dbEndpoint(ctx)
if err != nil {
return err
}
if _, err := exec.LookPath("pg_dump"); err != nil {
return errors.New("pg_dump is not on PATH, so the database cannot be dumped.\nRun this inside the backend container, where it is installed:\n " + composeExec + "backup --out /data/blobs/warmbly-backup.tar.gz")
}
m := manifest{
Version: bundleVersion,
CreatedAt: time.Now().UTC(),
AppVersion: version.String(),
AppURL: strings.TrimSpace(os.Getenv("APP_URL")),
Database: databaseName(dsn),
}
// Counts first: they are the line that tells an operator this bundle is of
// the instance they meant, and a database that cannot be counted cannot be
// dumped either, so failing here fails before anything is written.
c, err := connect(ctx)
if err != nil {
return err
}
m.Organizations, m.Users, m.Mailboxes = instanceCounts(ctx, c)
c.close()
fmt.Printf("Backing up %s\n", redact(dsn))
dumpFile, dumpSize, dumpSHA, err := dumpDatabase(ctx, dsn)
if err != nil {
return err
}
defer func() {
_ = os.Remove(dumpFile)
}()
m.DatabaseSHA = dumpSHA
fmt.Printf(" database %s\n", humanBytes(dumpSize))
var blobFiles []blobEntry
m.BlobProvider = strings.ToLower(strings.TrimSpace(os.Getenv("BLOB_PROVIDER")))
if m.BlobProvider == "" {
m.BlobProvider = "filesystem"
}
switch {
case *noBlobs:
fmt.Println(" blobs skipped (--no-blobs)")
case m.BlobProvider != "filesystem":
fmt.Printf(" blobs not included: this instance stores them in %s. Back that store up separately.\n", m.BlobProvider)
default:
root := blobRoot()
m.BlobRoot = root
// The documented hand-off writes the bundle into the blob root, because
// that is the one path the container and the host both see. Archiving
// it into itself would grow a bundle by the size of the last one on
// every run, so the output is excluded by path.
blobFiles, err = collectBlobs(root, target)
if err != nil {
return err
}
for _, b := range blobFiles {
m.BlobBytes += b.size
}
m.BlobFiles = len(blobFiles)
fmt.Printf(" blobs %d files, %s from %s\n", m.BlobFiles, humanBytes(m.BlobBytes), root)
}
keys := map[string]string{}
if !*noKeys {
for _, k := range backupKeys {
if v := strings.TrimSpace(os.Getenv(k)); v != "" {
keys[k] = v
m.Keys = append(m.Keys, k)
}
}
sort.Strings(m.Keys)
missing := missingUnrecoverable(keys)
if len(missing) > 0 {
warn("%s not set in this environment, so %s not in the bundle. Restoring it elsewhere will not open sealed mailbox credentials.",
strings.Join(missing, " and "), plural(len(missing), "it is", "they are"))
}
fmt.Printf(" keys %d (%s)\n", len(m.Keys), strings.Join(m.Keys, ", "))
} else {
fmt.Println(" keys skipped (--no-keys); this bundle cannot restore a working instance on its own")
}
if err := writeBundle(target, m, dumpFile, blobFiles, keys); err != nil {
return err
}
st, _ := os.Stat(target)
size := int64(0)
if st != nil {
size = st.Size()
}
fmt.Printf("\nWrote %s (%s)\n", target, humanBytes(size))
steps := []string{
"Copy it off this host. It holds every mailbox credential and the keys that open them,",
"so treat the file as you would the instance itself: 0600, encrypted at rest, off-site.",
"",
"Restore it on another host with:",
" " + composeExec + "restore --file /path/to/" + filepath.Base(target),
}
if len(m.Keys) == 0 {
steps = append(steps, "", "This bundle carries no keys. Copy CREDENTIALS_ENCRYPTION_KEY and KMS_LOCAL_MASTER_KEY", "from this instance's .env by hand, or the restored mailboxes will not connect.")
}
printSteps("Next:", steps)
return nil
}
// ---------- restore ----------
func runRestore(ctx context.Context, args []string) error {
fs := newFlagSet("restore")
file := fs.String("file", "", "Path to a bundle written by warmblyctl backup (required)")
yes := fs.Bool("yes", false, "Skip the typed confirmation. For scripts.")
noBlobs := fs.Bool("no-blobs", false, "Restore the database only, leaving the blob root alone")
force := fs.Bool("force", false, "Restore even though this host's encryption keys differ from the bundle's")
if err := fs.Parse(args); err != nil {
return err
}
if err := noExtraArgs(fs); err != nil {
return err
}
if strings.TrimSpace(*file) == "" {
return errors.New("--file is required. Point it at a bundle written by `warmblyctl backup`.")
}
if _, err := exec.LookPath("psql"); err != nil {
return errors.New("psql is not on PATH, so the database cannot be restored.\nRun this inside the backend container, where it is installed.")
}
m, err := readManifest(*file)
if err != nil {
return err
}
if m.Version != bundleVersion {
return fmt.Errorf("this bundle is layout version %d and this warmblyctl reads version %d. Restore it with the Warmbly it was written by (%s).", m.Version, bundleVersion, m.AppVersion)
}
dsn, err := dbEndpoint(ctx)
if err != nil {
return err
}
fmt.Printf("Bundle %s\n", *file)
fmt.Printf(" written %s by Warmbly %s\n", m.CreatedAt.Local().Format(time.RFC1123), m.AppVersion)
if m.AppURL != "" {
fmt.Printf(" from %s\n", m.AppURL)
}
fmt.Printf(" holds %d organizations, %d users, %d mailboxes\n", m.Organizations, m.Users, m.Mailboxes)
if m.BlobFiles > 0 {
fmt.Printf(" %d blob files, %s\n", m.BlobFiles, humanBytes(m.BlobBytes))
}
fmt.Printf("Target %s\n\n", redact(dsn))
// The key check is the point of this command. A restore onto a host whose
// keys differ produces an instance that looks fine and whose every mailbox
// fails to authenticate, days later, with no error that names the cause.
keysMatch, err := checkRestoreKeys(*file, m, *force)
if err != nil {
return err
}
if !*yes {
fmt.Printf("This REPLACES everything in %s. Every organization, user, campaign and\nmailbox currently on this instance is dropped and replaced by the bundle's.\n\n", databaseName(dsn))
ok, cerr := confirmPhrase("Type 'restore' to continue: ", "restore")
if cerr != nil {
return cerr
}
if !ok {
return errors.New("nothing was changed")
}
fmt.Println()
}
// Checked before anything is dropped: the restore empties the schema first,
// so finding out afterwards that the bundle is truncated leaves the target
// with neither its own data nor the bundle's.
if m.DatabaseSHA != "" {
if err := verifyDump(*file, m.DatabaseSHA); err != nil {
return err
}
fmt.Println(" bundle intact")
}
fmt.Println("Restoring the database...")
if err := restoreDatabase(ctx, dsn, *file); err != nil {
return err
}
fmt.Println(" database restored")
if !*noBlobs && m.BlobFiles > 0 {
root := blobRoot()
n, bytes, rerr := restoreBlobs(*file, root)
if rerr != nil {
return rerr
}
fmt.Printf(" blobs %d files, %s into %s\n", n, humanBytes(bytes), root)
}
steps := []string{}
switch {
case len(m.Keys) == 0:
case keysMatch:
steps = append(steps,
"The bundle's keys match this host, so sealed mailbox credentials open as they did.",
)
default:
steps = append(steps,
"The keys did NOT match and --force was given, so every restored mailbox",
"credential is unreadable. Reconnect each mailbox, or put the bundle's keys",
"in .env and restore again.",
)
}
steps = append(steps,
"Restart the stack so every service picks the restored database up:",
" docker compose -p warmbly restart",
"",
"Then check it:",
" "+composeExec+"status",
)
printSteps("Next:", steps)
return nil
}
// checkRestoreKeys refuses a restore whose ciphertext this host cannot open.
//
// The bundle's keys are compared against the environment rather than written
// anywhere: warmblyctl runs inside a container and the .env that would have to
// change is on the host, so the honest thing is to print the two lines and
// stop, not to half-apply and report success.
// checkRestoreKeys reports whether this host can open the bundle's ciphertext,
// and refuses the restore when it cannot unless force overrides it.
func checkRestoreKeys(file string, m manifest, force bool) (bool, error) {
if len(m.Keys) == 0 {
warn("This bundle carries no encryption keys. If this host's CREDENTIALS_ENCRYPTION_KEY and\nKMS_LOCAL_MASTER_KEY are not the ones the bundle was written with, every restored mailbox\ncredential will fail to decrypt.")
return false, nil
}
bundled, err := readKeys(file)
if err != nil {
return false, err
}
var mismatched, missing []string
for _, k := range unrecoverableKeys {
want, inBundle := bundled[k]
if !inBundle {
continue
}
got := strings.TrimSpace(os.Getenv(k))
switch {
case got == "":
missing = append(missing, k+"="+want)
case got != want:
mismatched = append(mismatched, k+"="+want)
}
}
if len(mismatched) == 0 && len(missing) == 0 {
return true, nil
}
lines := append(append([]string{}, missing...), mismatched...)
msg := strings.Builder{}
msg.WriteString("This host's encryption keys are not the ones the bundle was sealed with.\n")
msg.WriteString("Restoring anyway gives you an instance whose mailbox credentials cannot be\ndecrypted, and there is no way to recover them afterwards.\n\n")
msg.WriteString("Put these in the install's .env on the host, recreate the containers, then run\nthe restore again:\n\n")
for _, l := range lines {
msg.WriteString(" " + l + "\n")
}
if force {
warn("%s", msg.String())
warn("--force was given, so the restore continues. Mailboxes will need reconnecting.")
return false, nil
}
msg.WriteString("\nPass --force only if you accept losing every stored mailbox credential.")
return false, errors.New(msg.String())
}
// ---------- database ----------
// dumpDatabase writes a plain-SQL dump to a temp file and returns its path,
// size and digest. Plain rather than custom format so the bundle can be
// inspected, and so a restore needs psql alone.
func dumpDatabase(ctx context.Context, dsn string) (string, int64, string, error) {
tmp, err := os.CreateTemp("", "warmbly-dump-*.sql")
if err != nil {
return "", 0, "", err
}
path := tmp.Name()
sum := sha256.New()
w := bufio.NewWriterSize(io.MultiWriter(tmp, sum), 1<<20)
// No owner or ACL statements: the destination's database user is whatever
// its own compose file created, and it is never guaranteed to carry the
// same name as this one's.
cmd := pgCommand(ctx, "pg_dump", dsn, "--no-owner", "--no-privileges", "--format=plain")
cmd.Stdout = w
var stderr strings.Builder
cmd.Stderr = &stderr
if rerr := cmd.Run(); rerr != nil {
_ = tmp.Close()
_ = os.Remove(path)
return "", 0, "", fmt.Errorf("pg_dump failed: %s", strings.TrimSpace(stderr.String()))
}
if ferr := w.Flush(); ferr != nil {
_ = tmp.Close()
_ = os.Remove(path)
return "", 0, "", ferr
}
size, _ := tmp.Seek(0, io.SeekCurrent)
if cerr := tmp.Close(); cerr != nil {
_ = os.Remove(path)
return "", 0, "", cerr
}
return path, size, hex.EncodeToString(sum.Sum(nil)), nil
}
// verifyDump reads the bundle's dump once and compares its digest with the one
// the manifest recorded, so a bundle truncated by a failed copy is refused
// rather than half-applied.
func verifyDump(bundle, want string) error {
sql, closeFn, err := openBundleEntry(bundle, databasePath)
if err != nil {
return err
}
defer closeFn()
sum := sha256.New()
if _, cerr := io.Copy(sum, sql); cerr != nil {
return fmt.Errorf("reading the bundle's database dump: %w", cerr)
}
got := hex.EncodeToString(sum.Sum(nil))
if got != want {
return fmt.Errorf("this bundle is damaged: its database dump hashes to %s and the manifest says %s.\nNothing was changed. Copy the bundle again from wherever it came from.", short(got), short(want))
}
return nil
}
// short trims a digest to something a person can compare by eye.
func short(sum string) string {
if len(sum) > 12 {
return sum[:12]
}
return sum
}
// restoreDatabase empties the schema and replays the dump into it. The schema
// is dropped rather than the dump carrying DROP statements, because the
// destination has already had migrations applied at boot and objects the
// bundle does not know about would otherwise survive into the restored
// instance.
func restoreDatabase(ctx context.Context, dsn, bundle string) error {
reset := pgCommand(ctx, "psql", dsn, "-v", "ON_ERROR_STOP=1", "-q",
"-c", "DROP SCHEMA IF EXISTS public CASCADE; CREATE SCHEMA public;")
var resetErr strings.Builder
reset.Stderr = &resetErr
if err := reset.Run(); err != nil {
return fmt.Errorf("could not empty the target schema: %s", strings.TrimSpace(resetErr.String()))
}
sql, closeFn, err := openBundleEntry(bundle, databasePath)
if err != nil {
return err
}
defer closeFn()
load := pgCommand(ctx, "psql", dsn, "-v", "ON_ERROR_STOP=1", "-q")
load.Stdin = sql
var loadErr strings.Builder
load.Stderr = &loadErr
if err := load.Run(); err != nil {
return fmt.Errorf("the dump did not load cleanly, so the database is now empty and unusable.\nFix the cause and run the restore again:\n%s", strings.TrimSpace(loadErr.String()))
}
return nil
}
// pgCommand builds a libpq invocation with the password moved out of argv.
//
// A connection string on the command line is readable by anyone on the host who
// can list processes, and this one opens the database that holds every sealed
// mailbox credential. libpq reads PGPASSWORD from the environment, which is not
// world-readable on Linux, so the DSN that reaches argv carries no password.
func pgCommand(ctx context.Context, name, dsn string, args ...string) *exec.Cmd {
clean, password := splitDSNPassword(dsn)
cmd := exec.CommandContext(ctx, name, append([]string{"--dbname=" + clean}, args...)...) //nolint:gosec // name is a literal at every call site
cmd.Env = os.Environ()
if password != "" {
cmd.Env = append(cmd.Env, "PGPASSWORD="+password)
}
return cmd
}
// splitDSNPassword returns the connection string with its password removed and
// the password separately. A DSN this cannot parse (libpq also accepts
// keyword/value form) is returned unchanged, because a connection that fails is
// worse than a password in argv.
func splitDSNPassword(dsn string) (string, string) {
parsed, err := url.Parse(dsn)
if err != nil || parsed.User == nil {
return dsn, ""
}
password, ok := parsed.User.Password()
if !ok || password == "" {
return dsn, ""
}
parsed.User = url.User(parsed.User.Username())
return parsed.String(), password
}
func instanceCounts(ctx context.Context, c *conn) (orgs, users, mailboxes int) {
_ = c.db.QueryRow(ctx, `SELECT count(*) FROM organizations`).Scan(&orgs)
_ = c.db.QueryRow(ctx, `SELECT count(*) FROM users`).Scan(&users)
_ = c.db.QueryRow(ctx, `SELECT count(*) FROM email_accounts`).Scan(&mailboxes)
return
}
// databaseName is the database a connection string names, for the line that
// tells an operator what is about to be replaced.
func databaseName(dsn string) string {
if i := strings.LastIndex(dsn, "/"); i >= 0 {
name := dsn[i+1:]
if j := strings.IndexAny(name, "?"); j >= 0 {
name = name[:j]
}
if name != "" {
return name
}
}
return "the database"
}
// ---------- blobs ----------
type blobEntry struct {
abs string
rel string
size int64
mode os.FileMode
}
func blobRoot() string {
if v := strings.TrimSpace(os.Getenv("BLOB_FS_ROOT")); v != "" {
return v
}
return "/data/blobs"
}
// collectBlobs walks the blob root, skipping the bundle being written and
// warning about any earlier one still sitting there.
func collectBlobs(root, exclude string) ([]blobEntry, error) {
if abs, aerr := filepath.Abs(exclude); aerr == nil {
exclude = abs
}
info, err := os.Stat(root)
if err != nil {
if os.IsNotExist(err) {
warn("the blob root %s does not exist on this host, so no bodies or attachments are in the bundle.", root)
return nil, nil
}
return nil, err
}
if !info.IsDir() {
return nil, fmt.Errorf("%s is not a directory", root)
}
var out []blobEntry
err = filepath.WalkDir(root, func(path string, d os.DirEntry, werr error) error {
if werr != nil {
return werr
}
// Symlinks are skipped rather than followed: the blob root is written
// by the app, and a link out of it would put arbitrary host files in a
// bundle the operator believes holds message bodies.
if d.IsDir() || !d.Type().IsRegular() {
return nil
}
if abs, aerr := filepath.Abs(path); aerr == nil && abs == exclude {
return nil
}
rel, rerr := filepath.Rel(root, path)
if rerr != nil {
return rerr
}
// An earlier bundle left in the blob root is not blob data, and putting
// it inside this one doubles the archive for nothing.
if !strings.Contains(rel, string(os.PathSeparator)) &&
strings.HasPrefix(rel, "warmbly-") && strings.HasSuffix(rel, ".tar.gz") {
warn("%s looks like an earlier backup and was left out of this one. Delete it: it is sitting in the blob root, where it is backed up over and over.", filepath.Join(root, rel))
return nil
}
st, serr := d.Info()
if serr != nil {
// Vanished between the walk and the stat. A live instance writes
// bodies constantly, and one disappearing is not a failed backup.
if os.IsNotExist(serr) {
return nil
}
return serr
}
out = append(out, blobEntry{abs: path, rel: filepath.ToSlash(rel), size: st.Size(), mode: st.Mode().Perm()})
return nil
})
if err != nil {
return nil, err
}
sort.Slice(out, func(i, j int) bool { return out[i].rel < out[j].rel })
return out, nil
}
// restoreBlobs unpacks the bundle's blob tree under root. Existing files are
// overwritten; files the bundle does not know about are left alone, so a
// restore onto a host that already holds bodies is additive rather than a
// silent deletion.
func restoreBlobs(bundle, root string) (int, int64, error) {
f, gz, tr, err := openBundle(bundle)
if err != nil {
return 0, 0, err
}
defer f.Close()
defer gz.Close()
var n int
var total int64
for {
hdr, nerr := tr.Next()
if nerr == io.EOF {
break
}
if nerr != nil {
return n, total, nerr
}
if hdr.Typeflag != tar.TypeReg || !strings.HasPrefix(hdr.Name, blobsPrefix) {
continue
}
rel := strings.TrimPrefix(hdr.Name, blobsPrefix)
dest, derr := safeJoin(root, rel)
if derr != nil {
return n, total, derr
}
if err := os.MkdirAll(filepath.Dir(dest), 0o750); err != nil {
return n, total, err
}
out, oerr := os.OpenFile(dest, os.O_CREATE|os.O_TRUNC|os.O_WRONLY, os.FileMode(hdr.Mode).Perm()) //nolint:gosec // path is checked by safeJoin
if oerr != nil {
return n, total, oerr
}
written, cerr := io.Copy(out, tr) //nolint:gosec // sizes come from a bundle the operator wrote
_ = out.Close()
if cerr != nil {
return n, total, cerr
}
n++
total += written
}
return n, total, nil
}
// safeJoin refuses a bundle entry that would land outside the destination.
// A backup is an operator's own file, but it is also the kind of file that
// gets emailed around, and a path traversal in one is a host compromise.
func safeJoin(root, rel string) (string, error) {
clean := filepath.Clean("/" + filepath.FromSlash(rel))
joined := filepath.Join(root, clean)
if !strings.HasPrefix(joined, filepath.Clean(root)+string(os.PathSeparator)) {
return "", fmt.Errorf("the bundle holds an entry that would be written outside %s: %s", root, rel)
}
return joined, nil
}
// ---------- bundle io ----------
func writeBundle(target string, m manifest, dumpFile string, blobs []blobEntry, keys map[string]string) error {
// 0600 from creation, not chmod after: the file holds every secret the
// instance has, and a window in which it is world-readable is a window.
f, err := os.OpenFile(target, os.O_CREATE|os.O_TRUNC|os.O_WRONLY, 0o600) //nolint:gosec // operator-supplied output path
if err != nil {
return err
}
defer f.Close()
gz := gzip.NewWriter(f)
tw := tar.NewWriter(gz)
if err := tarBytes(tw, manifestPath, mustJSON(m), 0o600, m.CreatedAt); err != nil {
return err
}
if len(keys) > 0 {
if err := tarBytes(tw, keysPath, []byte(renderKeys(keys)), 0o600, m.CreatedAt); err != nil {
return err
}
}
if err := tarFile(tw, databasePath, dumpFile, 0o600, m.CreatedAt); err != nil {
return err
}
for _, b := range blobs {
if err := tarFile(tw, blobsPrefix+b.rel, b.abs, b.mode, m.CreatedAt); err != nil {
return err
}
}
if err := tw.Close(); err != nil {
return err
}
if err := gz.Close(); err != nil {
return err
}
return f.Close()
}
func tarBytes(tw *tar.Writer, name string, body []byte, mode os.FileMode, when time.Time) error {
if err := tw.WriteHeader(&tar.Header{
Name: name, Mode: int64(mode), Size: int64(len(body)), ModTime: when, Typeflag: tar.TypeReg,
}); err != nil {
return err
}
_, err := tw.Write(body)
return err
}
// tarFile writes one file, tolerating the instance writing to it underneath.
//
// A tar entry's size is fixed by its header, so a file that grows between the
// stat and the copy has to be truncated and one that shrinks has to be padded;
// getting either wrong corrupts the whole archive. A file that disappears is
// skipped, because a live instance is writing message bodies the entire time a
// backup runs and one of them vanishing is not a failed backup.
func tarFile(tw *tar.Writer, name, path string, mode os.FileMode, when time.Time) error {
st, err := os.Stat(path)
if err != nil {
if os.IsNotExist(err) {
return nil
}
return err
}
f, err := os.Open(path) //nolint:gosec // paths come from the instance's own blob root
if err != nil {
if os.IsNotExist(err) {
return nil
}
return err
}
defer f.Close()
size := st.Size()
if err := tw.WriteHeader(&tar.Header{
Name: name, Mode: int64(mode), Size: size, ModTime: when, Typeflag: tar.TypeReg,
}); err != nil {
return err
}
written, err := io.CopyN(tw, f, size)
if err != nil && err != io.EOF {
return err
}
if written < size {
// It shrank. The header is already committed, so the entry is padded
// to the length it promised.
if _, perr := io.CopyN(tw, zeroReader{}, size-written); perr != nil {
return perr
}
}
return nil
}
// zeroReader pads a tar entry whose file shrank while it was being read.
type zeroReader struct{}
func (zeroReader) Read(p []byte) (int, error) {
for i := range p {
p[i] = 0
}
return len(p), nil
}
func openBundle(path string) (*os.File, *gzip.Reader, *tar.Reader, error) {
f, err := os.Open(path) //nolint:gosec // operator-supplied bundle path
if err != nil {
return nil, nil, nil, fmt.Errorf("could not read %s: %w", path, err)
}
gz, gerr := gzip.NewReader(bufio.NewReaderSize(f, 1<<20))
if gerr != nil {
_ = f.Close()
return nil, nil, nil, fmt.Errorf("%s is not a Warmbly bundle (it is not gzip): %w", path, gerr)
}
return f, gz, tar.NewReader(gz), nil
}
// openBundleEntry streams one entry out of the bundle. The caller closes.
func openBundleEntry(path, want string) (io.Reader, func(), error) {
f, gz, tr, err := openBundle(path)
if err != nil {
return nil, func() {}, err
}
closeFn := func() {
_ = gz.Close()
_ = f.Close()
}
for {
hdr, nerr := tr.Next()
if nerr == io.EOF {
closeFn()
return nil, func() {}, fmt.Errorf("%s holds no %s; it is not a Warmbly bundle", path, want)
}
if nerr != nil {
closeFn()
return nil, func() {}, nerr
}
if hdr.Name == want {
return tr, closeFn, nil
}
}
}
func readBundleEntry(path, want string, limit int64) ([]byte, error) {
r, closeFn, err := openBundleEntry(path, want)
if err != nil {
return nil, err
}
defer closeFn()
return io.ReadAll(io.LimitReader(r, limit))
}
func readManifest(path string) (manifest, error) {
var m manifest
raw, err := readBundleEntry(path, manifestPath, 1<<20)
if err != nil {
return m, err
}
if err := json.Unmarshal(raw, &m); err != nil {
return m, fmt.Errorf("%s holds an unreadable manifest: %w", path, err)
}
return m, nil
}
func readKeys(path string) (map[string]string, error) {
raw, err := readBundleEntry(path, keysPath, 1<<20)
if err != nil {
return nil, err
}
out := map[string]string{}
for _, line := range strings.Split(string(raw), "\n") {
line = strings.TrimSpace(line)
if line == "" || strings.HasPrefix(line, "#") {
continue
}
k, v, ok := strings.Cut(line, "=")
if !ok {
continue
}
out[strings.TrimSpace(k)] = strings.TrimSpace(v)
}
return out, nil
}
func renderKeys(keys map[string]string) string {
names := make([]string, 0, len(keys))
for k := range keys {
names = append(names, k)
}
sort.Strings(names)
b := strings.Builder{}
b.WriteString("# Warmbly instance keys. These belong in the destination's .env BEFORE the\n")
b.WriteString("# restore runs; warmblyctl restore refuses to run without them and says so.\n")
for _, k := range names {
b.WriteString(k + "=" + keys[k] + "\n")
}
return b.String()
}
// ---------- small helpers ----------
func missingUnrecoverable(keys map[string]string) []string {
var out []string
for _, k := range unrecoverableKeys {
if keys[k] == "" {
out = append(out, k)
}
}
return out
}
func mustJSON(v any) []byte {
b, err := json.MarshalIndent(v, "", " ")
if err != nil {
return []byte("{}")
}
return append(b, '\n')
}
// confirmPhrase reads one line and reports whether it is the phrase. It needs
// a TTY; piped into a container without one, --yes is the documented path.
func confirmPhrase(prompt, phrase string) (bool, error) {
fmt.Print(prompt)
reader := bufio.NewReader(os.Stdin)
line, err := reader.ReadString('\n')
if err != nil && line == "" {
return false, errors.New("could not read the confirmation. Pass --yes to skip it, or run this without `exec -T`.")
}
return strings.TrimSpace(line) == phrase, nil
}
func plural(n int, one, many string) string {
if n == 1 {
return one
}
return many
}
+10
View File
@@ -69,6 +69,10 @@ func dispatch(ctx context.Context, args []string) error {
return runUser(ctx, args[1:])
case "org":
return runOrg(ctx, args[1:])
case "backup":
return runBackup(ctx, args[1:])
case "restore":
return runRestore(ctx, args[1:])
case "api":
return runAPI(ctx, args[1:])
}
@@ -99,6 +103,8 @@ var commands = []command{
{"user revoke-admin", "Take platform admin away from an account", composeExec + "user revoke-admin --email old@example.com"},
{"user disable-2fa", "Clear an account's TOTP enrollment after a lost authenticator", composeExec + "user disable-2fa --email you@example.com"},
{"hash-password", "Print an argon2 hash for WARMBLY_BOOTSTRAP_PASSWORD_HASH", "printf '%s' 'your-password' | docker compose -p warmbly exec -T backend warmblyctl hash-password"},
{"backup", "Write the whole instance (database, blobs, keys) to one restorable bundle", composeExec + "backup --out /data/blobs/warmbly-backup.tar.gz"},
{"restore", "Restore a bundle onto this instance, replacing everything on it", composeExec + "restore --file /data/blobs/warmbly-backup.tar.gz"},
{"org list", "List the workspaces on this instance with their id, owner, and size", composeExec + "org list"},
{"org export", "Write a whole workspace to a portable archive file", composeExec + "org export --org you@example.com --out /tmp/workspace.warmbly.zip"},
{"org import", "Apply an archive to a workspace on this instance", composeExec + "org import --org you@example.com --file /tmp/workspace.warmbly.zip"},
@@ -146,6 +152,10 @@ Environment:
WARMBLY_API_URL API base URL for the API commands. Defaults to this instance's
API_PUBLIC_URL, then the hosted service.
backup and restore read the same crypto settings as org export/import, plus
BLOB_FS_ROOT for the blob tree. Run them inside the backend container, which
is also where pg_dump and psql live.
org export and org import additionally read the instance's crypto settings,
because a workspace's sealed values have to be opened on the way out and
re-sealed on the way in: KMS_PROVIDER (with KMS_LOCAL_MASTER_KEY or the AWS
+13
View File
@@ -11,6 +11,7 @@ import (
"github.com/warmbly/warmbly/internal/app/instancecheck"
"github.com/warmbly/warmbly/internal/config"
"github.com/warmbly/warmbly/internal/models"
"github.com/warmbly/warmbly/internal/version"
)
// errChecksFailed is how `make doctor` fails a script. The findings are already
@@ -41,6 +42,9 @@ type instanceStatus struct {
Checks []instancecheck.Finding `json:"checks"`
Summary instancecheck.Summary `json:"summary"`
Version string `json:"version"`
Commit string `json:"commit,omitempty"`
}
func runStatus(ctx context.Context, args []string) error {
@@ -120,6 +124,9 @@ func collectStatus(ctx context.Context, c *conn) (*instanceStatus, error) {
AppURL: config.AppBaseURL(),
AppURLSource: appURLSource(),
Version: version.String(),
Commit: version.ShortCommit(),
}
state.NextSteps = nextSteps(state)
state.Checks, state.Summary = runChecks(ctx, c)
@@ -133,7 +140,13 @@ func printStatus(s *instanceStatus) {
}
mail := fmt.Sprintf("%s (%s; %s)", s.MailTransport, mailEffect(s.MailTransport, s.MailDelivers), s.MailTransportSource)
build := s.Version
if s.Commit != "" {
build += " (" + s.Commit + ")"
}
fmt.Println("Instance")
fmt.Printf(" Version %s\n", build)
fmt.Printf(" Accounts %d\n", s.Accounts)
fmt.Printf(" Claimed %s\n", claimed)
fmt.Printf(" Platform admins %d\n", s.AdminCount)
+33 -1
View File
@@ -31,6 +31,32 @@ INTERNAL_API_TOKEN=change-me-internal-token
PRIMARY_DB=postgres://warmbly:warmbly@localhost:5432/warmbly_dev?sslmode=disable
REDIS=redis://localhost:6379
# === Release and storage layout (docker-compose.yml only) ===
# Both keys exist on every app service, so this one compose file either pulls
# the published images or builds this checkout. Pin a release and every later
# `docker compose up` is reproducible; `install.sh` writes these for you.
# WARMBLY_TAG=v1.4.2 # default: prod (a moving tag)
# WARMBLY_IMAGE_PREFIX=ghcr.io/warmbly/warmbly
#
# One variable per store. Compose reads a source starting with / as a bind
# mount and anything else as a named volume, so a path here is the difference
# between "rsync this directory" and "ask docker volume inspect where it went".
# WARMBLY_PG_DATA=/mnt/data/warmbly/postgres
# WARMBLY_BLOBS=/mnt/data/warmbly/blobs
# WARMBLY_NATS_DATA=/mnt/data/warmbly/nats
# WARMBLY_REDIS_DATA=/mnt/data/warmbly/redis
# WARMBLY_WORKER_STATE=/mnt/data/warmbly/worker
# WARMBLY_UPDATER_STATE=/mnt/data/warmbly/updater
# === Database-backed settings, seeded on first boot ===
# The sync budgets and retention windows live in the database and are edited in
# the admin panel under Instance > Instance settings. This seeds them on an
# instance that has never saved that document, so an unattended install can
# ship its data-control answers with the rest of the environment. It is a no-op
# from the first save onwards, so leaving it here never undoes an edit made
# there. https://docs.warmbly.com/development/data-control/
# WARMBLY_SETTINGS_BOOTSTRAP={"sync":{"backfill_days":30},"retention":{"engagement_event_days":30,"form_event_days":30,"audit_log_days":30}}
# === Provider switches (no-cloud defaults) ===
AWS_CONFIG_ENABLED=false # true => read secrets from AWS SSM/Secrets Manager
@@ -99,7 +125,9 @@ GIN_MODE=release # debug | release
# PUBLIC_HOST=192.168.1.50
# API_PUBLIC_URL is the compose input for the API base the frontends and blob
# URLs use (falls back to http://<PUBLIC_HOST>:8080); set it explicitly behind
# a reverse proxy. ENV_LABEL labels the admin panel (compose: WARMBLY_ENV_LABEL).
# a reverse proxy. Recipient unsubscribe links (the List-Unsubscribe header
# and the link in campaign emails) are served on this origin too, so it must
# be reachable from the public internet. ENV_LABEL labels the admin panel (compose: WARMBLY_ENV_LABEL).
# API_PUBLIC_URL=https://api.example.com
# ENV_LABEL=production
APP_URL=http://localhost:5173
@@ -244,6 +272,10 @@ TRACKING_PAGEHIT_RATE_LIMIT_PER_MIN=60
# The header that proxy sets with the client address (x-forwarded-for, or
# cf-connecting-ip behind Cloudflare). Nothing else is read.
# TRACKING_CLIENT_IP_HEADER=x-forwarded-for
# Secret the source-address token in tracking events is keyed with; defaults
# to INTERNAL_API_TOKEN. Set it apart to rotate the token without rotating
# the internal API token.
# TRACKING_IP_HASH_KEY=
# === Forms service (hosted form pages, embeds, submissions) ===
# Its own process (cmd/forms) on FORMS_PORT, so form traffic never touches the
+15 -5
View File
@@ -15,6 +15,8 @@ FROM --platform=$BUILDPLATFORM golang:1.25-alpine AS builder
ARG GO_TAGS=""
ARG TARGETOS TARGETARCH
# Build identity shown in the admin panel; see internal/version.
ARG VERSION="" COMMIT="" BUILT_AT=""
RUN apk add --no-cache git ca-certificates && \
if echo "$GO_TAGS" | grep -qw kafka; then apk add --no-cache gcc musl-dev librdkafka-dev; fi
@@ -27,16 +29,20 @@ RUN --mount=type=cache,target=/go/pkg/mod \
--mount=type=cache,target=/root/.cache/go-build \
set -eux; \
if echo "$GO_TAGS" | grep -qw kafka; then CGO=1; TAGS="musl kafka"; else CGO=0; TAGS=""; fi; \
CGO_ENABLED=$CGO GOOS=$TARGETOS GOARCH=$TARGETARCH go build -tags "$TAGS" -ldflags="-s -w" -o /out/backend ./cmd/backend; \
CGO_ENABLED=0 GOOS=$TARGETOS GOARCH=$TARGETARCH go build -ldflags="-s -w" -o /out/seed ./cmd/seed; \
CGO_ENABLED=0 GOOS=$TARGETOS GOARCH=$TARGETARCH go build -ldflags="-s -w" -o /out/migrate ./cmd/migrate; \
CGO_ENABLED=0 GOOS=$TARGETOS GOARCH=$TARGETARCH go build -ldflags="-s -w" -o /out/warmblyctl ./cmd/warmblyctl
CGO_ENABLED=$CGO GOOS=$TARGETOS GOARCH=$TARGETARCH go build -tags "$TAGS" -ldflags="-s -w -X github.com/warmbly/warmbly/internal/version.Version=$VERSION -X github.com/warmbly/warmbly/internal/version.Commit=$COMMIT -X github.com/warmbly/warmbly/internal/version.BuiltAt=$BUILT_AT" -o /out/backend ./cmd/backend; \
CGO_ENABLED=0 GOOS=$TARGETOS GOARCH=$TARGETARCH go build -ldflags="-s -w -X github.com/warmbly/warmbly/internal/version.Version=$VERSION -X github.com/warmbly/warmbly/internal/version.Commit=$COMMIT -X github.com/warmbly/warmbly/internal/version.BuiltAt=$BUILT_AT" -o /out/seed ./cmd/seed; \
CGO_ENABLED=0 GOOS=$TARGETOS GOARCH=$TARGETARCH go build -ldflags="-s -w -X github.com/warmbly/warmbly/internal/version.Version=$VERSION -X github.com/warmbly/warmbly/internal/version.Commit=$COMMIT -X github.com/warmbly/warmbly/internal/version.BuiltAt=$BUILT_AT" -o /out/migrate ./cmd/migrate; \
CGO_ENABLED=0 GOOS=$TARGETOS GOARCH=$TARGETARCH go build -ldflags="-s -w -X github.com/warmbly/warmbly/internal/version.Version=$VERSION -X github.com/warmbly/warmbly/internal/version.Commit=$COMMIT -X github.com/warmbly/warmbly/internal/version.BuiltAt=$BUILT_AT" -o /out/warmblyctl ./cmd/warmblyctl; \
CGO_ENABLED=0 GOOS=$TARGETOS GOARCH=$TARGETARCH go build -ldflags="-s -w -X github.com/warmbly/warmbly/internal/version.Version=$VERSION -X github.com/warmbly/warmbly/internal/version.Commit=$COMMIT -X github.com/warmbly/warmbly/internal/version.BuiltAt=$BUILT_AT" -o /out/warmbly ./cmd/cli
# Runtime stage
FROM alpine:3.23
ARG GO_TAGS=""
RUN apk add --no-cache ca-certificates tzdata && \
# postgresql-client is here for `warmblyctl backup` and `warmblyctl restore`:
# the instance bundle is a pg_dump and the restore replays it with psql, and the
# backend container is where the CLI already has PRIMARY_DB and the blob root.
RUN apk add --no-cache ca-certificates tzdata postgresql-client && \
if echo "$GO_TAGS" | grep -qw kafka; then apk add --no-cache librdkafka; fi && \
adduser -D -u 1000 warmbly
@@ -55,6 +61,10 @@ COPY --from=builder /out/migrate /app/migrate
# `docker compose exec backend warmblyctl status` and not a path.
COPY --from=builder /out/warmblyctl /usr/local/bin/warmblyctl
# The customer CLI ships alongside it, so an operator who has exec on the box
# can drive the product as well as recover it without installing anything.
COPY --from=builder /out/warmbly /usr/local/bin/warmbly
# Installer script the worker orchestrator uploads + runs over SSH, and serves
# at GET /worker-install.sh. The mode is explicit because COPY otherwise keeps
# the checkout's: on a filesystem without POSIX permissions that is 0700, and
+46
View File
@@ -0,0 +1,46 @@
# The `warmbly` CLI as an image, for CI jobs and anywhere installing a binary
# is more trouble than pulling one:
#
# docker run --rm -e WARMBLY_TOKEN ghcr.io/warmbly/warmbly/cli campaign list
#
# Distroless-style: the CLI is a static binary that talks to one HTTPS API, so
# the runtime needs certificates, timezone data and nothing else.
FROM --platform=$BUILDPLATFORM golang:1.25-alpine AS builder
ARG TARGETOS
ARG TARGETARCH
ARG VERSION=""
ARG COMMIT=""
ARG BUILT_AT=""
WORKDIR /app
COPY go.mod go.sum ./
RUN --mount=type=cache,target=/go/pkg/mod go mod download
COPY . .
RUN --mount=type=cache,target=/go/pkg/mod \
--mount=type=cache,target=/root/.cache/go-build \
CGO_ENABLED=0 GOOS=$TARGETOS GOARCH=$TARGETARCH go build \
-ldflags="-s -w \
-X github.com/warmbly/warmbly/internal/version.Version=$VERSION \
-X github.com/warmbly/warmbly/internal/version.Commit=$COMMIT \
-X github.com/warmbly/warmbly/internal/version.BuiltAt=$BUILT_AT" \
-o /out/warmbly ./cmd/cli
FROM alpine:3.23
RUN apk add --no-cache ca-certificates tzdata && adduser -D -u 1000 warmbly
COPY --from=builder /out/warmbly /usr/local/bin/warmbly
# A container has no browser, so `warmbly auth login` cannot finish here.
# WARMBLY_TOKEN is the documented way in, and the config directory is a volume
# mount point for anyone who would rather bring their hosts.yml.
ENV WARMBLY_CONFIG_DIR=/home/warmbly/.config/warmbly \
WARMBLY_NO_UPDATE_CHECK=1
USER warmbly
WORKDIR /home/warmbly
ENTRYPOINT ["warmbly"]
CMD ["--help"]
+3 -1
View File
@@ -7,6 +7,8 @@ FROM --platform=$BUILDPLATFORM golang:1.25-alpine AS builder
ARG GO_TAGS=""
ARG TARGETOS TARGETARCH
# Build identity shown in the admin panel; see internal/version.
ARG VERSION="" COMMIT="" BUILT_AT=""
RUN apk add --no-cache git ca-certificates && \
if echo "$GO_TAGS" | grep -qw kafka; then apk add --no-cache gcc musl-dev librdkafka-dev; fi
@@ -19,7 +21,7 @@ RUN --mount=type=cache,target=/go/pkg/mod \
--mount=type=cache,target=/root/.cache/go-build \
set -eux; \
if echo "$GO_TAGS" | grep -qw kafka; then CGO=1; TAGS="musl kafka"; else CGO=0; TAGS=""; fi; \
CGO_ENABLED=$CGO GOOS=$TARGETOS GOARCH=$TARGETARCH go build -tags "$TAGS" -ldflags="-s -w" -o /out/consumer ./cmd/consumer
CGO_ENABLED=$CGO GOOS=$TARGETOS GOARCH=$TARGETARCH go build -tags "$TAGS" -ldflags="-s -w -X github.com/warmbly/warmbly/internal/version.Version=$VERSION -X github.com/warmbly/warmbly/internal/version.Commit=$COMMIT -X github.com/warmbly/warmbly/internal/version.BuiltAt=$BUILT_AT" -o /out/consumer ./cmd/consumer
# Runtime stage
FROM alpine:3.23
+5 -1
View File
@@ -9,6 +9,8 @@
# $TARGETARCH (no QEMU).
FROM --platform=$BUILDPLATFORM node:22-alpine AS appbuilder
WORKDIR /app
# No TTY in a build: CI=true makes pnpm reinstall instead of prompting.
ENV CI=true
RUN corepack enable && corepack prepare pnpm@11.9.0 --activate
COPY forms/package.json forms/pnpm-lock.yaml forms/pnpm-workspace.yaml ./
RUN pnpm install --frozen-lockfile
@@ -18,6 +20,8 @@ RUN pnpm build
FROM --platform=$BUILDPLATFORM golang:1.25-alpine AS builder
ARG TARGETOS TARGETARCH
# Build identity shown in the admin panel; see internal/version.
ARG VERSION="" COMMIT="" BUILT_AT=""
RUN apk add --no-cache git ca-certificates
WORKDIR /app
@@ -27,7 +31,7 @@ RUN --mount=type=cache,target=/go/pkg/mod go mod download
COPY . .
RUN --mount=type=cache,target=/go/pkg/mod \
--mount=type=cache,target=/root/.cache/go-build \
CGO_ENABLED=0 GOOS=$TARGETOS GOARCH=$TARGETARCH go build -ldflags="-s -w" -o /out/forms ./cmd/forms
CGO_ENABLED=0 GOOS=$TARGETOS GOARCH=$TARGETARCH go build -ldflags="-s -w -X github.com/warmbly/warmbly/internal/version.Version=$VERSION -X github.com/warmbly/warmbly/internal/version.Commit=$COMMIT -X github.com/warmbly/warmbly/internal/version.BuiltAt=$BUILT_AT" -o /out/forms ./cmd/forms
# Runtime stage
FROM alpine:3.23
+37
View File
@@ -0,0 +1,37 @@
# syntax=docker/dockerfile:1.7
#
# The updater is the host-side agent behind "Update and restart" in the admin
# panel: it pulls the checkout, rebuilds the images and recreates the
# containers. It needs git and the docker CLI with the compose plugin, and the
# docker socket mounted at runtime (see the updater service in
# docker-compose.yml). Builder runs on $BUILDPLATFORM and cross-compiles.
FROM --platform=$BUILDPLATFORM golang:1.25-alpine AS builder
ARG TARGETOS TARGETARCH
ARG VERSION="" COMMIT="" BUILT_AT=""
RUN apk add --no-cache git ca-certificates
WORKDIR /app
COPY go.mod go.sum ./
RUN --mount=type=cache,target=/go/pkg/mod go mod download
COPY . .
RUN --mount=type=cache,target=/go/pkg/mod \
--mount=type=cache,target=/root/.cache/go-build \
CGO_ENABLED=0 GOOS=$TARGETOS GOARCH=$TARGETARCH go build -ldflags="-s -w -X github.com/warmbly/warmbly/internal/version.Version=$VERSION -X github.com/warmbly/warmbly/internal/version.Commit=$COMMIT -X github.com/warmbly/warmbly/internal/version.BuiltAt=$BUILT_AT" -o /out/updater ./cmd/updater
# Runtime: the official docker CLI image already carries the compose plugin.
FROM docker:27-cli
RUN apk add --no-cache git ca-certificates tzdata && mkdir -p /var/lib/warmbly-updater
COPY --from=builder /out/updater /app/updater
# Runs as root on purpose: it holds the docker socket, and chowns what git
# wrote back to the checkout's owner after every update.
EXPOSE 8095
HEALTHCHECK --interval=30s --timeout=3s --start-period=5s \
CMD wget --no-verbose --tries=1 --spider http://127.0.0.1:8095/health || exit 1
ENTRYPOINT ["/app/updater"]
+3 -1
View File
@@ -7,6 +7,8 @@ FROM --platform=$BUILDPLATFORM golang:1.25-alpine AS builder
ARG GO_TAGS=""
ARG TARGETOS TARGETARCH
# Build identity shown in the admin panel; see internal/version.
ARG VERSION="" COMMIT="" BUILT_AT=""
RUN apk add --no-cache git ca-certificates && \
if echo "$GO_TAGS" | grep -qw kafka; then apk add --no-cache gcc musl-dev librdkafka-dev; fi
@@ -19,7 +21,7 @@ RUN --mount=type=cache,target=/go/pkg/mod \
--mount=type=cache,target=/root/.cache/go-build \
set -eux; \
if echo "$GO_TAGS" | grep -qw kafka; then CGO=1; TAGS="musl kafka"; else CGO=0; TAGS=""; fi; \
CGO_ENABLED=$CGO GOOS=$TARGETOS GOARCH=$TARGETARCH go build -tags "$TAGS" -ldflags="-s -w" -o /out/worker ./cmd/worker
CGO_ENABLED=$CGO GOOS=$TARGETOS GOARCH=$TARGETARCH go build -tags "$TAGS" -ldflags="-s -w -X github.com/warmbly/warmbly/internal/version.Version=$VERSION -X github.com/warmbly/warmbly/internal/version.Commit=$COMMIT -X github.com/warmbly/warmbly/internal/version.BuiltAt=$BUILT_AT" -o /out/worker ./cmd/worker
# Runtime stage
FROM alpine:3.23
+11
View File
@@ -12,6 +12,7 @@ them is [Deploying without Docker](https://docs.warmbly.com/development/bare-met
| `warmbly-tracking.service` | `/opt/warmbly/bin/tracking` | `/etc/warmbly/warmbly.env` |
| `warmbly-realtime.service` | `/opt/warmbly/realtime/bin/realtime start` | `/etc/warmbly/warmbly.env` |
| `warmbly-worker.service` | `/opt/warmbly/bin/worker` | `/etc/warmbly/worker.env` |
| `warmbly-updater.service` | `/opt/warmbly/bin/updater` | `/etc/warmbly/updater.env` |
Every unit runs as the unprivileged `warmbly` user, may write only under
`/var/lib/warmbly` (blob storage), and restarts on failure. Copy them to
@@ -21,3 +22,13 @@ Every unit runs as the unprivileged `warmbly` user, may write only under
`deploy/config/env.example` is the template for `warmbly.env`; the worker env
file is what `POST /api/v1/workers/enroll` returns for an enrollment token.
`warmbly-updater.service` is the exception to the unprivileged rule: it runs as
the user who owns the checkout (`deploy` in the unit; change it) and drives
`scripts/upgrade-bare-metal.sh`, which builds unprivileged and then runs
`warmbly-install-release.sh` (installed root-owned at
`/usr/local/sbin/warmbly-install-release`) through sudo. That installer takes no
arguments, touches only fixed paths and refuses symlinks, and is the single
command to allow in sudoers. `updater.env` holds only `UPDATER_TOKEN` (the
backend's `INTERNAL_API_TOKEN`). See
[Updates](https://docs.warmbly.com/development/updates/).
+100
View File
@@ -0,0 +1,100 @@
#!/usr/bin/env bash
#
# The privileged half of a bare-metal upgrade: install the artifacts that
# scripts/upgrade-bare-metal.sh built under /opt/warmbly/src/out and friends,
# then restart the units, backend first. It takes no arguments and touches only
# fixed paths, so it is the one command the checkout's owner may run through
# sudo without a password. Install it root-owned and not writable by that user:
#
# sudo install -o root -g root -m 0755 deploy/systemd/warmbly-install-release.sh /usr/local/sbin/warmbly-install-release
# deploy ALL=(root) NOPASSWD: /usr/local/sbin/warmbly-install-release # visudo
#
# A symlink under the build output is refused: a caller who owns the checkout
# must not be able to make root read or install a file from somewhere else.
set -euo pipefail
SRC=/opt/warmbly/src
PREFIX=/opt/warmbly
HEALTH=http://127.0.0.1:8080/health
[[ "$(id -u)" -eq 0 ]] || { echo "run through sudo" >&2; exit 1; }
[[ $# -eq 0 ]] || { echo "takes no arguments" >&2; exit 2; }
log() { printf '==> %s\n' "$*"; }
regular() { [[ -f "$1" && ! -L "$1" ]]; }
# A directory qualifies only when nothing inside it is a symlink either: the
# checkout's owner must not be able to point root at a file elsewhere.
plain_dir() {
[[ -d "$1" && ! -L "$1" ]] || return 1
if find "$1" -type l -print -quit | grep -q .; then
log "refusing $1: it contains a symlink"
return 1
fi
}
has_unit() { systemctl list-unit-files "warmbly-$1.service" --no-legend 2>/dev/null | grep -q .; }
log "installing binaries"
for bin in backend forms consumer worker migrate warmblyctl updater; do
f="$SRC/out/$bin"
if regular "$f"; then
install -o root -g root -m 0755 "$f" "$PREFIX/bin/$bin"
fi
done
ln -sf "$PREFIX/bin/warmblyctl" /usr/local/bin/warmblyctl
if has_unit tracking && regular "$SRC/tracking/target/release/tracking"; then
install -o root -g root -m 0755 "$SRC/tracking/target/release/tracking" "$PREFIX/bin/tracking"
fi
if has_unit realtime && plain_dir "$SRC/realtime/_build/prod/rel/realtime"; then
log "installing realtime"
rm -rf "$PREFIX/realtime"
cp -r --no-dereference "$SRC/realtime/_build/prod/rel/realtime" "$PREFIX/realtime"
chown -R warmbly:warmbly "$PREFIX/realtime"
fi
# The runtime config.js is written by hand on a bare-metal install and a
# rebuilt dist/ would drop it, so it is kept across the copy.
for app in web admin; do
if plain_dir "$SRC/$app/dist" && plain_dir "$PREFIX/$app"; then
log "installing $app"
cfg="$(mktemp)"
[[ -f "$PREFIX/$app/config.js" ]] && cp "$PREFIX/$app/config.js" "$cfg"
rm -rf "$PREFIX/$app"
cp -r --no-dereference "$SRC/$app/dist" "$PREFIX/$app"
[[ -s "$cfg" ]] && cp --remove-destination "$cfg" "$PREFIX/$app/config.js"
rm -f "$cfg"
chown -R root:root "$PREFIX/$app"
chmod -R a+rX "$PREFIX/$app"
fi
done
if plain_dir "$SRC/forms/dist" && plain_dir "$PREFIX/forms"; then
log "installing forms"
rm -rf "$PREFIX/forms/dist"
cp -r --no-dereference "$SRC/forms/dist" "$PREFIX/forms/dist"
chown -R root:root "$PREFIX/forms/dist"
chmod -R a+rX "$PREFIX/forms/dist"
fi
log "restarting backend"
systemctl restart warmbly-backend
healthy=0
deadline=$((SECONDS + 120))
while [[ $SECONDS -lt $deadline ]]; do
if curl -fsS --connect-timeout 2 --max-time 5 "$HEALTH" >/dev/null 2>&1; then healthy=1; break; fi
sleep 2
done
if [[ "$healthy" -ne 1 ]]; then
log "the backend did not answer at $HEALTH within two minutes; the other services were not restarted"
exit 1
fi
rest=()
for svc in forms consumer tracking realtime worker; do
has_unit "$svc" && rest+=("warmbly-$svc")
done
if [[ ${#rest[@]} -gt 0 ]]; then
log "restarting ${rest[*]}"
systemctl restart "${rest[@]}"
fi
log "installed"
+28
View File
@@ -0,0 +1,28 @@
[Unit]
Description=Warmbly updater (one-click update from the admin panel)
Documentation=https://docs.warmbly.com/development/updates/
After=network-online.target
Wants=network-online.target
[Service]
Type=simple
# Runs as the user who owns the checkout, so git writes stay theirs. The
# upgrade script builds unprivileged and then runs the fixed-path installer
# (/usr/local/sbin/warmbly-install-release, root-owned) through sudo; that
# installer is the only command to allow in sudoers. See the docs.
User=deploy
Group=deploy
EnvironmentFile=/etc/warmbly/updater.env
WorkingDirectory=/opt/warmbly/src
Environment=UPDATER_MODE=command
Environment=UPDATER_REPO_DIR=/opt/warmbly/src
Environment=UPDATER_COMMAND=/opt/warmbly/src/scripts/upgrade-bare-metal.sh
Environment=UPDATER_BACKEND_HEALTH_URL=http://127.0.0.1:8080/health
Environment=UPDATER_STATE_DIR=/var/lib/warmbly-updater
Environment=UPDATER_ADDR=127.0.0.1:8095
ExecStart=/opt/warmbly/bin/updater
Restart=always
RestartSec=3
[Install]
WantedBy=multi-user.target
+108 -7
View File
@@ -35,6 +35,19 @@
#
# To run on Kafka + S3 instead, set the provider vars above and rebuild the Go
# images with --build-arg GO_TAGS=kafka (tracking: --build-arg CARGO_FEATURES=kafka).
#
# EVERY APP SERVICE CARRIES BOTH image: AND build:, so this one file serves both
# installs. `docker compose pull && docker compose up -d` runs the published
# release images and compiles nothing; `docker compose up --build` still builds
# from this checkout. Pin a release with WARMBLY_TAG=v1.4.2 in .env, or point
# WARMBLY_IMAGE_PREFIX at your own registry.
#
# EVERY STORE'S VOLUME SOURCE IS A VARIABLE. Compose reads a source starting
# with / or . as a bind mount and anything else as a named volume, so
# WARMBLY_PG_DATA=/mnt/ssd/warmbly/postgres puts the database on a path you can
# rsync and snapshot, while the default keeps the named volume. The full set is
# WARMBLY_PG_DATA, WARMBLY_REDIS_DATA, WARMBLY_NATS_DATA, WARMBLY_BLOBS,
# WARMBLY_WORKER_STATE and WARMBLY_UPDATER_STATE.
# Everything below is overridable from a single .env next to this file. The
# defaults are the no-cloud stack, so an empty .env still boots; set only what
@@ -216,7 +229,7 @@ services:
POSTGRES_DB: warmbly_dev
ports: ["15432:5432"]
volumes:
- postgres_data:/var/lib/postgresql/data
- ${WARMBLY_PG_DATA:-postgres_data}:/var/lib/postgresql/data
healthcheck:
test: ["CMD-SHELL", "pg_isready -U warmbly"]
interval: 5s
@@ -228,7 +241,7 @@ services:
image: redis:7-alpine
ports: ["16379:6379"]
volumes:
- redis_data:/data
- ${WARMBLY_REDIS_DATA:-redis_data}:/data
healthcheck:
test: ["CMD", "redis-cli", "ping"]
interval: 5s
@@ -242,7 +255,7 @@ services:
command: ["-js", "-sd", "/data", "-m", "8222"]
ports: ["4222:4222", "8222:8222"]
volumes:
- nats_data:/data
- ${WARMBLY_NATS_DATA:-nats_data}:/data
healthcheck:
test: ["CMD", "wget", "--spider", "-q", "http://localhost:8222/healthz"]
interval: 5s
@@ -295,9 +308,14 @@ services:
backend:
restart: unless-stopped
image: ${WARMBLY_IMAGE_PREFIX:-ghcr.io/warmbly/warmbly}/backend:${WARMBLY_TAG:-prod}
build:
context: .
dockerfile: deploy/docker/backend.Dockerfile
args:
VERSION: ${WARMBLY_BUILD_VERSION:-}
COMMIT: ${WARMBLY_BUILD_COMMIT:-}
BUILT_AT: ${WARMBLY_BUILD_TIME:-}
ports: ["8080:8080"]
environment:
<<: *selfhost-env
@@ -340,8 +358,21 @@ services:
WEBAUTHN_RP_ORIGINS: ${WEBAUTHN_RP_ORIGINS:-}
# Worker image the orchestrator installs on remote machines.
WORKER_IMAGE: ${WORKER_IMAGE:-}
# Update indicator: polls GitHub Releases and shows a newer version in the
# admin panel's top bar. Applying it goes through the updater service
# below. The address is always set because a profile cannot be tested
# here; with the profile off the host does not resolve and the backend
# reports the updater as not running (report-only), never as broken.
# UPDATER_URL=none in .env turns the button off explicitly.
UPDATE_CHECK_ENABLED: ${UPDATE_CHECK_ENABLED:-true}
UPDATE_CHECK_INTERVAL: ${UPDATE_CHECK_INTERVAL:-30m}
UPDATE_CHANNEL: ${UPDATE_CHANNEL:-stable}
RELEASES_GITHUB_REPO: ${RELEASES_GITHUB_REPO:-warmbly/warmbly}
RELEASES_GITHUB_TOKEN: ${RELEASES_GITHUB_TOKEN:-}
UPDATER_URL: ${UPDATER_URL:-http://updater:8095}
UPDATER_TOKEN: ${UPDATER_TOKEN:-${INTERNAL_API_TOKEN:-local-dev-internal-token}}
volumes:
- blobs:/data/blobs
- ${WARMBLY_BLOBS:-blobs}:/data/blobs
depends_on:
postgres: { condition: service_healthy }
redis: { condition: service_healthy }
@@ -357,9 +388,14 @@ services:
# database; the backend's internal API is its only dependency.
forms:
restart: unless-stopped
image: ${WARMBLY_IMAGE_PREFIX:-ghcr.io/warmbly/warmbly}/forms:${WARMBLY_TAG:-prod}
build:
context: .
dockerfile: deploy/docker/forms.Dockerfile
args:
VERSION: ${WARMBLY_BUILD_VERSION:-}
COMMIT: ${WARMBLY_BUILD_COMMIT:-}
BUILT_AT: ${WARMBLY_BUILD_TIME:-}
ports: ["${FORMS_PORT:-8090}:8090"]
environment:
GIN_MODE: release
@@ -383,14 +419,22 @@ services:
consumer:
restart: unless-stopped
image: ${WARMBLY_IMAGE_PREFIX:-ghcr.io/warmbly/warmbly}/consumer:${WARMBLY_TAG:-prod}
build:
context: .
dockerfile: deploy/docker/consumer.Dockerfile
args:
VERSION: ${WARMBLY_BUILD_VERSION:-}
COMMIT: ${WARMBLY_BUILD_COMMIT:-}
BUILT_AT: ${WARMBLY_BUILD_TIME:-}
environment:
<<: *selfhost-env
ENCRYPTED_KEYS_PROVIDER: postgres
# Optional, as on the backend: turns an open or click's network into a
# country and city on the contact's timeline.
GEODB_PATH: ${GEODB_PATH:-/app/data/GeoLite2-City.mmdb}
volumes:
- blobs:/data/blobs
- ${WARMBLY_BLOBS:-blobs}:/data/blobs
depends_on:
backend: { condition: service_healthy }
nats: { condition: service_healthy }
@@ -400,9 +444,14 @@ services:
# belong to the mail provider, so more workers = more parallelism, not more IPs.
worker:
restart: unless-stopped
image: ${WARMBLY_IMAGE_PREFIX:-ghcr.io/warmbly/warmbly}/worker:${WARMBLY_TAG:-prod}
build:
context: .
dockerfile: deploy/docker/worker.Dockerfile
args:
VERSION: ${WARMBLY_BUILD_VERSION:-}
COMMIT: ${WARMBLY_BUILD_COMMIT:-}
BUILT_AT: ${WARMBLY_BUILD_TIME:-}
environment:
<<: *selfhost-env
# Dovecot in the sandbox profile uses a self-signed cert. Set
@@ -423,14 +472,15 @@ services:
BOX_OUTLOOK_CLIENT_ID: ${BOX_OUTLOOK_CLIENT_ID:-}
BOX_OUTLOOK_CLIENT_SECRET: ${BOX_OUTLOOK_CLIENT_SECRET:-}
volumes:
- blobs:/data/blobs
- worker_state:/data/state
- ${WARMBLY_BLOBS:-blobs}:/data/blobs
- ${WARMBLY_WORKER_STATE:-worker_state}:/data/state
depends_on:
backend: { condition: service_healthy }
nats: { condition: service_healthy }
tracking:
restart: unless-stopped
image: ${WARMBLY_IMAGE_PREFIX:-ghcr.io/warmbly/warmbly}/tracking:${WARMBLY_TAG:-prod}
build:
context: ./tracking
dockerfile: Dockerfile
@@ -451,6 +501,7 @@ services:
TRACKING_PAGEHIT_RATE_LIMIT_PER_MIN: ${TRACKING_PAGEHIT_RATE_LIMIT_PER_MIN:-}
TRACKING_TRUSTED_PROXIES: ${TRACKING_TRUSTED_PROXIES:-}
TRACKING_CLIENT_IP_HEADER: ${TRACKING_CLIENT_IP_HEADER:-}
TRACKING_IP_HASH_KEY: ${TRACKING_IP_HASH_KEY:-}
SENTRY_DSN: ${SENTRY_DSN:-}
# Overridable so `make dev` (native backend) can point at
# host.docker.internal:8080 while `docker compose up` uses the container.
@@ -463,6 +514,7 @@ services:
realtime:
restart: unless-stopped
image: ${WARMBLY_IMAGE_PREFIX:-ghcr.io/warmbly/warmbly}/realtime:${WARMBLY_TAG:-prod}
build:
context: .
dockerfile: deploy/docker/realtime.Dockerfile
@@ -497,6 +549,7 @@ services:
# use `make app` / docker-compose.dev.yml instead.
web:
restart: unless-stopped
image: ${WARMBLY_IMAGE_PREFIX:-ghcr.io/warmbly/warmbly}/web:${WARMBLY_TAG:-prod}
build:
context: ./web
dockerfile: Dockerfile
@@ -513,6 +566,7 @@ services:
admin:
restart: unless-stopped
image: ${WARMBLY_IMAGE_PREFIX:-ghcr.io/warmbly/warmbly}/admin:${WARMBLY_TAG:-prod}
build:
context: ./admin
dockerfile: Dockerfile
@@ -525,13 +579,56 @@ services:
depends_on:
backend: { condition: service_healthy }
# One-click updates from the admin panel (the version pill in the top bar).
# Pulls this checkout, rebuilds the images and recreates the containers when
# the backend asks, then waits for the backend to answer again. It holds the
# docker socket, which is root on this host, so it only runs under the
# "updater" profile: `make up` enables it, and COMPOSE_PROFILES=updater in
# .env does the same for a plain `docker compose up`. Leave the profile off
# to update by hand with `git pull && make up`.
updater:
restart: unless-stopped
image: ${WARMBLY_IMAGE_PREFIX:-ghcr.io/warmbly/warmbly}/updater:${WARMBLY_TAG:-prod}
build:
context: .
dockerfile: deploy/docker/updater.Dockerfile
args:
VERSION: ${WARMBLY_BUILD_VERSION:-}
COMMIT: ${WARMBLY_BUILD_COMMIT:-}
BUILT_AT: ${WARMBLY_BUILD_TIME:-}
environment:
UPDATER_TOKEN: ${UPDATER_TOKEN:-${INTERNAL_API_TOKEN:-local-dev-internal-token}}
# The checkout is mounted at its host path so the compose file's relative
# paths resolve identically inside and outside the container.
UPDATER_REPO_DIR: ${WARMBLY_REPO_DIR:-${PWD}}
UPDATER_COMPOSE_PROJECT: warmbly
UPDATER_BACKEND_HEALTH_URL: http://backend:8080/health
UPDATER_FETCH_INTERVAL: ${UPDATER_FETCH_INTERVAL:-30m}
UPDATER_PRUNE: ${UPDATER_PRUNE:-true}
UPDATER_ALLOW_DIRTY: ${UPDATER_ALLOW_DIRTY:-false}
working_dir: ${WARMBLY_REPO_DIR:-${PWD}}
volumes:
- /var/run/docker.sock:/var/run/docker.sock
- ${WARMBLY_REPO_DIR:-${PWD}}:${WARMBLY_REPO_DIR:-${PWD}}
- ${WARMBLY_UPDATER_STATE:-updater_state}:/var/lib/warmbly-updater
profiles: ["updater"]
# ─── one-shots ────────────────────────────────────────────────────────
# Optional demo seed. Run explicitly: docker compose --profile seed run --rm seed
# Build-only on purpose, unlike every service above. The seed is not a
# published image (it plants a known super-admin credential, so it must never
# be one `docker compose pull` away on a real host), and sharing the backend's
# image tag would mean `make seed-demo` retagged the image the running backend
# was created from.
seed:
build:
context: .
dockerfile: deploy/docker/backend.Dockerfile
args:
VERSION: ${WARMBLY_BUILD_VERSION:-}
COMMIT: ${WARMBLY_BUILD_COMMIT:-}
BUILT_AT: ${WARMBLY_BUILD_TIME:-}
entrypoint: ["/app/seed"]
environment:
<<: *selfhost-env
@@ -543,9 +640,13 @@ services:
backend: { condition: service_healthy }
profiles: ["seed"]
# Declared for the defaults. A store pointed at a host path by its
# WARMBLY_*_DATA variable simply stops using the named volume below; an unused
# declaration costs nothing and keeps switching back a one-line .env edit.
volumes:
postgres_data:
redis_data:
nats_data:
blobs:
worker_state:
updater_state:
+22
View File
@@ -17,6 +17,28 @@ The `wmbly_` prefix identifies the key as a Warmbly key. The remaining 43 charac
Keys are stored as a SHA-256 hash; the plaintext is shown exactly once on creation. To help you spot a key in the dashboard without exposing the secret, we store the first 8 characters (`key_prefix`, e.g. `wmbly_ab`) and the last 4 characters (`key_suffix`, e.g. `wxyz`). Render them as `wmbly_ab…wxyz`.
## Three ways to get a key
1. **The dashboard.** Settings > API keys, pick the scopes, copy the secret. This is the right path for a key a server will use.
2. **The CLI.** [`warmbly auth login`](/api/cli/) opens a browser approval and mints a key named for the machine that asked, then stores it at 0600. This is the right path for a key you will use yourself, and it is the only one that does not involve pasting a secret into a shell.
3. **The API.** `POST /v1/api-keys` with a key that carries `API_KEYS`. The secret is in that response and nowhere else.
All three produce the same thing: a `wmbly_` key with a scope bitmask, listed under Settings > API keys, revocable there.
### The CLI device flow
`warmbly auth login` uses a device-code handshake, so the terminal never handles your password and the browser never handles the key:
1. The CLI calls `POST /v1/auth/cli/code` with the scopes it wants and the machine's hostname. It gets back a `device_code` it keeps, a `user_code` it prints, and a `verification_uri_complete` it opens.
2. You approve at `app.warmbly.com/cli`, choosing which workspace the key belongs to. The approval is what mints the key, so it requires the `MANAGE_API_KEYS` organization permission.
3. The CLI polls `POST /v1/auth/cli/poll` and receives the key exactly once, on the first poll after approval.
Codes expire after ten minutes, both halves are per-IP rate limited, and the `device_code` is stored hashed. Anything else can drive the same flow: it is two public endpoints and a browser.
### Ending a key
`DELETE /v1/api-keys/:id` revokes any key in the workspace and needs the `API_KEYS` scope. `DELETE /v1/api-keys/self` revokes the key the call was made with and needs no scope at all, so a narrowly scoped credential can always end itself. This is what `warmbly auth logout` uses.
## Using your API key
Include your API key in the `Authorization` header of every request:
+382
View File
@@ -0,0 +1,382 @@
---
title: CLI
description: The warmbly command line interface. Sign in once, then drive campaigns, contacts, mailboxes and the inbox from your terminal, from CI, or from an agent.
---
`warmbly` is the command line interface to Warmbly. It signs in as you, holds one credential per host, and speaks only the public REST API, so it works against the hosted service and against any self-hosted instance you can reach.
```bash
warmbly auth login
warmbly campaign list
warmbly inbox list --unseen
warmbly api "/campaigns?limit=10"
```
<Callout type="info" title="This is not warmblyctl">
There are two CLIs and they answer different questions. `warmbly` is the one you install on your machine to use the product. [`warmblyctl`](/development/warmblyctl/) is the operator's tool: it talks to Postgres directly, runs inside the backend container, and exists for recovery, accounts, health and backups. If you are asking "what is wrong with this install", that is the one you want.
</Callout>
## Install
Pick one. All of them produce the same single binary.
**macOS and Linux**
```bash
curl -fsSL https://warmbly.com/cli.sh | sh
```
Installs to `~/.local/bin`, so it needs no root and no toolchain. The script verifies what it downloaded against the published checksum and installs nothing if they disagree, writes shell completions, and tells you the one line to add to your profile if that directory is not already on your PATH.
**Windows**
```powershell
irm https://warmbly.com/cli.ps1 | iex
```
Installs to `%LOCALAPPDATA%\Warmbly\bin` and adds it to your user PATH. No admin rights.
**Homebrew** (macOS and Linux)
```bash
brew install warmbly/tap/warmbly
```
**Scoop** (Windows)
```powershell
scoop bucket add warmbly https://github.com/warmbly/homebrew-tap
scoop install warmbly
```
**Docker**, for CI or anywhere installing a binary is more trouble than pulling one:
```bash
docker run --rm -e WARMBLY_TOKEN ghcr.io/warmbly/warmbly/cli campaign list
```
A container has no browser, so sign in with `WARMBLY_TOKEN` rather than `auth login`.
**From source**, if you have Go:
```bash
go install github.com/warmbly/warmbly/cmd/cli@latest
mv "$(go env GOPATH)/bin/cli" "$(go env GOPATH)/bin/warmbly"
```
The package directory is `cmd/cli`, so `go install` names the binary `cli`. Rename it, or use one of the channels above.
**Already have a Warmbly instance?** The binary ships inside the backend image:
```bash
docker compose -p warmbly exec backend warmbly --help
```
### Keeping it current
```bash
warmbly version
warmbly upgrade
```
`upgrade` replaces the binary in place, verifying the download against the release checksums first. When the CLI came from Homebrew or Scoop it says which command to run instead, rather than overwriting a file a package manager owns. The CLI also checks for a new release once a day and mentions it in one line; `WARMBLY_NO_UPDATE_CHECK=1` turns that off, and it never runs in CI or when output is piped.
### Installer flags
The install script takes flags after `--`, and every one of them is also an environment variable, so the same install runs from Ansible, cloud-init or a Dockerfile:
```bash
curl -fsSL https://warmbly.com/cli.sh | sh -s -- --dir /usr/local/bin
curl -fsSL https://warmbly.com/cli.sh | sh -s -- --version v1.4.0
curl -fsSL https://warmbly.com/cli.sh | sh -s -- --no-modify-path --no-completions
curl -fsSL https://warmbly.com/cli.sh | sh -s -- --dry-run
curl -fsSL https://warmbly.com/cli.sh | sh -s -- --uninstall
```
| Flag | Variable | What it does |
|---|---|---|
| `--dir PATH` | `WARMBLY_INSTALL_DIR` | Where the binary goes. Default `~/.local/bin` |
| `--version TAG` | `WARMBLY_CLI_VERSION` | Pin a release instead of taking the newest |
| `--base-url URL` | `WARMBLY_CLI_BASE_URL` | Download from an internal mirror of the release assets |
| `--no-modify-path` | `WARMBLY_NO_MODIFY_PATH` | Never touch a shell profile |
| `--no-completions` | `WARMBLY_NO_COMPLETIONS` | Skip the completion files |
| `--dry-run` | | Print what would happen, change nothing |
| `--uninstall` | | Remove the binary and completions, keep your sign-ins |
`--help` lists them in the terminal. `sh -s -- --help` works through the pipe.
### Reading it before you run it
Piping a script into a shell is worth being careful about, which is why the checksum is published next to it:
```bash
curl -fsSLO https://warmbly.com/cli.sh
curl -fsSLO https://warmbly.com/cli.sh.sha256
sha256sum -c cli.sh.sha256
less cli.sh && sh cli.sh
```
Every release archive is covered by `checksums.txt` on the release, and the installer verifies the archive against it before unpacking. If the two disagree it installs nothing and says so.
### Completions
The installer writes them for your shell. To do it by hand:
```bash
warmbly completion bash > /etc/bash_completion.d/warmbly
warmbly completion zsh > "${fpath[1]}/_warmbly"
warmbly completion fish > ~/.config/fish/completions/warmbly.fish
```
## Signing in
```bash
warmbly auth login
```
It asks two questions: which instance (the hosted service, or a hostname of your own), and how (a browser approval, or pasting a key you already have).
The browser path shows an eight character code, opens `app.warmbly.com/cli`, and waits. You approve there, choosing which workspace the CLI is signing in to. The approval creates **one API key named for your machine**, which appears under Settings > API keys and is revocable there or with `warmbly auth logout`. The terminal never handles your password, and the browser never handles the key.
```
Your code: K4TM-9RQD
Approve at: https://app.warmbly.com/cli?code=K4TM-9RQD
… Waiting for approval (the code expires in 10 minutes)
✓ Signed in to warmbly.com as jane@example.com
Workspace Acme
```
Non-interactive forms, for a script or an agent:
```bash
warmbly auth login --hostname warmbly.acme.com --web
echo "$WARMBLY_KEY" | warmbly auth login --with-token
warmbly auth login --scopes read-only
```
### Scopes
The CLI asks for full access by default, and the approval screen lists exactly what that means before you agree. Narrow it with `--scopes`:
```bash
warmbly auth login --scopes read-only
warmbly auth login --scopes read_campaigns,read_contacts,send_campaigns
```
A key's scopes are fixed once created, so widening them means a new key. `warmbly auth refresh --scopes full` runs the sign-in again and revokes the key it replaces, which is why refreshing does not leave a trail of keys behind.
Scope names are the ones in the [permissions reference](/api/permissions/), in either case, with `full` and `read-only` as shorthands.
### Several instances
The CLI holds one credential per host. The `*` in `auth status` is the one commands use.
```bash
warmbly auth login --hostname warmbly.acme.com
warmbly auth status
warmbly auth switch warmbly.acme.com
warmbly campaign list --host warmbly.com # one command, other host
```
`warmbly auth status` also tells you where the token came from, which is the answer nine times out of ten when a command fails unexpectedly:
```
* warmbly.com
✓ signed in as jane@example.com
- workspace: Acme
- scopes: all 24
- api: https://api.warmbly.com
- token from: /home/jane/.config/warmbly/hosts.yml
- token: wmbly_ab********wxyz
```
### In CI
Set `WARMBLY_TOKEN` and skip the login entirely. It overrides the file and is never written to it.
```yaml
env:
WARMBLY_TOKEN: ${{ secrets.WARMBLY_TOKEN }}
WARMBLY_HOST: warmbly.com # omit for the hosted service
run: warmbly campaign list --json
```
`warmbly auth token` prints the active token and nothing else, for handing to another tool.
## Output
Commands print a table on a terminal and JSON everywhere else, so the same command is readable by a person and parseable by a pipe.
```bash
warmbly campaign list # a table
warmbly campaign list > campaigns.json # JSON, no flag needed
warmbly campaign list --json | jq '.data[].name'
warmbly campaign list --fields name,status
warmbly campaign list --template '{{range .data}}{{.name}}{{"\n"}}{{end}}'
```
`--all` walks the cursor on any list command and merges every page into one response.
## Sending real mail
Anything that puts mail on the wire asks first, and refuses rather than sending when there is no terminal to ask on:
```
$ warmbly campaign start 6f1c…
! `warmbly campaign start` sends real mail. Continue? [y/N]
```
`--yes` is the only way past it, which makes it the flag to grep for in a script review. The commands that behave this way are `campaign start`, `campaign test`, `mailbox send`, `inbox reply`, `inbox compose` and `inbox approve-draft`.
## Commands
Run `warmbly <command> --help` for the flags, and `warmbly <command> <subcommand> --help` for one command's arguments.
| Command | What it covers |
|---|---|
| `auth` | login, logout, status, token, switch, refresh |
| `status` | one screen: mailboxes needing attention, what is sending, what is unread |
| `browse` | open the dashboard, or one record, in a browser |
| `campaign` | list, view, create, edit, steps, senders, segments, preflight, test, start, stop, logs |
| `contact` | list, view, create, edit, delete, lookup, timeline, notes, import, export, verify |
| `mailbox` | list, view, edit, health checks, sync state, sending behaviour, warmup, hold, send |
| `inbox` | list, view, threads, read, reply, compose, drafts, scheduled sends, snoozes |
| `suppression` | the addresses and domains that get no campaign mail |
| `segment` | live audiences and their conditions |
| `template` | reply templates |
| `automation` | automations, their runs and test firing |
| `form` | lead capture forms, submissions and stats |
| `deal`, `pipeline`, `task` | the CRM |
| `analytics` | dashboard, deliverability, warmup, per-mailbox and per-campaign numbers |
| `audit` | the workspace's audit trail |
| `advisor` | recommendations, and applying or dismissing them |
| `webhook` | endpoints, deliveries, redelivery, event types |
| `key` | API keys, their scopes and their usage |
| `oauth-app` | OAuth applications you publish |
| `integration` | third-party connections |
| `org` | which workspace this credential belongs to |
| `team` | named groups of members, for CRM ownership and routing |
| `settings` | workspace-wide outreach and suppression settings |
| `warmup-routing` | which mailboxes warm with which |
| `tool` | the AI tool registry, listed and called |
| `events` | the live event stream |
| `api` | any endpoint at all |
| `upgrade` | replace this binary with the newest release |
| `config`, `alias`, `completion`, `version` | the CLI itself |
### Examples
```bash
# Create a campaign, add a step, check it, start it
warmbly campaign create --name "Q3 outbound" --daily-limit 40
warmbly campaign add-step CAMPAIGN_ID --subject "Quick question" --wait-after 0
warmbly campaign preflight CAMPAIGN_ID
warmbly campaign start CAMPAIGN_ID
# Mailbox health across the workspace
warmbly mailbox list
warmbly mailbox check MAILBOX_ID
warmbly mailbox edit MAILBOX_ID --daily-limit 40
# The inbox
warmbly inbox list --unseen --limit 20
warmbly inbox thread --email-id EMAIL_ID
# Contacts in and out
warmbly contact create --email jane@example.com --first-name Jane --company Acme
warmbly contact list --all --json > contacts.json
```
## What needs the dashboard
Three things the CLI deliberately does not do, because the API does not let a key do them:
- **Connecting a mailbox.** It needs OAuth consent or a credential form in a browser. `warmbly browse mailboxes` opens the right page.
- **Workspace administration.** Members, roles, invitations, workspace exports and the danger zone are session-only on the API: they depend on a human-bound session and refuse an API key. `warmbly browse members` and `warmbly browse settings` open them.
- **Billing.** Plans, checkout and credits are session-only for the same reason. `warmbly browse billing`.
`warmbly org view` still shows which workspace you are in, and `warmbly team`, `warmbly settings` and `warmbly audit` cover the workspace surface a key can reach.
## Watching events
`warmbly events tail` streams the developer WebSocket into your terminal: the same events the dashboard runs on, printed as they happen. It is the fastest way to see whether an integration is receiving what you think it is, without standing up a public endpoint first.
```bash
warmbly events tail
warmbly events tail --intent EMAIL --intent CAMPAIGN
warmbly events tail --json | jq 'select(.event_type == "EMAIL_REPLIED")'
```
It needs a key with `REALTIME_SUBSCRIBE`; `warmbly auth refresh --scopes full` gets one. The stream, its intents and its event types are documented under [Realtime](/api/realtime/).
## Calling the API directly
`warmbly api` reaches every endpoint, including the ones with no command of their own. Paths are relative to `/v1`.
```bash
warmbly api /me
warmbly api "/campaigns?limit=10" --paginate
warmbly api /contacts -f email=jane@example.com -f first_name=Jane
warmbly api /campaigns/CAMPAIGN_ID -X PATCH -F daily_limit=40
warmbly api /contacts/search -X POST --input filter.json
warmbly api /webhooks/WEBHOOK_ID -X DELETE
```
`-f` keeps a value a string. `-F` guesses the type, so `true`, `false`, `null` and numbers arrive as themselves, `@file` reads a value from a file, `key[sub]=v` nests and repeated `key[]=v` builds an array. `--paginate` follows the cursor, `-i` includes the status and headers, `--idempotency-key` rides the [documented header](/api/authentication/).
## Configuration
Two files under `~/.config/warmbly` (or `XDG_CONFIG_HOME`, or `WARMBLY_CONFIG_DIR`):
- `hosts.yml`, one credential per host, written 0600
- `config.yml`, preferences and aliases
Signing in records the instance's own API and dashboard URLs alongside the credential, taken from what the instance reports, so `browse` and `events tail` work on a self-hosted layout without anyone configuring a second address.
```bash
warmbly config list
warmbly config set output json # default to JSON even on a terminal
warmbly config set confirm always # confirm every write, not only sends
warmbly config set browser firefox
```
### Aliases
```bash
warmbly alias set hot "campaign list --status active"
warmbly hot --json
```
Anything you type after the alias is appended, so an alias is a starting point rather than a fixed command.
### Environment
| Variable | What it does |
|---|---|
| `WARMBLY_TOKEN` | The API key to use. Overrides `hosts.yml` and is never written to it |
| `WARMBLY_API_KEY` | The same thing under the name `warmblyctl` uses |
| `WARMBLY_HOST` | Which signed-in host to use |
| `WARMBLY_API_URL` | The API base URL, when it is not derivable from the host |
| `WARMBLY_CONFIG_DIR` | Where the two files live |
| `WARMBLY_NO_UPDATE_CHECK` | Never check for a newer release |
| `NO_COLOR` | Turns colour off, as everywhere else |
## Exit codes
| Code | Meaning |
|---|---|
| `0` | It worked |
| `1` | The command failed, or you declined a prompt |
| `2` | The command line was wrong, or an answer was needed with no terminal to ask on |
| `4` | Not signed in, or the credential was rejected or lacks a scope |
Every API failure prints the response's machine-readable `code` and `request_id` to stderr, so a script can branch without reading prose.
## See also
- [Authentication](/api/authentication/) for how the device flow mints a key
- [Permissions](/api/permissions/) for what each scope allows
- [Endpoint scope map](/api/endpoints/) for what `warmbly api` can reach
- [Realtime](/api/realtime/) for the stream behind `warmbly events tail`
- [warmblyctl](/development/warmblyctl/) for the operator's CLI
+19
View File
@@ -23,6 +23,7 @@ All paths below are relative to the versioned base URL `https://api.warmbly.com/
| GET | `/emails/:id` | `READ_EMAILS` |
| PATCH | `/emails/:id` | `WRITE_EMAILS` |
| PATCH | `/emails/tags` | `WRITE_EMAILS` |
| GET | `/emails/allowance` | `READ_EMAILS` |
| GET | `/emails/:id/track` | `READ_EMAILS` |
| PATCH | `/emails/:id/track` | `WRITE_EMAILS` |
| POST | `/emails/:id/track/verify` | `WRITE_EMAILS` |
@@ -43,11 +44,15 @@ All paths below are relative to the versioned base URL `https://api.warmbly.com/
|--------|------|----------------|
| GET | `/campaigns` | `READ_CAMPAIGNS` |
| GET | `/campaigns-overview` | `READ_CAMPAIGNS` |
| POST | `/campaigns-estimate` | `READ_CAMPAIGNS` |
| POST | `/campaigns` | `WRITE_CAMPAIGNS` |
| GET | `/campaigns/:id` | `READ_CAMPAIGNS` |
| PATCH | `/campaigns/:id` | `WRITE_CAMPAIGNS` |
| DELETE | `/campaigns/:id` | `WRITE_CAMPAIGNS` |
| POST | `/campaigns/:id/duplicate` | `WRITE_CAMPAIGNS` |
| GET | `/campaigns/:id/attachments` | `READ_CAMPAIGNS` |
| POST | `/campaigns/:id/attachments` | `WRITE_CAMPAIGNS` |
| DELETE | `/campaigns/:id/attachments/:attachmentId` | `WRITE_CAMPAIGNS` |
| GET | `/campaigns/:id/segments` | `READ_CAMPAIGNS` |
| PUT | `/campaigns/:id/segments` | `WRITE_CAMPAIGNS` |
| GET | `/campaigns/:id/advanced` | `READ_CAMPAIGNS` |
@@ -201,6 +206,8 @@ The `/unibox/drafts` endpoints hold autosaved compose drafts, scoped to the call
`POST /emails/:id/hold` keeps a mailbox out of campaign sending until `POST /emails/:id/release` puts it back; warmup is unaffected and the automatic rest logic never releases a hold. `release` is also the manual exit for a mailbox that is `resting` automatically. Both are bodyless and idempotent, so they take no `Idempotency-Key`. See [holding a mailbox yourself](/guides/mailboxes/#holding-a-mailbox-yourself).
`GET /emails/allowance` reports how many mailboxes the workspace holds (`used`), how many it may hold (`allowance`, `null` for unlimited), `remaining`, and the `basis` of the number: `fair_use` (the plan's daily sends divided by `sends_per_mailbox`), `plan`, `override` (an approved request), `free`, or `unlimited`. `pending_request` is the open limit-increase request for mailboxes, if any. Every connect path refuses with `mailbox_allowance_reached` once `remaining` is `0`. See [mailbox allowance](/guides/mailboxes/#mailbox-allowance).
`PATCH /emails/:id` accepts `save_to_sent` (boolean) on SMTP/IMAP mailboxes: when true, which is the default, the worker files a copy of each outbound message in the mailbox's Sent folder. It has no effect on Gmail and Outlook mailboxes, whose APIs file their own copy. See [keeping a copy of sent mail](/guides/mailboxes/#keeping-a-copy-of-sent-mail).
`GET /unibox` and `GET /unibox/thread` return message previews: each row carries `snippet`, a one-line summary, not the message body. Read a full message with `GET /unibox/:id`, which returns `body_plain` plus `body_html`. The HTML is sanitized before it leaves the API (scripts, event handlers, embedded frames, and unsafe URL schemes are removed), so it is safe to render, and links carry `target="_blank"` with `rel="noopener"`. `body_truncated` is `true` on the rare message whose stored body could not be read, where `body_plain` falls back to the snippet.
@@ -259,6 +266,9 @@ Changing Advisor settings (`PATCH /advisor/settings`) is JWT only, alongside the
| GET | `/api-keys/:id` | `API_KEYS` |
| PATCH | `/api-keys/:id` | `API_KEYS` |
| DELETE | `/api-keys/:id` | `API_KEYS` |
| DELETE | `/api-keys/self` | none |
`DELETE /api-keys/self` revokes the key the request was made with, and is the one route here that needs no scope. A credential must always be able to end itself: requiring `API_KEYS` to sign out would leave a read-only key on a laptop someone is handing back live, which is what [`warmbly auth logout`](/api/cli/) promises to prevent. A JWT caller gets a `400`: there is no key in that request to end, only a session, which `POST /auth/logout` ends.
### OAuth apps
@@ -283,6 +293,9 @@ Registering and managing the OAuth apps your workspace owns. The flow itself (au
|--------|------|----------------|
| GET/PATCH | `/outreach/settings` | `WRITE_CAMPAIGNS` |
| POST | `/deliverability/events` | `WRITE_CAMPAIGNS` |
| GET | `/suppressions` | `READ_CONTACTS` |
| POST | `/suppressions` | `WRITE_CONTACTS` |
| DELETE | `/suppressions/:id` | `WRITE_CONTACTS` |
| GET | `/tasks/dlq` | `SEND_CAMPAIGNS` |
| POST | `/tasks/dlq/:id/replay` | `SEND_CAMPAIGNS` |
| GET/POST/PATCH/DELETE | `/webhooks[/:id]` | `WEBHOOKS` |
@@ -323,11 +336,14 @@ These never accept an API key. They depend on a human-bound session: billing flo
- `POST /auth/login`, `/auth/login/confirm`, `/auth/register`, `/auth/register/confirm`, `/auth/refresh`, `/auth/reset-password`, `/auth/reset-password/confirm`
- `GET /auth/config` (public deployment capabilities: which sign-in methods this backend has enabled, whether a login code step follows, whether signups are open, whether the instance still needs claiming)
- `GET /auth/instance` (JWT only: the running Warmbly version of a self-hosted instance and whether a newer release exists, for the dashboard's version pill; a hosted deployment answers `self_hosted: false` and nothing else)
`POST /auth/register` accepts an optional `invite` field carrying an invitation token. On a deployment running `DISABLE_REGISTRATION=invite_only` it is what permits the signup, and the account is created inside the inviting organization rather than in a new one. The token must resolve to a live invitation whose email equals the submitted address, otherwise the request is refused with `invitation_invalid`. Omitting it on a closed deployment returns `registration_invite_only` or `registration_closed`. See [error codes](/api/error-codes/#registration-and-invitation-refusals).
`GET /auth/config` gained two fields: `invites_required` (boolean, true when an invitation token is needed to create an account) and `docs_url` (string, the deployment's link to the accounts and access documentation, for a client to surface next to a refusal).
`GET /auth/config` also carries `websocket_url` and `app_url` (both strings, each omitted when the instance has none). They are the realtime gateway a developer client connects to and the dashboard origin a client sends someone to. Both are served here because on a self-hosted instance the host layout is whatever the operator chose, and there is no other way to discover it: the [CLI](/api/cli/) reads them for `warmbly events tail` and `warmbly browse`.
`GET /auth/config` also carries `billing_enabled` (boolean). It is `false` when the deployment runs with `BILLING_PROVIDER=none`, which is the self-host default: every feature is unlocked server-side, so the dashboard shows the workspace as self-hosted instead of on a free trial and hides the billing and referral pages. `self_hosted` alone does not imply this, because a self-hosted install may still run Stripe.
- `POST /auth/setup` (first-run claim: exchanges the one-time token printed at boot for the owner account. Refused once any account exists)
- `GET /auth/providers`, `POST /auth/apple`, `POST /auth/google` (native-app social sign-in)
@@ -337,6 +353,7 @@ These never accept an API key. They depend on a human-bound session: billing flo
- `POST /auth/logout`, `POST /auth/logout-all`, `GET /auth/me`, `PATCH /auth/me/onboarding`
- `POST /auth/me/avatar`, `DELETE /auth/me/avatar`
- `POST /emails/onboarding/oauth/start`, `POST /emails/onboarding/oauth/finish`, `POST /emails/onboarding/smtp-imap`
- `POST /emails/onboarding/smtp-imap/bulk` (up to `50` SMTP/IMAP rows in `accounts`, answered `200` with a per-row `status` of `connected`, `skipped` or `failed` and a `code`; rows past the workspace's [mailbox allowance](/guides/mailboxes/#mailbox-allowance) fail with `mailbox_allowance_reached` before any credential is dialled. Naturally retry-safe: an already connected mailbox is `skipped`, so it takes no `Idempotency-Key`)
- `POST /emails/onboarding/oauth/reauth/:id`, `PUT /emails/onboarding/smtp-imap/:id` (reconnect an existing mailbox after a credential change; JWT permission `MANAGE_EMAILS`)
- `GET /oauth/authorize/details`, `POST /oauth/authorize` (the consent flow: a human approves a third-party app)
- `GET /oauth/authorized-apps`, `DELETE /oauth/authorized-apps/:id` (apps the user has authorized)
@@ -347,6 +364,7 @@ These never accept an API key. They depend on a human-bound session: billing flo
- All of `/organization/*` (create, switch, members, invitations, transfer ownership, avatar, danger zone)
- `GET /website-tracking/settings`, `PATCH /website-tracking/settings`, `POST /website-tracking/settings/rotate-key` (the [website tracking](/guides/website-tracking/) snippet's consent mode, location precision, allowed hosts and retention; JWT permission `MANAGE_SETTINGS`. The rotate is bodyless and safe to repeat, each call issues a new key)
- All of `/subscription/*` (checkout, portal, cancel, change-plan, preview-change, enterprise-inquiry, discounts, referrals, etc.)
- All of `/auth/cli/*` except the two handshake routes below (`GET /auth/cli/codes/:code`, `POST /auth/cli/codes/:code/approve`, `POST /auth/cli/codes/:code/deny`: the browser half of `warmbly auth login`, where a signed-in member reviews the code a CLI is showing and authorizes it. Approving mints an ordinary API key, so it requires the `MANAGE_API_KEYS` organization permission and is session-only: an API key must not be able to mint another one this way)
- All of `/pool-link/*` and `/cloud-link/*` (the self-hosted warmup pool link: approving an instance's code, listing and unlinking instances, and on a self-hosted instance the connect flow and mailbox enrollment). `POST /pool-link/codes` and `POST /pool-link/poll` are public and per-IP rate limited: they are the device-code handshake an instance uses before it has a token, and `/pool-link/instance/*` accepts only an instance token. `/pool-link/instance/oauth/*`, `/pool-link/instance/mailboxes/:id/token`, `/pool-link/instance/workspace-mailboxes` and `/pool-link/instance/mailboxes/adopt` are the cloud-managed mailbox surface (Google and Microsoft sign-in on Warmbly's OAuth apps, brokered access tokens); their instance-side counterparts are `/cloud-link/oauth/*` and `/cloud-link/workspace-mailboxes/*`
- All of `/admin/*`
@@ -425,6 +443,7 @@ External MCP servers whose tools the assistant can use (see [Connect MCP tools](
## Public
- `GET /health`
- `POST /auth/cli/code`, `POST /auth/cli/poll` (the [CLI](/api/cli/) device-code handshake, per-IP rate limited. Public by necessity: the CLI has no credential until the flow completes. `POST /auth/cli/code` returns `device_code`, `user_code`, `verification_uri`, `verification_uri_complete`, `expires_in` and `interval`; polling returns `{"status":"pending"}` until a member decides, then `{"status":"approved"}` carrying the minted key exactly once, or `{"status":"denied"}`. An unknown or expired `device_code` is a `404`, so a poller cannot probe for live handshakes)
- `POST /webhooks/github/releases` (HMAC-SHA256 signature)
- `POST /webhook/stripe` (Stripe signature)
- `POST /webhook/campaign`, `/webhook/email`, `/webhook/user-email` (Google OIDC token from Cloud Tasks)
+37 -2
View File
@@ -26,10 +26,10 @@ All errors follow this structure:
| Code | Error | Description |
|------|-------|-------------|
| 400 | Bad Request | Invalid request syntax or parameters |
| 400 | Bad Request | Invalid request syntax or parameters, or a quota that would be passed (`storage_limit_reached`) |
| 401 | Unauthorized | Missing or invalid authentication |
| 402 | Payment Required | Out of AI credits (`insufficient_credits`) |
| 403 | Forbidden | Authenticated but lacks permission |
| 403 | Forbidden | Authenticated but lacks permission, or the workspace's mailbox allowance is full (`mailbox_allowance_reached`) |
| 404 | Not Found | Resource doesn't exist |
| 409 | Conflict | Resource already exists |
| 422 | Unprocessable | Validation failed |
@@ -317,6 +317,41 @@ A `503` whose `code` is `mailbox_provider_not_configured` is not transient and r
- Or connect the mailbox over SMTP and IMAP instead, which needs no configuration
- Full walkthrough: [connect mailboxes](/development/deployment-guide/#connect-mailboxes)
#### `mailbox_allowance_reached`
A `403` whose `code` is `mailbox_allowance_reached` comes from every path that connects a mailbox: `POST /emails/onboarding/oauth/start`, `POST /emails/onboarding/oauth/finish`, `POST /emails/onboarding/smtp-imap`, and per row inside `POST /emails/onboarding/smtp-imap/bulk`. It is not a permission problem: the workspace holds its whole [mailbox allowance](/guides/mailboxes/#mailbox-allowance), which on a paid plan is one mailbox for every send a day the plan includes, and `10` on a free workspace. Nothing was connected.
```json
{
"error": "Forbidden",
"message": "This workspace holds 15000 of its 15000 mailboxes. Request an increase, or move to a plan with more daily sends.",
"code": "mailbox_allowance_reached",
"request_id": "4bbbd1b2-8f86-47dd-8a7f-9476501ad20e"
}
```
**How to fix:**
- Read `GET /emails/allowance` first: `remaining` says how many connects will succeed, and `pending_request` whether an increase is already asked for
- Submit a limit-increase request for `max_email_accounts` via `POST /organization/:orgId/limit-requests`, or move to a plan with more daily sends. An approved request raises the allowance immediately; retry the connect then
- Reconnecting an existing mailbox never returns this code
#### `storage_limit_reached`
A `400` whose `code` is `storage_limit_reached` comes from `POST /campaigns/:id/attachments` and from a campaign duplicate that would copy attachments. The workspace's attachment storage, summed across every campaign, would pass its quota. The check and the write happen together under a per-workspace lock, so two uploads racing for the last of the quota cannot both get in. Nothing was stored.
```json
{
"error": "Bad Request",
"message": "Storage limit reached: 51190 MB of 51200 MB used, 12 MB to add. Remove attachments or upgrade your plan.",
"code": "storage_limit_reached",
"request_id": "4bbbd1b2-8f86-47dd-8a7f-9476501ad20e"
}
```
**How to fix:**
- `GET /organization/current/limits` reports `storage.used_bytes` and `storage.limit_bytes`, and `storage.over_quota` when a plan change left the workspace above the quota. Existing attachments keep sending either way
- Delete attachments you no longer need (`DELETE /campaigns/:id/attachments/:attachmentId`), or move to a paid plan for the larger quota
#### `mailbox_worker_unreachable`
A `503` whose `code` is `mailbox_worker_unreachable` comes from `DELETE /emails/{id}`. Disconnecting a mailbox has to reach the machine that syncs it before the record goes, because once the record is gone nothing can tell that machine to stop. When the instruction cannot be delivered, nothing is removed and the mailbox is left exactly as it was.
+1
View File
@@ -6,6 +6,7 @@
"index",
"authentication",
"sdks",
"cli",
"oauth",
"permissions",
"endpoints",
+1 -1
View File
@@ -41,7 +41,7 @@ After connecting, join one or more topics with a `phx_join` message:
| `account:<account_id>` | One mailbox's sync and warmup events | Requires `manage_emails` |
| `bulk:<operation_id>` | Progress of one bulk operation | Import/export progress; only the user who started the operation receives its events |
Events arrive as channel messages whose event name is the event type, for example `EMAIL_SENT`, `EMAIL_OPENED`, `EMAIL_REPLIED`, `EMAIL_RECEIVED`, `AI_DRAFT_READY`, `CAMPAIGN_COMPLETED`, `TASK_PROGRESS`, `ACCOUNT_HEALTH_CHANGED`, `ACCOUNT_SYNC_STATE`, `AUDIT_CREATED`, `AUTOMATION_RUN`, `MEETING_BOOKED`, `NOTIFICATION_CREATED`, `PAGE_HIT`. Payloads always include `event_type` and a timestamp, plus the relevant ids (`campaign_id`, `contact_id`, `thread_id`, and so on).
Events arrive as channel messages whose event name is the event type, for example `EMAIL_SENT`, `EMAIL_OPENED`, `EMAIL_REPLIED`, `EMAIL_RECEIVED`, `AI_DRAFT_READY`, `CAMPAIGN_COMPLETED`, `CAMPAIGN_IDLE` (a continuous campaign ran out of leads and is waiting), `TASK_PROGRESS`, `ACCOUNT_HEALTH_CHANGED`, `ACCOUNT_SYNC_STATE`, `AUDIT_CREATED`, `AUTOMATION_RUN`, `MEETING_BOOKED`, `NOTIFICATION_CREATED`, `PAGE_HIT`. Payloads always include `event_type` and a timestamp, plus the relevant ids (`campaign_id`, `contact_id`, `thread_id`, and so on). `EMAIL_OPENED` and `EMAIL_CLICKED` also carry `occurred_at` (when the tracking service saw it; `timestamp` is the publish time), `machine` (true for a mail client prefetch or a security scanner's fetch), and `client`, `device_type`, `country_code` and `city` when the consumer could tell.
`AI_DRAFT_READY` fires when the [inbox agent](/guides/inbox-agent/) drafts a suggested reply awaiting review; it carries `thread_id` and `draft_id` and requires `access_unibox`.
@@ -470,7 +470,7 @@ Auth: Session only (not available to API keys).
`DELETE /auth/me/avatar`
Clears the profile image. Returns `204 No Content`.
Clears the profile image and deletes the stored file on a best-effort basis. Returns `204 No Content`.
Auth: Session only (not available to API keys).
@@ -1456,7 +1456,7 @@ Auth: Session only (not available to API keys). Requires a selected organization
`DELETE /organization/avatar`
Clears the workspace image. Owner only. Returns `204 No Content`.
Clears the workspace image and deletes the stored file on a best-effort basis. Owner only. Returns `204 No Content`.
Auth: Session only (not available to API keys). Requires a selected organization (owner only).
+11 -2
View File
@@ -29,6 +29,7 @@ Returns the main dashboard overview for the active organization: aggregate stats
"total_opens": 612,
"machine_opens": 88,
"total_clicks": 143,
"machine_clicks": 6,
"total_replies": 57,
"total_bounces": 9,
"open_rate": 49.35,
@@ -239,6 +240,7 @@ Returns a single campaign's performance summary plus per-sequence-step stats. Th
"unique_opens": 410,
"machine_opens": 52,
"unique_clicks": 99,
"machine_clicks": 4,
"replies": 41,
"bounces": 5,
"unsubscribes": 3,
@@ -258,11 +260,18 @@ Returns a single campaign's performance summary plus per-sequence-step stats. Th
"replies": 28,
"bounces": 3
}
]
],
"engagement": {
"countries": [{ "key": "US", "opens": 120, "clicks": 31 }, { "key": "DE", "opens": 44, "clicks": 9 }],
"clients": [{ "key": "Gmail", "opens": 98, "clicks": 20 }, { "key": "Outlook", "opens": 51, "clicks": 12 }],
"devices": [{ "key": "desktop", "opens": 140, "clicks": 37 }, { "key": "mobile", "opens": 24, "clicks": 3 }]
}
}
```
`machine_opens` is the subset of `unique_opens` from automated fetchers (Apple MPP prefetch, UA-less clients); human opens are `unique_opens` minus `machine_opens`.
`engagement` groups the campaign's human opens and clicks by country (ISO code), mail client or browser, and device type (`desktop`, `mobile`, `tablet`), counting distinct contacts per bucket; a click counts as an open. Each list holds the busiest eight; an empty `key` is unknown. Country needs the GeoLite2 database on the consumer, otherwise every country row is unknown.
`machine_opens` is the subset of `unique_opens` from automated fetchers (Apple MPP prefetch, UA-less clients, opens inside ten seconds of the send); human opens are `unique_opens` minus `machine_opens`. `machine_clicks` counts steps whose only clicks were automated (a security gateway walking the links); those are not part of `unique_clicks` or `total_clicks`, which only ever count a person's click.
## Get campaign daily stats
+109 -22
View File
@@ -18,6 +18,7 @@ Search and page through the organization's campaigns. **Scope** `READ_CAMPAIGNS`
| `q` | query | string | Free-text filter on campaign name. Optional. |
| `folder` | query | string | Restrict to a single folder id. Optional. |
| `status` | query | string | Status bucket filter: `draft`, `active`, `paused` (matches every paused variant), or `completed`. Any other value returns `400`. Optional. |
| `kind` | query | string | `sequence` or `one_time`. Any other value returns `400`. Optional. |
| `cursor` | query | string | Opaque cursor from the previous page's `pagination.next_cursor`. Optional. |
| `limit` | query | string | Page size. Optional. |
@@ -35,6 +36,7 @@ A `data` plus `pagination` envelope. `next_cursor` is the campaign id to resume
"name": "Q3 outbound",
"description": "",
"status": "active",
"kind": "sequence",
"stop_on_reply": true,
"open_tracking": true,
"link_tracking": true,
@@ -65,8 +67,14 @@ A `data` plus `pagination` envelope. `next_cursor` is the campaign id to resume
"esp_match_mode": "off",
"max_new_leads_per_day": 0,
"prioritize_new_leads": false,
"continuous": false,
"idle_since": null,
"tracking_domain": "",
"tracking_domain_verified": false,
"utm_tracking": true,
"utm_source": "",
"utm_medium": "",
"utm_campaign": "",
"updated_at": "2026-06-10T12:00:00Z",
"created_at": "2026-06-01T09:00:00Z"
}
@@ -83,7 +91,7 @@ A `data` plus `pagination` envelope. `next_cursor` is the campaign id to resume
`GET /campaigns-overview`
Status-bucket counts plus per-folder totals for the organization, used to drive campaign browsing UIs. `paused` sums every paused variant (`paused`, `paused_no_accounts`, `paused_trial_expired`). The path has no campaign id, so it lives beside `/campaigns` rather than under it. **Scope** `READ_CAMPAIGNS` · **Org permission** `view_campaigns`.
Status-bucket counts plus per-folder totals for the organization, used to drive campaign browsing UIs. `paused` sums every paused variant (`paused`, `paused_no_accounts`, `paused_trial_expired`); `one_time` counts campaigns of that kind whatever their status. The path has no campaign id, so it lives beside `/campaigns` rather than under it. **Scope** `READ_CAMPAIGNS` · **Org permission** `view_campaigns`.
### Response
@@ -94,12 +102,45 @@ Status-bucket counts plus per-folder totals for the organization, used to drive
"paused": 2,
"draft": 4,
"completed": 3,
"one_time": 2,
"folders": [
{ "folder_id": "6b9c1c8e-1f2a-4d3b-8c7e-9a0b1c2d3e4f", "total": 5 }
]
}
```
## Estimate a send
`POST /campaigns-estimate`
Project an audience against a sender pool before a campaign exists: how many contacts the segments resolve to, how many mailboxes would send, the pool's daily ceiling under the campaign limit, and the day the last send is expected to land. Nothing is written, so it needs no `Idempotency-Key`. The dashboard's one-time email wizard shows this on its last step. **Scope** `READ_CAMPAIGNS` · **Org permission** `view_campaigns`.
### Request body
| Field | Type | Required | Description |
| --- | --- | --- | --- |
| `segment_ids` | string[] | yes | Segments making up the audience (at most 20). A contact in several of them is counted once. |
| `email_tag_ids` | string[] | no | Mailbox tags that resolve the pool. Omit or send `[]` for every active mailbox in the workspace. |
| `daily_limit` | integer | no | Per-mailbox campaign cap to apply (defaults to `50`). Each mailbox counts the smaller of this and its own cap. |
| `days` | integer (0-127) | no | Weekday bitmask of sending days, bit 0 = Monday. Defaults to weekdays. |
| `timezone` | string | no | IANA timezone the days are counted in. Defaults to UTC. |
| `start_date` | string (RFC 3339) | no | When sending begins. Omit for now. |
### Response
`sending_days` and `estimated_finish_at` are `null` when the audience is empty, the pool has no capacity, or the send would take longer than two years. Today only contributes what the pool has not already sent (`remaining_today`).
```json
{
"recipients": 1000,
"mailboxes": 4,
"daily_capacity": 200,
"remaining_today": 140,
"sending_days": 5,
"estimated_finish_at": "2026-09-09T00:00:00+02:00"
}
```
## Create a campaign
`POST /campaigns`
@@ -112,10 +153,15 @@ Create a campaign. Only `name` is required, every other field is optional and ap
| --- | --- | --- | --- |
| `name` | string | yes | Campaign name. |
| `description` | string | no | Free-text description. |
| `kind` | string | no | `sequence` (default) or `one_time`. A one-time email accepts at most one entry in `steps` here and refuses further email steps later; it is otherwise a normal campaign. Fixed at creation. |
| `stop_on_reply` | boolean | no | Stop sending to a contact once they reply. |
| `open_tracking` | boolean | no | Insert the open pixel. |
| `link_tracking` | boolean | no | Rewrite links through the tracking ticket service. |
| `text_only` | boolean | no | Send plain text only. |
| `link_tracking` | boolean | no | Rewrite links through the tracking ticket service. Each link gets its own ticket, so clicks are attributed per link. |
| `utm_tracking` | boolean | no | Tag every link with `utm_source`, `utm_medium`, `utm_campaign` and a per-link `utm_content` at send time. Default `false`. Values already on a link are kept. |
| `utm_source` | string | no | Overrides the default `warmbly`. Empty means the default. Up to 128 characters. |
| `utm_medium` | string | no | Overrides the default `email`. Empty means the default. |
| `utm_campaign` | string | no | Overrides the default, the campaign name as a slug. Empty means the default. |
| `text_only` | boolean | no | Send plain text only: no HTML part, and open and click tracking are off regardless of their flags. |
| `daily_limit` | integer | no | Per-campaign daily send cap. |
| `unsubscribe_header` | boolean | no | Add the RFC 8058 one-click unsubscribe header. |
| `risky_emails` | boolean | no | Allow sending to risky/unverified addresses. |
@@ -127,6 +173,7 @@ Create a campaign. Only `name` is required, every other field is optional and ap
| `days` | integer (0-127) | no | Legacy weekday bitmask (superseded by `schedule_windows`). |
| `start_time` | string | no | Legacy daily start (`HH:MM`). |
| `end_time` | string | no | Legacy daily end (`HH:MM`). |
| `schedule_windows` | array | no | Per-day sending windows, 7 arrays indexed by weekday (Sunday = 0) of `{start, end}` minute-of-day intervals. When non-empty it supersedes `days`, `start_time` and `end_time`. |
| `email_tag_ids` | string[] | no | Mailbox tag ids that resolve the sender pool (tags strategy). |
| `folder_ids` | string[] | no | Folder ids to file the campaign under. |
| `sender_strategy` | string | no | `tags` (default) or `explicit`. |
@@ -139,6 +186,7 @@ Create a campaign. Only `name` is required, every other field is optional and ap
| `esp_match_mode` | string | no | `off`, `prefer`, or `strict`. |
| `max_new_leads_per_day` | integer | no | New-lead throttle, `0` is unlimited. |
| `prioritize_new_leads` | boolean | no | Prefer new leads in each send window. |
| `continuous` | boolean | no | Keep running for new leads: out of leads, the campaign stays `active` and waits instead of finishing. Linking a segment turns it on. Default `false`. |
| `tracking_domain` | string | no | Campaign-scoped tracking domain (honored only once verified). |
| `steps` | object[] | no | Initial sequence steps in order (see create sequence input below). They are connected in order: each step routes unconditionally to the next, waiting that step's `wait_after` days. The first step's `wait_after` defaults to `0`, follow-ups to `3`. |
| `variants` | object[] | no | A/B variants for the first step (same shape as create A/B variant). |
@@ -200,7 +248,9 @@ Patch any subset of campaign fields. Omitted fields are left unchanged. The expl
### Request body
Every field is optional. Scalar fields use nullable pointers, so any field you send is applied. `start_date` and `end_date` additionally accept an explicit `null` to clear the stored date: a null `start_date` means "start now" and a null `end_date` means "run open-ended". Changing any schedule field (`start_date`, `end_date`, `timezone`, `days`, `start_time`, `end_time`, `schedule_windows`) on an active campaign reschedules its next send immediately, so clearing a future start date takes effect right away. Notable fields: `name`, `description`, `status`, `stop_on_reply`, `open_tracking`, `link_tracking`, `text_only`, `daily_limit`, `unsubscribe_header`, `risky_emails`, `cc`, `bcc`, `start_date`, `end_date`, `timezone`, `days`, `start_time`, `end_time`, `schedule_windows`, `email_tags`, `folders`, `contact_order_by`, `contact_order_dir`, `contact_order_field`, `sender_strategy`, `rotation_mode`, `ramp_enabled`, `ramp_start`, `ramp_increment`, `ramp_ceiling`, `esp_match_mode`, `max_new_leads_per_day`, `prioritize_new_leads`, `tracking_domain`.
Every field is optional. Scalar fields use nullable pointers, so any field you send is applied. `start_date` and `end_date` additionally accept an explicit `null` to clear the stored date: a null `start_date` means "start now" and a null `end_date` means "run open-ended". Changing any schedule field (`start_date`, `end_date`, `timezone`, `days`, `start_time`, `end_time`, `schedule_windows`) on an active campaign reschedules its next send immediately, so clearing a future start date takes effect right away. Notable fields: `name`, `description`, `status`, `stop_on_reply`, `open_tracking`, `link_tracking`, `text_only`, `daily_limit`, `unsubscribe_header`, `risky_emails`, `cc`, `bcc`, `start_date`, `end_date`, `timezone`, `days`, `start_time`, `end_time`, `schedule_windows`, `email_tags`, `folders`, `contact_order_by`, `contact_order_dir`, `contact_order_field`, `sender_strategy`, `rotation_mode`, `ramp_enabled`, `ramp_start`, `ramp_increment`, `ramp_ceiling`, `esp_match_mode`, `max_new_leads_per_day`, `prioritize_new_leads`, `continuous`, `tracking_domain`, `utm_tracking`, `utm_source`, `utm_medium`, `utm_campaign`.
A campaign with `continuous` set stays `active` when it runs out of leads and carries `idle_since` while it waits; the timestamp clears once it has something to send. A continuous campaign can be started with no leads at all. Adding a lead to a `completed` campaign by any path (this API, a linked segment, an automation) restarts it through the same launch checks as starting it by hand; a refused restart is written to the campaign's activity log.
```json
{
@@ -519,7 +569,7 @@ An `ABWinnerAnalysis` object.
`GET /campaigns/:id/attachments`
List the campaign's attachments. Each entry carries a short-lived presigned download `url`. **Scope** `READ_CAMPAIGNS` · **Org permission** `view_campaigns`.
List every attachment of the campaign. Each entry carries a short-lived presigned download `url` and its `step_id`: a file scoped to a step is sent by that step alone, and a `null` `step_id` means every step of the campaign sends it. **Scope** `READ_CAMPAIGNS` · **Org permission** `view_campaigns`.
| Parameter | In | Type | Description |
| --- | --- | --- | --- |
@@ -550,13 +600,13 @@ A `data` array of attachment objects (no pagination wrapper).
`POST /campaigns/:id/attachments`
Upload a file to attach to the campaign (or one step). Sent as `multipart/form-data`, not JSON. **Scope** `WRITE_CAMPAIGNS` · **Org permission** `manage_campaigns`.
Upload a file to attach to the campaign, or to one of its steps. Sent as `multipart/form-data`, not JSON. **Scope** `WRITE_CAMPAIGNS` · **Org permission** `manage_campaigns`.
| Parameter | In | Type | Description |
| --- | --- | --- | --- |
| `id` | path | uuid | Campaign id. |
| `file` | form (multipart) | file | Required. The file to upload (max 15 MB). Executable and script types are rejected. |
| `step_id` | form (multipart) | uuid | Optional. Scope the attachment to one sequence step. |
| `step_id` | form (multipart) | uuid | Optional. Scope the attachment to one sequence step of this campaign, which is then the only step that sends it. Omit it to attach the file to every step. A step of another campaign returns `404`. |
### Response
@@ -616,7 +666,9 @@ A `PreflightReport` object.
`POST /campaigns/:id/test-email`
Send a one-off preview of a sequence step to a chosen recipient through a chosen mailbox. Defaults to the first step when `step_id` is omitted. **Scope** `SEND_CAMPAIGNS` · **Org permission** `send_campaigns`.
Send a one-off test of a sequence step to a chosen recipient through a chosen mailbox. The message is assembled the way a real send is: merge fields and spintax resolve for the contact, the files that step sends are attached (the campaign's unscoped attachments plus that step's own), the mailbox signature and the campaign's opt-out footer are appended, and a plain-text campaign ships without an HTML part. The subject is prefixed with `[TEST]`. Opens and clicks on a test are never tracked, and its opt-out link names no contact, so clicking it suppresses nobody. Defaults to the first step when `step_id` is omitted. **Scope** `SEND_CAMPAIGNS` · **Org permission** `send_campaigns`.
The mailbox may be any mailbox of the organization, not only one the caller connected, and it does not have to be in the campaign's sender pool. An API key with an `allowed_email_accounts` list can only test from those mailboxes. Passing `contact_id` additionally requires `READ_CONTACTS` (org permission `view_contacts`), since the rendered copy reads that contact's fields.
| Parameter | In | Type | Description |
| --- | --- | --- | --- |
@@ -629,31 +681,45 @@ Send a one-off preview of a sequence step to a chosen recipient through a chosen
| `account_id` | uuid | yes | Sending mailbox id. |
| `recipient` | string (email) | yes | Where to send the test. |
| `step_id` | uuid | no | Step to render and send, defaults to the first step. |
| `contact_id` | uuid | no | Contact of the organization to render the copy for, including its custom fields. Omitted renders for a placeholder contact (`Test Recipient` at `Test Company`, with the recipient's address and no custom fields). |
```json
{
"account_id": "5c7d...",
"recipient": "me@example.com",
"step_id": "7e3b..."
"step_id": "7e3b...",
"contact_id": "9a2c..."
}
```
### Response
`contact_id` is present only when one was given.
```json
{
"message": "test email sent",
"recipient": "me@example.com",
"subject": "Quick question, Alex",
"account_id": "5c7d..."
"subject": "Quick question, {{.FirstName}}",
"account_id": "5c7d...",
"step_id": "7e3b...",
"contact_id": "9a2c..."
}
```
### Errors
| Status | Code | When |
| --- | --- | --- |
| `404` | `not_found` | The campaign, step, mailbox or contact does not belong to the caller's organization. |
| `403` | `forbidden` | `contact_id` given without contact read permission, or `account_id` outside the API key's allowed mailboxes. |
| `400` | `bad_request` | The campaign has no steps. |
## Start a campaign
`POST /campaigns/:id/start`
Start (activate) the campaign so it begins sending real mail. Works from `draft`, any paused status, or `completed` (a campaign closed by a passed end date resumes once the date is extended or cleared; one with nothing left to send re-completes with a `400` explaining why). **Scope** `SEND_CAMPAIGNS` · **Org permission** `send_campaigns`.
Start (activate) the campaign so it begins sending real mail. Works from `draft`, any paused status, or `completed` (a campaign closed by a passed end date resumes once the date is extended or cleared; one with nothing left to send re-completes with a `400` explaining why, unless it is `continuous`, in which case it starts and waits for leads with `idle_since` set). **Scope** `SEND_CAMPAIGNS` · **Org permission** `send_campaigns`.
| Parameter | In | Type | Description |
| --- | --- | --- | --- |
@@ -787,7 +853,7 @@ Return the segments linked to the campaign as live audience sources. **Scope** `
### Response
A `data` array of link objects (no pagination wrapper).
A `data` array of link objects (no pagination wrapper). The counts are evaluated when you ask: `contact_count` is how many contacts the segment matches now, `lead_count` how many of them are leads of this campaign, and `held_out_count` how many are not leads because they were removed from the campaign by hand (see [replace linked segments](#replace-linked-segments)).
```json
{
@@ -798,6 +864,8 @@ A `data` array of link objects (no pagination wrapper).
"color": "#0ea5e9",
"description": "Replied or clicked in the last 30 days",
"contact_count": 412,
"lead_count": 409,
"held_out_count": 3,
"linked_at": "2026-06-10T12:00:00Z"
}
]
@@ -828,7 +896,7 @@ Atomically replace the campaign's linked segments with the supplied set (up to 2
### Response
The resulting links plus how many leads the call enrolled.
The resulting links plus how many leads the call enrolled. `added` is `0` when every member was already a lead, when the segments match no contacts yet, or when the only members are held out; the per-link counts tell these apart. The links and the enrolment are written in one transaction, so a failed enrolment returns an error and changes nothing rather than `200` with `added: 0`.
```json
{
@@ -839,6 +907,8 @@ The resulting links plus how many leads the call enrolled.
"color": "#0ea5e9",
"description": "Replied or clicked in the last 30 days",
"contact_count": 412,
"lead_count": 412,
"held_out_count": 0,
"linked_at": "2026-06-10T12:00:00Z"
}
],
@@ -993,7 +1063,7 @@ Delete a sequence step. **Scope** `WRITE_CAMPAIGNS` · **Org permission** `manag
`POST /campaign-template-preview`
Render subject and body templates against a sample (or supplied) contact exactly as the send path would, and report parse errors plus any unresolved `{{...}}` tokens. No side effects, no campaign id. **Scope** `READ_CAMPAIGNS` · **Org permission** `view_campaigns`.
Render subject and body templates for a contact exactly as the send path would, and report parse errors plus any unresolved `{{...}}` tokens. With `campaign_id` and `account_id` the preview also goes through the rest of the send assembly: the plain-text rule, the mailbox signature and the campaign's opt-out footer are applied in send order, and the response names the sender and lists the attachments the send carries. Tracking pixels and link rewriting are left out. No side effects. **Scope** `READ_CAMPAIGNS` · **Org permission** `view_campaigns`. Passing `contact_id` additionally requires `READ_CONTACTS` (org permission `view_contacts`), and an API key with an `allowed_email_accounts` list can only name those mailboxes in `account_id`.
### Request body
@@ -1002,29 +1072,46 @@ Render subject and body templates against a sample (or supplied) contact exactly
| `subject` | string | no | Subject template. |
| `body_html` | string | no | HTML body template. |
| `body_plain` | string | no | Plain-text body template. |
| `contact` | object | no | Override fields on the built-in sample contact: `first_name`, `last_name`, `email`, `company`, `phone`, and a `custom_fields` map of string to string. |
| `contact_id` | uuid | no | A contact of the organization to render for, with its custom fields. Omitted uses the built-in sample contact. |
| `contact` | object | no | Override fields on the contact being rendered for (the sample or the one from `contact_id`): `first_name`, `last_name`, `email`, `company`, `phone`, and a `custom_fields` map of string to string. |
| `campaign_id` | uuid | no | Campaign of the organization whose opt-out footer, plain-text setting and attachments apply. The opt-out link in the preview names no contact. |
| `step_id` | uuid | no | The step being previewed, so `attachments` lists what that step sends. Without it only the campaign-wide files are listed. |
| `account_id` | uuid | no | Mailbox of the organization whose signature is appended (when signature sync is on) and which is reported as `from`. |
```json
{
"subject": "Hi {{first_name}} at {{company}}",
"body_plain": "Hey {{first_name}}, I saw {{company}} is hiring. {{unknown_token}}",
"contact": { "first_name": "Sam", "company": "Globex" }
"body_html": "<p>Hey {{first_name}}, I saw {{company}} is hiring. {{unknown_token}}</p>",
"contact_id": "9a2c...",
"campaign_id": "8f1d...",
"account_id": "5c7d..."
}
```
### Response
A `TemplatePreview` object. `errors` lists template parse errors that would block sending, `unresolved` lists literal tokens left after render. Both are omitted when empty.
A `TemplatePreview` object. `errors` lists template parse errors that would block sending, `unresolved` lists literal tokens left after render. `from` is present when `account_id` was given, `attachments` when `campaign_id` was given and there are files on the send (the campaign-wide ones plus `step_id`'s own). All four are omitted when empty.
```json
{
"subject": "Hi Sam at Globex",
"body_html": "",
"body_plain": "Hey Sam, I saw Globex is hiring. {{unknown_token}}",
"unresolved": ["{{unknown_token}}"]
"body_html": "<p>Hey Sam, I saw Globex is hiring. {{unknown_token}}</p><br><br><p>Best, Ana</p><p style=\"font-size:12px;color:#64748b\">Don't want these emails? <a href=\"https://app.example.com/u/...\">Unsubscribe</a></p>",
"body_plain": "Hey Sam, I saw Globex is hiring. {{unknown_token}}\n\nBest, Ana\n\nDon't want these emails? https://app.example.com/u/...",
"unresolved": ["{{unknown_token}}"],
"from": { "name": "Ana Silva", "email": "ana@globex.com" },
"attachments": [
{ "id": "3b0e...", "filename": "deck.pdf", "size": 482113, "mime_type": "application/pdf" }
]
}
```
### Errors
| Status | Code | When |
| --- | --- | --- |
| `404` | `not_found` | `contact_id`, `campaign_id` or `account_id` does not belong to the caller's organization. |
| `403` | `forbidden` | `contact_id` given without contact read permission, or `account_id` outside the API key's allowed mailboxes. |
## Generate copy with the writing assistant
`POST /generation/write`
+38 -8
View File
@@ -537,7 +537,9 @@ Auth: **Scope** `READ_CONTACTS` · **Org permission** `view_contacts`
}
```
When the contact is suppressed, `suppression` is an object: `{ "reason", "source", "expires_at", "created_at" }` where `source` is `bounce`, `complaint`, or `unsubscribe`.
`engagement.total_opened` and `engagement.last_opened_at` count opens by a person; opens a mail client or security gateway fetched automatically are left out, as they are in campaign analytics. A person's click counts as an open too; an automated click does not add to either field.
When the contact is suppressed, `suppression` is an object: `{ "id", "kind", "value", "reason", "source", "expires_at", "created_at" }`. `kind` is `email` when the contact's own address is on the list or `domain` when its whole domain is, `value` is the matching entry, and `source` is `bounce`, `complaint`, `unsubscribe`, `manual`, or `import`. `id` is the suppression entry, which `DELETE /suppressions/:id` lifts; see [deliverability and ops](/api/reference/deliverability-ops/#list-the-suppression-list).
## Update a contact
@@ -662,23 +664,46 @@ Returns a `data` array plus a `pagination` envelope.
`GET /contacts/:id/timeline`
Returns the merged activity feed for a contact: sends, opens, clicks, replies, bounces, deliverability and suppression events, notes, meeting bookings, and lifecycle events (the contact's creation with its first-touch source, and every time it joined or left a campaign or a category). Requires a selected organization (org-scoped events would otherwise be hidden), so a request with no organization returns `400`.
Returns the merged activity feed for a contact: sends, opens, clicks (one per link, naming the link), replies, bounces, deliverability and suppression events, notes, meeting bookings, and lifecycle events (the contact's creation with its first-touch source, and every time it joined or left a campaign or a category). Requires a selected organization (org-scoped events would otherwise be hidden), so a request with no organization returns `400`.
Auth: **Scope** `READ_CONTACTS` · **Org permission** `view_contacts`
| Parameter | In | Type | Description |
| --- | --- | --- | --- |
| `id` | path | UUID | Contact ID. |
| `limit` | query | integer | Page size, 1 to 200 (default 50). |
| `before` | query | string (RFC 3339 nano) | The `at` timestamp of the oldest event from the previous page. |
| `limit` | query | integer | Page size, 1 to 200 (default 50). Anything else is a `400`. |
| `cursor` | query | string | Opaque pagination cursor from `pagination.next_cursor`. A malformed cursor is a `400`. |
| `before` | query | string (RFC 3339 nano) | Deprecated. Returns the events strictly older than this timestamp, which can skip events that share an instant with the page boundary; use `cursor`. A value that is not an RFC 3339 timestamp is a `400`. Ignored when `cursor` is set. |
### Response
Returns a `data` array and a `has_more` flag (not a cursor envelope). Paginate by passing the oldest event's `at` as `before`.
Returns a `data` array and the standard `pagination` envelope. Paginate by passing `pagination.next_cursor` back as `cursor` until `has_more` is `false`. The cursor is the exact position of the last event on the page (its time, source and row), so events that share a timestamp, common when a send, its open and a note land in the same second, are never skipped or repeated across pages. The top-level `has_more` mirrors `pagination.has_more` and is kept for clients written before the envelope. `pagination.total` is always `null`: the feed is merged from several tables and is never counted.
```json
{
"data": [
{
"type": "email_clicked",
"at": "2026-06-09T11:42:00Z",
"email_account_id": "e1...",
"email_account_email": "rep@yourco.com",
"campaign_id": "c1...",
"campaign_name": "Q3 Outbound",
"step_id": "s1...",
"step_name": "Intro",
"subject": "Quick question",
"machine": false,
"link": {
"id": "7c0f...",
"url": "https://yourco.com/pricing?utm_source=warmbly&utm_medium=email&utm_campaign=q3_outbound&utm_content=pricing",
"label": "Pricing",
"utm_source": "warmbly",
"utm_medium": "email",
"utm_campaign": "q3_outbound",
"utm_content": "pricing",
"user_agent": "Mozilla/5.0 ..."
}
},
{
"type": "email_replied",
"at": "2026-06-09T11:02:00Z",
@@ -710,13 +735,18 @@ Returns a `data` array and a `has_more` flag (not a cursor envelope). Paginate b
"user_id": "u1..."
}
],
"has_more": false
"has_more": false,
"pagination": { "total": null, "next_cursor": null, "has_more": false }
}
```
`type` is one of `email_sent`, `email_opened`, `email_clicked`, `email_replied`, `email_bounced`, `reply_received`, `deliverability`, `suppressed`, `note`, `meeting_booked`, `meeting_rescheduled`, `meeting_canceled`, `contact_created`, `campaign_added`, `campaign_removed`, `category_added`, or `category_removed`. Fields not relevant to an event type are omitted.
`type` is one of `email_sent`, `email_opened`, `email_clicked`, `email_replied`, `email_bounced`, `reply_received`, `deliverability`, `suppressed`, `note`, `meeting_booked`, `meeting_rescheduled`, `meeting_canceled`, `contact_created`, `campaign_added`, `campaign_removed`, `category_added`, `category_removed`, `form_submitted`, or `page_hit`. Fields not relevant to an event type are omitted.
Lifecycle events carry the name of what changed as it was at the time (`campaign_name`, or `category_id` plus `category_title`), so a later rename or deletion does not rewrite history. A `contact_created` event carries `source` (`manual`, `campaign`, `import`, `sheet_sync`, `api`, `ai_assistant`, or `unknown` for contacts that predate attribution) and `source_detail` (the file, campaign, sheet or API key name). The same values are on the contact itself as `source`, `source_detail` and `first_seen_at`, and never change after creation.
`email_opened` and `email_clicked` carry `machine`: `true` when an automated fetcher did it rather than the person (a mail privacy proxy, a fetch inside ten seconds of the send, several links followed within seconds). Per-link `email_clicked` events carry `machine_reason` (`prefetch`, `instant` or `burst`), and per-event `email_opened` rows carry it too (`prefetch` or `instant`); an open summarised from the lead alone carries only the flag. Both kinds carry an `origin` object when the event was logged: `client` when the user agent names a mail client or image proxy, `device_type`, `os`, `browser`, `browser_version`, and `country_code`, `region`, `city` when the consumer could resolve them. Opens appear once per event, so a contact who opened from two devices has two rows. Automated clicks are on the feed for the record but never count the step as clicked. An `email_clicked` event carries `link` with the link's `id`, `url`, `label` (its anchor text), the `utm_*` parameters the URL carried, and the `user_agent`; every link in an email is tracked on its own, so each link clicked is its own event. Clicks recorded before per-link attribution have no `link`.
A `form_submitted` event carries `form_id` and `form_name`. A `page_hit` event is a page view on your own site from a browser tied to the contact through an email-link ticket (see [Website tracking](/guides/website-tracking/)); `subject` is the page title, or its path when the page has none, and `page_hit` carries the full view: `url`, `path`, `title`, `referrer`, `referrer_domain`, `landing` (the first view of a session), the `utm_*` parameters, `device_type`, `os`, `browser`, `browser_version`, `device_brand`, `language`, `timezone`, `screen_width`, `screen_height`, and `country_code`, `region`, `city` when known.
Lifecycle events carry the name of what changed as it was at the time (`campaign_name`, or `category_id` plus `category_title`), so a later rename or deletion does not rewrite history. A `contact_created` event carries `source` (`manual`, `campaign`, `import`, `sheet_sync`, `api`, `form`, `automation`, `ai_assistant`, or `unknown` for contacts that predate attribution) and `source_detail` (the file, campaign, sheet, form, automation or API key name). The same values are on the contact itself as `source`, `source_detail` and `first_seen_at`, and never change after creation.
## Get a contact's campaign state
@@ -1,6 +1,6 @@
---
title: Deliverability and ops
description: Ingest deliverability events, replay dead-lettered tasks, manage warmup routing rules, reply templates, and org-level outreach settings.
description: Ingest deliverability events, manage the suppression list, replay dead-lettered tasks, manage warmup routing rules, reply templates, and org-level outreach settings.
---
This group covers the operational control surface for sending: posting deliverability signals (bounces, complaints, unsubscribes) back into the platform, listing and replaying dead-lettered tasks, defining premium-pool warmup routing preferences, managing reply templates, and reading or updating organization-wide advanced outreach settings. All routes are organization-scoped, so the caller must have an active organization selected (API keys are always bound to one organization).
@@ -75,10 +75,18 @@ Returns the `AdvancedOutreachSettings` object directly (not wrapped in an envelo
"show_intent_summary": true,
"show_dlq_stats": true
},
"unsubscribe": {
"mode": "text",
"text": "If this isn't relevant, just reply and let me know and I won't email you again.",
"link_intro": "Not the right person, or not interested?",
"link_text": "Unsubscribe"
},
"custom": {}
}
```
The `unsubscribe` block is the workspace default for the opt-out appended after the signature of every campaign email: `mode` is `text` (a reply-to-opt-out sentence, the default), `link` (a sentence with the recipient's signed unsubscribe link) or `off`; a campaign's own `unsubscribe_mode` (`inherit` by default) overrides it. Copy fields are single lines of at most 300 characters and fall back to the defaults when blank. See the [unsubscribe guide](/guides/unsubscribe/).
<Callout type="info" title="Content score floor is clamped">
`preflight.min_content_score` is stored clamped to `1`-`100`. A value outside that range is corrected on write rather than rejected, so a floor above `100` cannot flag every campaign permanently. Set `preflight.check_content_score` to `false` to turn the check off; the score never blocks or delays a send either way. See [Content checks](/guides/campaigns/).
</Callout>
@@ -171,6 +179,94 @@ Auth: **Scope** `WRITE_CAMPAIGNS` · **Org permission** `send_campaigns`
`202 Accepted` with no body. The event is queued for processing.
## List the suppression list
`GET /suppressions`
Pages the workspace suppression list, newest first: every address and domain no campaign will email, whatever put it there. Entries whose `expires_at` has passed are not returned.
Auth: **Scope** `READ_CONTACTS` · **Org permission** `view_contacts`
| Parameter | In | Type | Description |
| --- | --- | --- | --- |
| `q` | query | string | Substring filter on the address or domain. |
| `limit` | query | integer | Page size, 1 to 200. Default 50. |
| `cursor` | query | string | Opaque cursor from the previous page's `pagination.next_cursor`. |
### Response
```json
{
"data": [
{
"id": "5a0c...",
"organization_id": "9b1e...",
"email": "dana@acme.com",
"kind": "email",
"reason": "clicked the unsubscribe link",
"source": "unsubscribe",
"campaign_id": "c1...",
"metadata": { "via": "link" },
"created_at": "2026-09-01T10:12:00Z",
"updated_at": "2026-09-01T10:12:00Z"
},
{
"id": "7d2f...",
"organization_id": "9b1e...",
"email": "competitor.io",
"kind": "domain",
"reason": "Competitor",
"source": "manual",
"metadata": { "added_by": "4e5f..." },
"created_at": "2026-08-20T09:00:00Z",
"updated_at": "2026-08-20T09:00:00Z"
}
],
"pagination": { "next_cursor": null, "has_more": false }
}
```
`kind` is `email` or `domain`; a domain entry keeps the bare host in `email` and matches every address at it. `source` is `bounce`, `complaint`, `unsubscribe`, `manual` (added one at a time) or `import` (added as a batch). An invalid `limit` or `cursor` returns `400`.
## Add to the suppression list
`POST /suppressions`
Adds addresses and domains. A value containing `@` is an address; a bare host, with or without a leading `@`, is a domain. Values that are neither are reported in `skipped` rather than failing the request, and a value already on the list is updated in place, so the call is safe to repeat. At most 5000 entries per request.
Auth: **Scope** `WRITE_CONTACTS` · **Org permission** `manage_contacts`
### Request
```json
{
"entries": [
{ "value": "dana@acme.com", "reason": "Asked us by phone" },
{ "value": "acme.com" },
{ "value": "@partner.io" }
],
"reason": "Existing customers"
}
```
`reason` on the request applies to every entry without its own. Adding an address also switches off the matching contact's `subscribed` flag.
### Response
```json
{ "added": 3, "skipped": [] }
```
## Remove from the suppression list
`DELETE /suppressions/:id`
Lifts one entry, so campaigns can email the address (or every address at the domain) again. Removing an address entry also restores the matching contact's `subscribed` flag. The removal is written to the audit log with the entry's value, kind and source, since lifting an opt-out the recipient made themselves is the action a compliance review looks for.
Auth: **Scope** `WRITE_CONTACTS` · **Org permission** `manage_contacts`
Returns `204` on success and `404` when the entry does not exist in this organization.
## List task dead letters
`GET /tasks/dlq`
+5 -5
View File
@@ -458,11 +458,11 @@ The full catalog is available at `GET /webhooks/event-types`. Below is the compl
| `campaign.deleted` | A campaign is deleted. | |
| `campaign.email_sent` | A campaign email is dispatched by a worker. | yes |
| `campaign.email_delivered` | A campaign email is accepted by the recipient's mail server. | yes |
| `campaign.email_opened` | A tracked open is recorded. | yes |
| `campaign.email_clicked` | A tracked link click is recorded. | yes |
| `campaign.email_opened` | A person opens a tracked email. Automated opens (mail privacy proxies, fetches inside seconds of the send) do not fire it. | yes |
| `campaign.email_clicked` | A person clicks a tracked link. The payload carries the `url` and the link's `link_label`. Automated clicks (security gateways walking the links) do not fire it. | yes |
| `campaign.email_bounced` | A campaign email bounces. | |
| `campaign.reply_received` | A human reply lands for a campaign thread. | |
| `campaign.unsubscribed` | A recipient clicks the unsubscribe link or requests removal. | |
| `campaign.unsubscribed` | A recipient opts out. `source` is `one_click` (the mail client's button), `link` (the link in the email), `reply` (a reply asking to stop) or `action` (a sequence's unsubscribe step). | |
| `campaign.deliverability_warning` | The campaign's rolling bounce or complaint rate enters the early-warning band. | |
| `campaign.action` | A sequence flow `notify` action node fires. | |
@@ -507,10 +507,10 @@ The full catalog is available at `GET /webhooks/event-types`. Below is the compl
| Event | Description | Firehose |
| --- | --- | --- |
| `contact.created` | A contact is created. | |
| `contact.created` | A contact is created. Carries `contact_id`, `contact_email`, `first_name`, `last_name`, `company`, `phone`, `subscribed`, `custom_fields`, `source`, `source_detail`, `campaign_ids`, `category_ids` and `created_at`. Fires for contacts added by hand, through the API, by a form or by an automation; file imports, sheet syncs and a single API request adding more than 100 contacts are bulk arrivals and do not raise it. | |
| `contact.updated` | A contact's fields change. | |
| `contact.deleted` | A contact is deleted. | |
| `form.submitted` | A hosted form received a submission. Carries the form id and name, the submission id, the answers, and the contact id when one was created or matched. | |
| `form.submitted` | A hosted form received a submission. Carries the form id and name, the submission id, the answers under `data`, the mapped contact columns (`contact_email`, `first_name`, `last_name`, `company`, `phone`), `source_url`, and the contact id when one was created or matched. | |
### CRM
+14 -7
View File
@@ -333,6 +333,9 @@ PHX_HOST=ws.example.com
PORT=4000
CHECK_ORIGIN=true
# ── GeoIP (the variable is required by the backend; the file is optional) ──
GEODB_PATH=/var/lib/warmbly/GeoLite2-City.mmdb
# ── Platform email (resets, invitations, digests) ────────────────
MAIL_TRANSPORT=smtp
SMTP_HOST=smtp.example.com
@@ -357,7 +360,7 @@ A few lines differ from the compose defaults on purpose:
- `API_HOST`, `TRACKING_HOST` and the realtime `PORT` sit on `127.0.0.1`, so only nginx reaches them. The realtime service binds all interfaces regardless, so keep `4000` closed at the firewall
- `TRUSTED_PROXIES` and `TRACKING_TRUSTED_PROXIES` name the proxy, so rate limits and audit records see the visitor's address instead of `127.0.0.1`
- `CHECK_ORIGIN=true` makes the websocket refuse browsers that are not on `PHX_HOST`'s origin list
- `GEODB_PATH` is left unset. It only adds a city to sessions and audit rows; set it to a MaxMind `GeoLite2-City.mmdb` if you have one
- `GEODB_PATH` points at a file that need not exist. The backend refuses to start without the variable, but a missing database only means sessions, audit rows and email opens and clicks carry no city; drop a MaxMind `GeoLite2-City.mmdb` at that path if you have one
Every other variable, with its default, is in the [configuration reference](/development/configuration/), and [`deploy/config/env.example`](https://github.com/warmbly/warmbly/blob/main/deploy/config/env.example) is the annotated template this file was cut down from. Mail relay, OAuth clients, single sign-on and the AI provider are configured exactly as in the [self-hosting guide](/development/deployment-guide/#platform-email); only the way values reach the process differs.
@@ -562,16 +565,20 @@ Because the worker is not in a container, the admin panel's SSH-driven day-two a
## Upgrading
Rebuild the artifacts that changed, then restart. Migrations are forward-only and apply on backend boot, so bring the backend up before the rest:
`scripts/upgrade-bare-metal.sh` is the build steps above in order: build every artifact this host runs as the checkout's owner, then run the root-owned installer `warmbly-install-release` through sudo, which installs them, restarts the backend first (migrations are forward-only and apply on its boot), then the rest, keeping each frontend's `config.js` across the copy. Install that one script root-owned once, then run the upgrade as the user who owns the checkout:
```bash
cd /opt/warmbly/src && sudo git pull # or checkout a newer tag
# repeat the build steps for the services that changed, then:
sudo systemctl restart warmbly-backend
sudo systemctl restart warmbly-forms warmbly-consumer warmbly-tracking warmbly-realtime warmbly-worker
sudo install -o root -g root -m 0755 /opt/warmbly/src/deploy/systemd/warmbly-install-release.sh /usr/local/sbin/warmbly-install-release
echo "$USER ALL=(root) NOPASSWD: /usr/local/sbin/warmbly-install-release" | sudo tee /etc/sudoers.d/warmbly-upgrade >/dev/null
sudo chmod 0440 /etc/sudoers.d/warmbly-upgrade
cd /opt/warmbly/src && scripts/upgrade-bare-metal.sh --pull # git pull --ff-only, build, install, restart
```
Frontend rebuilds overwrite `config.js` when you copy `dist/` over, so rewrite it afterwards (or keep the two files somewhere and copy them back). A build script that does all of it in order is worth writing the second time you upgrade.
The sudoers line is what lets the script call the installer without a password prompt; it allows that one root-owned command and nothing else.
To pin a release instead of following the branch, check the tag out first (`git checkout vX.Y.Z`) and run it without `--pull`.
The admin panel can run the same script for you. The version pill in its top bar shows when a newer release exists, and with the updater unit installed, **Update and restart** in it pulls, runs the script and reports back. [Updates](/development/updates/#without-docker) has the unit, the sudoers line it needs and the backend variable that points at it.
## Backups
@@ -141,6 +141,7 @@ TRUSTED_PROXIES=10.0.0.0/8,172.16.0.0/12
| `DISABLE_PASSWORD_LOGIN` | Turns off email and password entirely, for single sign-on only deployments | `false` | yes |
| `SSO_AUTO_PROVISION` | `true` lets a verified identity provider assertion create an account regardless of `DISABLE_REGISTRATION` | `false` | yes |
| `AUTH_IP_RATE_LIMIT` | Unauthenticated auth requests allowed per source IP per 15 minutes | `60` | yes |
| `CLI_AUTH_IP_RATE_LIMIT` | CLI sign-in handshake requests allowed per source IP per 15 minutes. Its own budget, because one `warmbly auth login` polls around 200 times and must not exhaust the allowance above | `500` | yes |
| `WARMBLY_BOOTSTRAP_EMAIL` | First owner's address, read only while the users table is empty | unset | yes |
| `WARMBLY_BOOTSTRAP_PASSWORD_HASH` | Argon2 PHC string for that owner. Preferred over the plaintext form | unset | yes |
| `WARMBLY_BOOTSTRAP_PASSWORD` | Plaintext convenience form. Warns at boot, and leaves a password in your process environment | unset | yes |
@@ -305,7 +306,7 @@ Redis holds rate limit counters, the organization key cache, the realtime bridge
|---|---|---|---|
| `GEODB_PATH` | Path to a GeoLite2 City database | none, and the backend refuses to start without the variable | yes |
The variable must be set on the backend in every environment. The file itself is optional: a missing file at that path means sessions and audit rows are recorded without a city, and nothing else changes.
The variable must be set on the backend in every environment. The file itself is optional: a missing file at that path means sessions and audit rows are recorded without a city, and nothing else changes. The consumer reads the same variable, optionally, to put a country and city on each email open and click; without it those records carry client and device only.
## Workers
@@ -381,6 +382,20 @@ Delayed sends run through the local poller, so the backend must be running for s
| `SENTRY_DSN` | Error reporting. Optional in every environment, including `prod` | unset |
| `APNS_KEY` or `APNS_KEY_PATH`, `APNS_KEY_ID`, `APNS_TEAM_ID`, `APNS_TOPIC` | Mobile push on backend and consumer. Partial configuration disables push with a warning, never a crash | unset |
## Updates
| Variable | What it does | Default | Restart needed |
|---|---|---|---|
| `UPDATE_CHECK_ENABLED` | Polls GitHub Releases for a newer Warmbly and shows it in the admin panel's top bar and on Setup and health | `true` | yes |
| `UPDATE_CHECK_INTERVAL` | How often the release check runs. Minimum `5m` | `30m` | yes |
| `UPDATE_CHANNEL` | `stable` follows releases; `dev` also offers prereleases | `stable` | yes |
| `RELEASES_GITHUB_REPO` | The `owner/repo` whose releases count as Warmbly versions. Point a fork's instance at the fork | `warmbly/warmbly` | yes |
| `RELEASES_GITHUB_TOKEN` | Optional GitHub token; only raises the API rate limit | unset | yes |
| `UPDATER_URL` | The host-side updater that applies an update (pull, rebuild, restart). Unset, or `none`, leaves the panel report-only | `http://updater:8095` under compose | yes |
| `UPDATER_TOKEN` | The bearer token the backend presents to the updater | `INTERNAL_API_TOKEN` | yes |
The updater's own variables (`UPDATER_MODE`, `UPDATER_COMMAND`, `UPDATER_REPO_DIR` and the rest) are on [Updates](/development/updates/#configuration); it is a separate process and its environment is not on the configuration page.
## Forms service
The public face of hosted forms (`cmd/forms`): it serves the React (TanStack) form app built from `forms/`, the per-form page shells, the embed loader and public submissions, on its own port so form traffic never touches the API origin. It reads its own environment; none of these appear in the admin panel except `FORM_IP_RATE_LIMIT`.
@@ -406,6 +421,7 @@ The Rust open and click service. It reads its own environment, so these have to
| `TRACKING_RATE_LIMIT_PER_MIN` | Counted pixel and click requests per source per minute. Over budget, pixels are still served but not counted, and click redirects get `429` | `300` |
| `TRACKING_PAGEHIT_RATE_LIMIT_PER_MIN` | Website page views accepted per source per minute, on top of the shared budget above. Over budget, the snippet gets `429` | `60` |
| `TRACKING_TRUSTED_PROXIES` | CIDRs the tracking service accepts a forwarded client address from. Empty trusts nothing and uses the socket peer, which is correct for a directly exposed service; set it behind a reverse proxy or the per-source rate limits and the location stored with page views are caller-controlled. Same convention as the backend's `TRUSTED_PROXIES` | empty |
| `TRACKING_IP_HASH_KEY` | Secret the source-address token in tracking events is keyed with. The token names one source for deduplication, rate limits and the click burst rule; keyed, it cannot be turned back into the address by enumeration | `INTERNAL_API_TOKEN` |
| `TRACKING_CLIENT_IP_HEADER` | The one header a trusted proxy sets with the client address. No other header is read, so a caller cannot smuggle an address past a generic proxy in `CF-Connecting-IP`. For `x-forwarded-for` the proxy-appended last entry is used; set `cf-connecting-ip` behind Cloudflare | `x-forwarded-for` |
| `EVENTBUS_PROVIDER` | `nats` or `kafka`. Kafka needs an image built with `CARGO_FEATURES=kafka` | `nats` |
| `NATS_URL`, `NATS_SUBJECT_PREFIX` | JetStream address and subject prefix. The publish subject is `<prefix>.<topic>` | `nats://localhost:4222`, `warmbly` |
@@ -456,11 +472,24 @@ These are the only settings a browser can change, and no environment variable ow
| `sync.backfill_messages` | integer, 1 to 100000 | `5000` | The most messages that import stores per mailbox |
| `sync.daily_messages_per_mailbox` | integer, 1 to 100000 | `2000` | New (live) messages one mailbox may store per UTC day. Over it, mail waits for the next day; replies to the mailbox's own sends have a separate budget of the same size |
| `sync.daily_messages_per_org` | integer, 1 to 2000000 | `25000` | New plus imported messages one organization may store per UTC day |
| `retention.engagement_event_days` | integer, 1 to 3650 | `365` | How long the per-event open and click logs (client, device, approximate location) are kept. Campaign progress keeps its own summary that outlives them, so counts, filters and branching never change |
| `retention.form_event_days` | integer, 1 to 3650 | `180` | How long form funnel events (views, starts, field-level drop-off) are kept. Funnel reports range up to 90 days, so anything shorter shortens the report too |
| `retention.audit_log_days` | integer, 1 to 3650 | `90` | How long the audit trail is kept. It carries IP addresses, user agents and change payloads, so this is also how long that data is held |
| `deliverability.enforce_domain_auth` | boolean | `true` | Whether a sending domain that fails SPF or DMARC stops cold campaign sending and warmup sending from every mailbox on it. Off keeps the check running and still shows the state and the Advisor card, it just never blocks |
| `deliverability.auth_grace_hours` | integer, 1 to 720 | `72` | How long a domain must stay failing before the gate applies. The clock starts when the background check first sees the failure, so this is also how much warning the owner gets |
The four `sync.*` values are read by the backend when a mailbox is loaded onto a worker (on connect, on reassignment, and by the reconciler's periodic republish), so a change reaches every mailbox within a few minutes without a restart. The fixed pacing numbers around them (burst per five minutes, hourly, backfill pace, the flood threshold and the chronic-overage rule) are compiled constants listed under **Instance > Limits**; see [Mailboxes](/guides/mailboxes/#what-gets-synced) for how the budgets behave.
The three `retention.*` values are read by the pruning sweeps on every pass, so shortening one takes effect on the next sweep rather than at the next restart. Deletion is permanent and there is no grace period: what already sits outside a shortened window goes on that sweep. See [data control](/development/data-control/#what-is-kept-and-for-how-long) for what each log holds and what a shorter window costs.
An unattended install can seed the whole document before anyone signs in, with `WARMBLY_SETTINGS_BOOTSTRAP` holding the same partial JSON the admin API takes:
```bash
WARMBLY_SETTINGS_BOOTSTRAP={"sync":{"backfill_days":30},"retention":{"audit_log_days":30}}
```
It is applied only while the settings row has never been written, so from the first save in the panel onwards the panel is authoritative and the variable is a no-op. That is what [`install.sh --wizard`](/development/install/) writes when you answer its retention questions.
The two `deliverability.*` values are read on every scheduling pass and every warmup send, so turning the gate off releases blocked mailboxes within the 30 second cache window. Turning it back on does not stop anything retroactively: a domain still has to spend its whole grace window failing first. Only a sustained failure gates, so a domain reading `unknown` (never checked, DNS could not answer, or a special-use domain that cannot resolve) always sends. Manual sends and unibox replies are never gated; see [domain authentication](/guides/deliverability/#domain-authentication).
Changing them is audited, and every value is validated and clamped server side on write as well as on read, so a row written by an older version still resolves.
@@ -476,6 +505,8 @@ Changing them is audited, and every value is validated and clamped server side o
## See also
- [Install](/development/install/) for the one-command install that writes all of this for you
- [Data control](/development/data-control/) for where each store lives and what each retention window governs
- [First run](/development/first-run/) for claiming a fresh instance
- [Accounts and access](/development/accounts-and-access/) for who may sign in and how to invite people
- [Instance health](/development/instance-health/) for the checks that read these values back
@@ -0,0 +1,214 @@
---
title: Data control
description: Where every store lives on disk, what each retention window governs, how backups work, and how to move an instance to another host.
---
Self-hosting Warmbly means holding mailbox credentials, message bodies and contact records on your own disk. This page is the whole answer to where that data is, how long it stays, and how you get it off this machine.
Three questions, and they have different answers:
| Question | Answer |
|---|---|
| Where is it? | [Six stores](#where-the-data-sits), each pointed at by one variable |
| How long does it stay? | [Retention windows](#what-is-kept-and-for-how-long), all editable in the admin panel |
| How do I move it? | [`warmblyctl backup`](#backups) for the instance, [workspace export](/guides/workspace-export-import/) for one workspace |
## Where the data sits
Every store's location is one variable in `.env`. Compose reads a source starting with `/` as a bind mount and anything else as a named volume, so the same variable covers both and there is no second compose file.
| Variable | Holds | Default |
|---|---|---|
| `WARMBLY_PG_DATA` | Postgres: organizations, users, mailboxes (credentials sealed), contacts, campaigns, the audit trail | `<data root>/postgres` |
| `WARMBLY_BLOBS` | Message bodies, attachments, avatars and logos | `<data root>/blobs` |
| `WARMBLY_NATS_DATA` | The event bus's JetStream state | `<data root>/nats` |
| `WARMBLY_REDIS_DATA` | Cache and rate-limit counters. Disposable | `<data root>/redis` |
| `WARMBLY_WORKER_STATE` | A worker's own id and sync cursors. Disposable | `<data root>/worker` |
| `WARMBLY_UPDATER_STATE` | The last update job's log. Disposable | `<data root>/updater` |
An [install from `install.sh`](/development/install/) points all six under one data root, so `/opt/warmbly/data` is the whole of it and you can `rsync` that path. A clone-and-build install keeps Docker named volumes unless you set the same variables.
Only the first two carry anything you cannot rebuild. The other four are state a fresh container reconstructs, which is why a backup does not include them.
### Encryption
Two keys, and they are not interchangeable.
| Key | Opens | Read by |
|---|---|---|
| `CREDENTIALS_ENCRYPTION_KEY` | Mailbox SMTP and IMAP credentials | Backend and workers, without an organization context |
| `KMS_LOCAL_MASTER_KEY` | The per-organization data keys, which in turn open everything else | Backend and consumer |
Both are unrecoverable. A database backup without them restores an instance whose mailboxes authenticate against nothing, and there is no way back from that. They are in `.env`, and `install.sh` also writes them to `keys-backup.txt` next to the install, which is on the same disk as the database and therefore not a backup either. Copy them somewhere else.
### External stores
Postgres, Redis and blob storage each accept an external target, set at install time or by editing `.env`:
```bash
PRIMARY_DB=postgres://user:pass@db.internal:5432/warmbly?sslmode=require
REDIS=redis://cache.internal:6379
BLOB_PROVIDER=s3
BLOB_BUCKET=warmbly
AWS_ENDPOINT_URL_S3=https://<account>.r2.cloudflarestorage.com
```
<Callout type="warn" title="Filesystem blobs stop working when workers run off-host">
A remote worker writes bodies to its own disk, so the dashboard on the control-plane host finds nothing. Any deployment with workers on more than one machine needs S3-compatible blob storage. This is the one storage choice that is not just a preference.
</Callout>
## What is kept, and for how long
Every window below lives in the database, not the environment, and is edited under **Instance > Instance settings** in the admin panel. A sweep runs a few times a day and reads the current value on every pass, so a change takes effect without a restart.
### Mailbox import and sync
| Setting | Default | Range | What it governs |
|---|---|---|---|
| `sync.backfill_days` | 90 | 1 to 730 | How far back the initial import reaches when a mailbox is connected, newest first |
| `sync.backfill_messages` | 5,000 | 1 to 100,000 | The most messages that import stores per mailbox |
| `sync.daily_messages_per_mailbox` | 2,000 | 1 to 100,000 | New mail one mailbox may store per UTC day |
| `sync.daily_messages_per_org` | 25,000 | 1 to 2,000,000 | New plus imported mail across one workspace per UTC day |
Mail over a daily budget is deferred, never dropped: the provider cursor is held and the mail is re-offered on the next pass. Replies to the mailbox's own outreach ride a separate budget of the same size and keep landing regardless.
### Event history
| Setting | Default | What it holds |
|---|---|---|
| `retention.engagement_event_days` | 365 | Per-event open and click logs: client, device, approximate location |
| `retention.form_event_days` | 180 | Form funnel events: views, starts, field-level drop-off |
| `retention.audit_log_days` | 90 | The audit trail: actor, IP address, user agent, change payload |
Each is between 1 and 3,650 days. These are the three settings a retention or privacy policy applies to, because each window is also how long the personal data in that log is held.
None of them change a number anyone reads. Campaign progress keeps its own summary of opens and clicks that outlives the per-event log, so counts, filters and branching are unaffected by shortening any of these. What gets shorter is what a contact's timeline can show, how far a funnel report reaches, and how far back an admin can audit.
<Callout type="warn" title="Shortening a window deletes on the next sweep">
There is no grace period and no copy. Take a backup first if you are not sure.
</Callout>
The **minimal retention** preset in the admin panel and in the installer sets all three to 30 days.
### Set them at install time
The installer writes the answers into `.env` as one document, applied on the first boot of a fresh database:
```bash
WARMBLY_SETTINGS_BOOTSTRAP={"sync":{"backfill_days":30},"retention":{"audit_log_days":30}}
```
It is read only while the settings row has never been written. From the first save in the admin panel onwards the panel is authoritative, so leaving the line in `.env` never undoes a later edit.
## Backups
`warmblyctl backup` writes one bundle holding the three things that only restore together:
- the database, as a `pg_dump`
- the blob root, when blobs are on the filesystem
- the encryption keys, unless you pass `--no-keys`
```bash
docker compose -p warmbly exec backend warmblyctl backup --out /data/blobs/warmbly.tar.gz
docker compose -p warmbly cp backend:/data/blobs/warmbly.tar.gz ./warmbly.tar.gz \
&& docker compose -p warmbly exec -T backend rm -f /data/blobs/warmbly.tar.gz
```
`/data/blobs` is a hand-off, not a destination: it is the one path the container and the host both see. The `&&` matters twice over. `backup` leaves its own output out of the archive, but a bundle left there is swept into the next run, so it has to be deleted; and a `cp` that failed must not be followed by deleting the only copy that exists.
The bundle is written 0600 and holds every mailbox credential on the instance plus the keys that open them. Treat the file as you would the instance itself.
An install from `install.sh --wizard` can schedule this for you: `backup.sh` next to the install, a systemd timer, a retention count, and an optional `aws s3 cp` to somewhere off the host. A backup that only exists on the machine it backs up is not one.
### Restore
On the destination host, with the same keys in place:
```bash
docker compose -p warmbly exec backend warmblyctl restore --file /data/blobs/warmbly.tar.gz
docker compose -p warmbly restart
```
The restore empties the schema and replays the dump, so it replaces everything currently on that instance and asks you to type `restore` first.
Before it does anything it compares the bundle's `CREDENTIALS_ENCRYPTION_KEY` and `KMS_LOCAL_MASTER_KEY` against the destination's and refuses to continue when they differ, printing the two lines to put in `.env`. That check is the point of the command: without it a restore looks like it worked and every mailbox fails to authenticate days later, with no error that names the cause.
## Moving an instance
Two ways, and they answer different questions.
### The whole instance, to a new host
Every workspace, every user, the platform admins, the API keys.
<Steps>
<Step>
### Install Warmbly on the new host
```bash
curl -fsSL https://warmbly.com/install.sh | sh -s -- --host <new-hostname>
```
</Step>
<Step>
### Put the old keys in the new `.env`
Copy `CREDENTIALS_ENCRYPTION_KEY` and `KMS_LOCAL_MASTER_KEY` from the old install, then recreate the containers so they take:
```bash
docker compose -p warmbly up -d
```
</Step>
<Step>
### Restore the bundle
```bash
docker compose -p warmbly exec backend warmblyctl restore --file /data/blobs/warmbly.tar.gz
docker compose -p warmbly restart
```
</Step>
<Step>
### Check it
```bash
docker compose -p warmbly exec backend warmblyctl status
```
Mailboxes should be connected, not needing a reconnect. If they need one, the keys did not match.
</Step>
</Steps>
The `rsync` alternative works too and is sometimes simpler, as long as the stack is **stopped** first: a running Postgres data directory copied file by file is not a consistent snapshot and can restore as a corrupt cluster. Stop it, copy the data root and the `.env` to the new host, start it there. It moves the same bytes; the bundle exists because it is the version that survives a different host layout, a different Postgres, a live instance, and a partial copy.
### One workspace, to another instance
[Workspace export and import](/guides/workspace-export-import/) moves a single organization's data between two running instances, re-sealing its secrets for the destination's keys. That is the per-customer tool; the bundle here is the instance-level one. They are not interchangeable: a bundle cannot be applied to one workspace, and a workspace archive cannot restore an instance.
## Outbound calls
A self-hosted Warmbly makes no outbound call of its own except one, and it is off with a single setting.
| Call | When | Turn it off |
|---|---|---|
| GitHub releases API | Every 30 minutes, to tell the admin panel a newer version exists | `UPDATE_CHECK_ENABLED=false` |
Everything else is you: mail through the mailboxes you connect, DNS lookups for the domains you check, and whatever integrations you configure. There is no telemetry, no phone-home, and no license check.
## See also
- [Install](/development/install/): the wizard that asks all of this up front
- [Configuration](/development/configuration/): every variable and every database-backed setting
- [warmblyctl](/development/warmblyctl/): `backup`, `restore`, and the operator commands
- [Workspace export and import](/guides/workspace-export-import/): the per-workspace story
@@ -5,6 +5,15 @@ description: A step-by-step guide to running the whole Warmbly platform yourself
Warmbly self-hosts with **no cloud account of any kind**: no AWS, no GCP, no Stripe, no Kafka. One `docker-compose.yml` runs everything on local, open-source pieces, and every external dependency is an environment variable you can flip later.
<Callout title="Just want it running?">
```bash
curl -fsSL https://warmbly.com/install.sh | sh
```
That pulls the published release images and skips everything on this page: no clone, no compiler, under two minutes. Add `--wizard` and it asks where your data lives, what is kept and for how long, and how it is backed up. See [Install](/development/install/).
This page is the build-from-source path, and the reference for every setting either path writes. Take it when you want to run modified code, a platform we publish no image for, or a build you produced yourself.
</Callout>
Three commands give you a working install. Everything after them is optional, and nothing below is needed before you have seen it running.
<Mermaid
@@ -226,7 +235,7 @@ These are the ones that break things quietly when they drift, because each servi
| realtime | `JWT_SECRET`, `SECRET_KEY_BASE`, `DATABASE_URL`, `REDIS_URL`, `PHX_HOST` |
| web / admin | Only `WARMBLY_*` URLs, read at container start and written into `/config.js`. The same image runs anywhere |
`APP_ENV` accepts `dev` or `prod`. No other value turns on production behavior. `prod` needs no cloud account: `SENTRY_DSN` stays optional in every environment, and `GEODB_PATH` must be *set* on the backend everywhere while the file it points at is optional.
`APP_ENV` accepts `dev` or `prod`. No other value turns on production behavior. `prod` needs no cloud account: `SENTRY_DSN` stays optional in every environment, and `GEODB_PATH` must be *set* on the backend everywhere while the file it points at is optional (the consumer reads it too, optionally, for the location on opens and clicks).
The complete list of variables, with defaults and whether a change needs a restart, is the [configuration reference](/development/configuration/).
@@ -740,22 +749,38 @@ CI publishes multi-arch images to `ghcr.io/<owner>/warmbly/`, which works on any
| Trigger | Images | Tags |
|---------|--------|------|
| Push to `main` | backend, consumer, worker, tracking, realtime | `:<sha>`, `:dev` |
| Push to `main` | backend, consumer, worker, forms, updater, tracking, realtime | `:<sha>`, `:dev` |
| Tag `vX.Y.Z` | all of the above plus web and admin | `:vX.Y.Z`, `:vX.Y`, `:vX`, `:prod` |
Every app service in `docker-compose.yml` carries both an `image:` and a `build:` key, so the same file serves both paths. `docker compose pull && docker compose up -d` runs the published images and compiles nothing; `docker compose up --build` still builds this checkout. Pin a release with `WARMBLY_TAG=v1.4.2` in `.env`, or point `WARMBLY_IMAGE_PREFIX` at your own registry.
To move a worker fleet onto a new release, update `WORKER_IMAGE` and use "Pull latest and restart" on each worker. Control-plane migrations are forward-only, so prefer rolling forward over rolling back.
## Upgrading and backups
The admin panel tells you when a newer version exists: the version pill in its top bar turns amber, and **Update and restart** in it pulls the checkout, rebuilds, restarts what changed and reconnects once the backend is back. `make up` starts the updater that makes the button work. The whole flow, what it does and how to run it without Docker, is on [Updates](/development/updates/).
By hand it is the same two steps:
```bash
git pull && make up # migrations apply on backend boot
make upgrade # git pull --ff-only, then make up; migrations apply on backend boot
```
Upgrading is safe with data in place: migrations are forward-only and apply on backend boot.
### Backing up
Back up three things. All three, or the backup is not a backup:
One command writes all three of the things that only restore together:
```bash
docker compose -p warmbly exec backend warmblyctl backup --out /data/blobs/warmbly.tar.gz
docker compose -p warmbly cp backend:/data/blobs/warmbly.tar.gz ./warmbly.tar.gz \
&& docker compose -p warmbly exec -T backend rm -f /data/blobs/warmbly.tar.gz
```
That bundle holds the database, the blob root and the encryption keys, and `warmblyctl restore` puts it back on another host, refusing to run when that host's keys are not the ones the bundle was sealed with. [Data control](/development/data-control/#backups) covers scheduling it and moving an instance.
By hand, if you would rather assemble it yourself, it is these three and no fewer:
```bash
docker compose -p warmbly exec -T postgres pg_dump -U warmbly warmbly_dev > backup.sql
+4 -1
View File
@@ -74,13 +74,16 @@ Produced by the Rust tracking service when a pixel loads or a tracked link is cl
"event_type": "EMAIL_OPENED",
"task_id": "uuid",
"original_url": "https://example.com/page",
"link_id": "uuid",
"timestamp": "2026-01-29T12:00:00Z",
"user_agent": "Mozilla/5.0...",
"ip_hash": "sha256..."
}
```
`event_type` is `EMAIL_OPENED` or `EMAIL_CLICKED`; `original_url` is set only for clicks; IPs are stored as hashes, never raw. The struct is `events.TrackingEvent` in `internal/events/schemas.go`, mirrored in `tracking/src/events.rs`.
`event_type` is `EMAIL_OPENED` or `EMAIL_CLICKED`; `original_url` and `link_id` (the click ticket, which names the link's stored destination and anchor text) are set only for clicks; IPs are stored as hashes, never raw. The struct is `events.TrackingEvent` in `internal/events/schemas.go`, mirrored in `tracking/src/events.rs`.
The consumer classifies each event before it counts. An open is automated when the user agent is a mail privacy proxy or missing, or when it arrives within `TrackingMachineWindowSeconds` of the step's dispatch; it is recorded with `opened_machine` and upgraded by a later human open. A click is automated for a missing user agent, for arriving inside the same window, or when the same source clicked another link of the same email within `TrackingClickBurstSeconds`; every click is logged per link in `email_link_clicks` with its reason, and only a human click stamps `clicked_at`, fires instant actions, or emits a webhook. A burst is only recognisable from its second click, so a human click's stamp and log row are written at once but its effects (evidence, instant actions, webhook, realtime event) run after the burst window plus a second, on the classification the click has by then; a burst recognised meanwhile relabels the earlier click and withdraws the stamp when no human click remains. A consumer restart inside the window loses only those deferred effects, and routing still follows the clicked branch at the next step boundary.
Website page views do not ride this topic. The tracking service forwards each accepted view to the backend's internal API (`POST /api/v1/internal/page-hits`) instead, because the backend is where the user agent and IP are turned into device and location, and the IP must not sit in a durable stream on the way there.
+7 -1
View File
@@ -7,12 +7,18 @@ A fresh instance has no accounts and is described as **unclaimed**. Claiming it
## Claim the instance
```bash
curl -fsSL https://warmbly.com/install.sh | sh
```
The [installer](/development/install/) starts the stack and prints the claim link when it finishes. From a clone it is the same two steps and a build:
```bash
git clone https://github.com/warmbly/warmbly && cd warmbly
make up
```
`make up` builds the images, starts the stack, waits for the backend to answer, then runs `make claim`, which prints a single-use link and ends on the command that reissues it:
Both print a single-use link when they finish. `make claim`, or `warmblyctl setup-link` inside the backend container, prints a fresh one at any time while the instance is still unclaimed:
```
No accounts exist yet. Open this link to claim the instance and
+318
View File
@@ -0,0 +1,318 @@
---
title: Install
description: One command installs Warmbly from published release images, with a wizard that asks where your data lives, what is kept and for how long, and how it is backed up.
---
```bash
curl -fsSL https://warmbly.com/install.sh | sh
```
No clone, no compiler, no toolchain. The script checks Docker, creates one directory, generates every secret, pulls the pinned release images, starts the stack and prints the link that claims your instance. On a clean box with Docker already installed it finishes in under two minutes.
Add `--wizard` and it asks the questions a config reference cannot ask for you.
```bash
curl -fsSL https://warmbly.com/install.sh | sh -s -- --wizard
```
## What it installs
| | |
|---|---|
| Where | `/opt/warmbly`, or wherever you point `--dir` |
| What | `docker-compose.yml`, a `.env` at 0600, `keys-backup.txt`, and the data root |
| Version | The newest release, resolved once and pinned in `.env`. Never `latest` |
| Images | `ghcr.io/warmbly/warmbly/*`, multi-arch for amd64 and arm64 |
| Services | Backend, consumer, worker, dashboard, admin, Postgres, Redis, NATS, and (unless you say core only) tracking, realtime and forms |
Nothing is built on your machine. The clone-and-build path still exists and is still supported; see [self-hosting](/development/deployment-guide/) for when you want it.
## Before you run it
Piping a script into a shell is a fair thing to be uneasy about, and the objection is only answerable if the script is versioned, inspectable and checksummed. All three are:
```bash
curl -fsSLO https://warmbly.com/install.sh
curl -fsSLO https://warmbly.com/install.sh.sha256
sha256sum -c install.sh.sha256
less install.sh
sh install.sh
```
The script itself is written to deserve that: `set -eu`, every line inside a function, and `main "$@"` on the very last line, so a download that is cut off half way executes nothing at all. It never asks to be piped into `sudo`; it asks for elevation only for the specific steps that need it, and names them.
You also do not have to run it to see what it would do:
```bash
sh install.sh --dry-run
```
That prints the exact `.env` and compose file it would write, and touches nothing.
### And after it runs
Every release publishes an `images.json` asset listing the manifest digest of each image in it. Once the pull finishes the installer compares what actually landed on the machine against that file, and stops before starting anything if they differ. You can check the same thing yourself at any time:
```bash
docker image inspect ghcr.io/warmbly/warmbly/backend:v1.4.2 --format '{{index .RepoDigests 0}}'
```
An install pointed at your own registry with `--registry` has no manifest to check against, and the installer says so rather than pretending it verified something.
## Requirements
| You need | Notes |
|---|---|
| Linux or macOS, amd64 or arm64 | On Windows, run it inside WSL |
| Docker 20.10+ with Compose v2 | The script offers to install Docker on Linux, and never does so quietly |
| ~2 GB disk | The images, not a build cache. There is no build |
| 4 GB RAM | The running stack idles near 300 MB |
| Free ports | 8080, 5173, 5174, and 3000, 4000, 8090 with every component |
Port `3000` is the one that actually collides: it is the default for a lot of other self-hosted software. The installer checks every port before it writes anything and tells you which one is taken.
## The wizard
`--wizard` walks nine steps. Every answer has a default that matches the fast path, so pressing enter through the whole thing produces exactly the install the one-liner produces.
Every menu takes the keys you already use:
| Keys | Does |
|---|---|
| `↑` `↓`, `j` `k`, `ctrl-n` `ctrl-p` | Move |
| `ctrl-d` `ctrl-u` | Half a screen |
| `gg`, `G` | First, last |
| `1` to `9` | Jump straight to that option |
| `enter`, `space`, `l` | Take it |
| `q`, `esc`, `ctrl-c` | Leave without installing |
The review screen at the end takes `enter` to install, `e` to go back into a section, and `q` to quit.
Steps append rather than replacing each other, so the whole run stays scrollable and nothing above the installer is touched. `--clear` redraws the screen at each step instead, if you prefer that.
<Steps>
<Step>
### Where it lives
One directory holds the compose file, the `.env`, and, unless you move it in step three, every store this instance writes to. Default `/opt/warmbly`.
</Step>
<Step>
### How it is reached
The hostname every link is built from: the dashboard URL, the unsubscribe link in campaign mail, the tracking pixel. Then how it is served:
| Choice | What it does |
|---|---|
| **Plain HTTP** | Ports published as they are. Correct for localhost or a private network |
| **Bundled Caddy** | Adds a Caddy service with automatic HTTPS, one hostname per surface, and binds everything else to loopback |
| **Behind your proxy** | Binds every service to `127.0.0.1` and sets `TRUSTED_PROXIES`, which is silently wrong on every proxied install that was configured by hand |
`TRUSTED_PROXIES` matters more than it looks. Without it every request appears to come from the proxy, so per-IP rate limits and click deduplication stop distinguishing between people.
The bundled Caddy option serves six hostnames under the domain you give it, and the installer prints the DNS records at the end:
```
app.example.com admin.example.com api.example.com
ws.example.com track.example.com forms.example.com
```
</Step>
<Step>
### Where the data sits
The question the rest of the install cannot answer for you, and the reason the wizard exists.
| Choice | What you get |
|---|---|
| **Under the install directory** | `/opt/warmbly/data` holds Postgres, blobs, NATS and worker state |
| **A path you choose** | An external disk or a mount you already snapshot |
| **Docker named volumes** | Managed by Docker, portable only through `docker volume` |
A real path is the default, because it is the choice that makes backup and migration obvious: with the stack stopped, moving the instance to another host is `rsync` of that path plus the `.env`. Do not copy it while Postgres is running; a live data directory is not a consistent snapshot. To move an instance without stopping it, use [`warmblyctl backup`](/development/data-control/#backups). Compose reads a volume source starting with `/` as a bind mount and anything else as a named volume, so this is one variable per store and no second compose file.
Then, behind one prompt: an external Postgres or Redis (the bundled container is left out entirely when you bring your own), and where message bodies, attachments and avatars go.
<Callout type="warn" title="Filesystem blobs are local to one machine">
A worker running on another host writes to its own disk. Once you scale workers out, blob storage has to be S3-compatible or bodies go missing. The wizard offers MinIO, R2, B2 and AWS.
</Callout>
</Step>
<Step>
### Keys and secrets
Five secrets are generated: `AUTH_SECRET`, `INTERNAL_API_TOKEN`, `SECRET_KEY_BASE`, `KMS_LOCAL_MASTER_KEY` and `CREDENTIALS_ENCRYPTION_KEY`. They are written 0600, `APP_ENV` is set to `prod`, and no published default is ever reachable through this path.
The last two are unrecoverable. Every mailbox credential and every per-organization key is sealed with them, so a database backup that does not travel with them restores an instance whose mailboxes authenticate against nothing. The wizard prints both, writes them to `keys-backup.txt` next to the install, and waits until you type that you have a copy somewhere else.
</Step>
<Step>
### What is kept, and for how long
Two groups, both editable afterwards in **Instance > Instance settings**:
| Setting | Default | What it governs |
|---|---|---|
| Import window | 90 days | How far back the initial import reaches when a mailbox is connected |
| Import cap | 5,000 | The most messages that import stores per mailbox |
| Daily per mailbox | 2,000 | New mail one mailbox may store per UTC day |
| Daily per organization | 25,000 | New plus imported mail across one workspace per UTC day |
| Opens and clicks | 365 days | Per-event logs, with the client, device and approximate location of each |
| Form funnel events | 180 days | Views, starts, field-level drop-off |
| Audit log | 90 days | Who did what, from which IP address and user agent |
The last three are also how long that personal data is held. A "minimal retention" preset sets every event window to 30 days.
</Step>
<Step>
### Backups
A scheduled `warmblyctl backup`: a target directory, a frequency, how many to keep, and whether the encryption keys travel in the bundle. It writes `backup.sh` next to the install and, on a systemd host, a service and a timer.
Including the keys makes the bundle as sensitive as the instance itself. Leaving them out means keeping `keys-backup.txt` somewhere else, or the bundles cannot be restored at all.
</Step>
<Step>
### Who gets in
The first owner, either as a printed claim link or unattended from `WARMBLY_BOOTSTRAP_EMAIL` plus a hash from `warmblyctl hash-password`. Then the registration posture, and platform mail: an SMTP relay, or skip it and have password resets and login codes printed to the backend log.
</Step>
<Step>
### Footprint
Everything, or core only (no tracking pixel, no websockets, no hosted forms). Ports are checked for collisions before anything is written. Last, the update check, which is one outbound call to the GitHub releases API and the only outbound call this instance makes on its own. There is no telemetry in Warmbly.
</Step>
<Step>
### Review
Every answer on one screen. Enter installs, `e` goes back to any section, `q` leaves without writing anything.
</Step>
</Steps>
## See it first
```bash
curl -fsSL https://warmbly.com/install.sh | sh -s -- --demo
```
The real wizard and the real review, with the pull, the container creation and the health wait played rather than run. It writes no file, pulls no image, creates no container and needs no Docker, so it answers "what does this actually do" without a server to try it on.
It takes about half a minute, paced so the phases are readable; a real install is several minutes, most of it the pull and the first boot's migrations. `WARMBLY_DEMO_FAST=1` collapses it to under ten seconds. From a checkout: `make installer-demo`.
## For agents
An agent should never drive the wizard: it reads raw keypresses and needs a
terminal. Every answer is also a flag, so `--yes` with flags is silent,
deterministic and idempotent, and `--print-env` and `--dry-run` show what a run
would produce without producing it.
The repository ships a `warmbly-install` skill under `skills/` covering exactly
that, plus backups and moving an instance. Install it the way your agent
expects, for example `cp -r skills/warmbly-install ~/.claude/skills/`.
## Unattended
Every answer is a flag and a `WARMBLY_*` environment variable, so the same script provisions from Ansible or cloud-init:
```bash
curl -fsSL https://warmbly.com/install.sh | sh -s -- --yes \
--dir /opt/warmbly \
--host warmbly.acme.com \
--tls caddy \
--data-root /mnt/data/warmbly \
--blobs s3 \
--retention-preset minimal \
--backup-dir /mnt/backups/warmbly \
--version v1.4.2
```
| Flag | Variable | Default |
|---|---|---|
| `--dir` | `WARMBLY_DIR` | `/opt/warmbly` |
| `--host` | `WARMBLY_HOST` | `localhost` |
| `--tls` | `WARMBLY_TLS` | `none` (`caddy`, `proxy`) |
| `--data-root` | `WARMBLY_DATA_ROOT` | `<dir>/data` (or `volumes`) |
| `--blobs` | `WARMBLY_BLOBS` | `filesystem` (`s3`) |
| `--database-url` | `WARMBLY_DATABASE_URL` | bundled Postgres |
| `--redis-url` | `WARMBLY_REDIS_URL` | bundled Redis |
| `--components` | `WARMBLY_COMPONENTS` | `full` (`core`) |
| `--version` | `WARMBLY_VERSION` | newest release |
| `--channel` | `WARMBLY_CHANNEL` | `stable` (`dev`) |
| `--registry` | `WARMBLY_IMAGE_PREFIX` | `ghcr.io/warmbly/warmbly` |
| `--backup-dir` | `WARMBLY_BACKUP_DIR` | none |
| `--backup-keep` | `WARMBLY_BACKUP_KEEP` | `14` |
| `--retention-preset` | | `default` (`minimal`) |
| `--no-update-check` | `WARMBLY_UPDATE_CHECK` | on |
`--print-env` emits the `.env` and exits. `--dry-run` prints every file it would write and exits. `--demo` walks the whole thing and installs nothing.
## Re-running it
The installer is idempotent. A second run:
- adopts the existing `.env` and never regenerates a secret, because a new `AUTH_SECRET` signs everyone out and a new `CREDENTIALS_ENCRYPTION_KEY` is permanent data loss
- keeps the existing data root, because pointing an install at a new path is not a move, it is an empty instance next to a full one
- keeps any keys you added to `.env` yourself, in a clearly marked block at the end
- regenerates `docker-compose.yml`, keeping the previous one as `docker-compose.yml.bak`
- refuses a directory it did not create, unless you pass `--force`
Put your own compose changes in a `docker-compose.override.yml`, which compose merges on top and the installer never touches.
## Updating
The version pill in the admin panel, then **Update and restart**. On an install created this way the updater runs in image mode: it writes the new tag into `.env`, pulls, and recreates the containers whose image changed. Nothing is built. See [updates](/development/updates/).
By hand, from the install directory:
```bash
docker compose -p warmbly pull && docker compose -p warmbly up -d
```
## Removing it
```bash
sh install.sh --uninstall
```
Stops and removes the containers and leaves every byte of data where it is. Adding `--purge-data` deletes the data root and the install directory as well, and asks you to type a phrase first, because that also destroys the only keys that could have opened a backup of it.
## When it goes wrong
| Symptom | Cause |
|---|---|
| `Docker is installed but not answering` | The daemon is not running, or your user is not in the `docker` group yet. `sudo systemctl start docker`, then log out and back in |
| `Something already listens on 3000` | Another self-hosted tool. Change the published port in `docker-compose.yml` before starting |
| `Could not pull the release images` | The tag does not exist, or this host cannot reach `ghcr.io`. Check [releases](https://github.com/warmbly/warmbly/releases) |
| The API never answers | The first boot applies every migration. `docker compose -p warmbly logs -f backend` |
| No claim link printed | The database already has accounts, so there is nothing to claim. See [first run](/development/first-run/) |
More in [troubleshooting](/development/troubleshooting/) and [instance health](/development/instance-health/).
## Next
- [First run](/development/first-run/): claiming the instance, the first owner, demo data
- [Data control](/development/data-control/): where every store lives, what each retention window governs, backups, and moving an instance
- [Configuration](/development/configuration/): every environment variable and every database-backed setting
- [Self-hosting](/development/deployment-guide/): the full guide, including building from source
@@ -61,6 +61,7 @@ Two neighbouring endpoints round out the surface, both under **Instance** in the
|---|---|---|
| `GET /admin/instance/config` | `manage_settings` | Every configuration entry with its resolved value, source and restart requirement. Sensitive keys return a fingerprint, never a value |
| `GET /admin/instance/limits` | `view_analytics` | The effective sending, warmup and rate limits this build compiles in |
| `GET /admin/instance/update` | `view_analytics` | The running build, the newest release, and the updater's state. Backs the version pill in the top bar; see [Updates](/development/updates/) |
## Security and secrets
@@ -164,6 +165,16 @@ Platform mail is sent from a domain that differs from the dashboard's. Mailbox p
`AUTH_LOGIN_CODE=always` was demoted to `new_device` at boot because the transport does not deliver. Otherwise nobody could ever complete a login. See [login codes](/development/accounts-and-access/#login-codes).
## Updates
### update_available
A newer Warmbly exists: a release on the configured channel is newer than the running build, or the checkout is behind its branch. The message names both versions and how to apply it. With the updater running, the version pill in the top bar has an **Update and restart** button; without it, `git pull && make up`. See [Updates](/development/updates/).
### updater_unreachable
`UPDATER_URL` is set but nothing answers there, so the update button cannot work. Start the updater (the `updater` compose profile, or the systemd unit on a bare-metal host), fix the address, or remove the variable to update by hand. See [enabling the updater](/development/updates/#enabling-the-updater).
## Accounts and access
### registration_mode
@@ -241,7 +252,7 @@ A reachable realtime service that still leaves the dashboard dead has a differen
| Service | Needs |
|---|---|
| backend | The five secrets, `PRIMARY_DB`, `REDIS`, the provider switches, the public URLs, `EMAIL_ADDRESS`, `EMAIL_NAME` and `GEODB_PATH` |
| consumer | The same shared block. It writes to Postgres, so it needs `PRIMARY_DB` and both encryption keys, and it needs the mail identity or it silently sends nothing |
| consumer | The same shared block. It writes to Postgres, so it needs `PRIMARY_DB` and both encryption keys, and it needs the mail identity or it silently sends nothing. `GEODB_PATH` is optional and only adds a location to opens and clicks |
| worker | No database. The event bus, `REDIS`, both encryption keys, `ENCRYPTED_KEYS_BACKEND_URL` plus the worker token, and the `BOX_*` OAuth clients |
| tracking | The event bus, plus `BACKEND_INTERNAL_URL` and `INTERNAL_API_TOKEN`. It exits at boot without either of those two |
| realtime | `JWT_SECRET` equal to the backend's `AUTH_SECRET`, plus `SECRET_KEY_BASE` and `DATABASE_URL`. It refuses to boot without all three. `REDIS_URL`, `PHX_HOST` and the connection limits have defaults |
+4
View File
@@ -4,13 +4,17 @@
"root": true,
"pages": [
"---Self-hosting---",
"install",
"deployment-guide",
"bare-metal",
"first-run",
"accounts-and-access",
"warmblyctl",
"data-control",
"configuration",
"instance-health",
"operator-notifications",
"updates",
"troubleshooting",
"---Development---",
"local-development",
@@ -0,0 +1,79 @@
---
title: Operator notifications
description: Send instance events to Discord, Slack, a signed webhook, or an email address.
---
Your instance can tell you when something happens on it: a new signup, a worker going offline, a workspace asking for more capacity. Configure the destinations in the admin panel under **Configuration > Notifications**.
These are operator alerts, not customer alerts. They are instance-wide and go to you. The per-workspace event delivery your customers configure is [Webhooks](/guides/webhooks/), which is a separate, queued and retried system.
## Channels
A channel is one destination. You can add up to 25.
| Type | What to paste | Notes |
| --- | --- | --- |
| Discord | An incoming webhook URL | Server settings, then Integrations, then New Webhook |
| Slack | An incoming webhook URL | Create a Slack app with an incoming webhook |
| Webhook | Any HTTPS endpoint | Receives the event as JSON, optionally signed |
| Email | An address | Needs a working platform mail transport |
Each channel picks which events it wants. Leave every event unchecked and it receives all of them, including ones added in later versions.
Use **Test** on a channel to deliver a sample alert immediately. A test ignores the channel's on/off switch and its event selection, so it always sends, and it reports the transport error directly when delivery fails.
### URL safety
Webhook URLs must be HTTPS and resolve to a publicly routable host. Inline credentials are rejected. This is the same posture customer webhooks use, and it exists so a notification channel cannot be pointed at your internal network.
To point a channel at a host on your own network, set `WARMBLY_ALLOW_UNSAFE_WEBHOOK_URLS=true`. That also permits plain HTTP. Only do this on a deployment where you control the network.
### Signing
A `webhook` channel with a signing secret sends:
```
X-Warmbly-Signature: t=<unix>,v1=<hex>
X-Warmbly-Event: <event key>
```
`v1` is `HMAC-SHA256(secret, "<unix>." + body)`, the same scheme as customer webhooks, so an existing verifier works unchanged. The body is:
```json
{
"event": "worker.offline",
"title": "Worker stopped responding",
"summary": "No healthy replacement of the same tier was available…",
"severity": "urgent",
"fields": [{ "label": "Worker", "value": "…" }],
"link": "https://app.example.com",
"timestamp": "2026-01-01T00:00:00Z"
}
```
### Credentials
A chat webhook URL is a bearer credential: anyone holding it can post to that channel. The admin API never returns one in full. A saved channel reads back with its URL reduced to a host and the tail of its path, and its signing secret replaced by a placeholder. Sending those values back unchanged means "keep what is stored", so saving an unrelated toggle cannot wipe a credential. To change a URL or secret, type the new value in full.
## Events
| Event | Fires when |
| --- | --- |
| Enterprise inquiry submitted | Someone asks for enterprise pricing from the plan chooser |
| Limit increase requested | A workspace asks for more capacity than its plan allows |
| Payment failed | Stripe could not collect an invoice |
| Workspace created | A new organization is created |
| User registered | A new account finishes signing up |
| Worker went offline | A worker stops heartbeating and no healthy replacement of its tier is available |
| Warmup ban appealed | A blocked mailbox asks to rejoin the warmup pool |
| Workspace risk escalated | Risk scoring moves a workspace into a different posture |
A self-hosted instance is offered the subset that means something without billing, so the panel never lists an event that cannot fire there.
Worker and risk alerts are deduplicated: a worker that stays down does not re-alert for six hours, and a risk alert only fires on an actual change of posture.
## Delivery behaviour
Delivery is best effort and deliberately so. An alert never blocks or fails the request that produced it, and a chat server that is slow or unreachable is not allowed to slow the product down. There is no retry queue. If a channel is down when an event fires, that alert is lost.
If alerts stop arriving, use **Test** first: it is synchronous and tells you exactly what the transport said.
@@ -64,6 +64,7 @@ Newer builds return the invite-only refusal with its own machine code, `registra
|---|---|
| `no space left on device`, often from a random service mid-compile | Docker is out of disk. Free space with `docker builder prune -af` and `docker image prune -af`, check the host has about 10 GB free, then re-run `make up` |
| `failed to authorize: ... EOF` while pulling a base image | A transient registry blip. Re-run `make up`; completed layers are cached |
| `ERR_PNPM_ABORTED_REMOVE_MODULES_DIR_NO_TTY` while building the `forms` image | A `forms/node_modules` from a native `make forms` or `make dev` was shipped into the Docker build context and overwrote the image's own install, and pnpm will not recreate it without a terminal. Current checkouts keep every `node_modules` out of the context and run pnpm in CI mode; on an older checkout, `rm -rf forms/node_modules` and re-run `make up` |
| `failed to xattr /path/._something: operation not permitted` on macOS | The checkout is on a filesystem without native extended attributes (exFAT, NTFS or a network share), so macOS writes `._*` sidecar files that BuildKit cannot read. Run `dot_clean -m .` then `find . -name '._*' -delete` and re-run. Cloning to an APFS volume avoids it |
## The stack is up but something is wrong
+195
View File
@@ -0,0 +1,195 @@
---
title: Updates
description: How a self-hosted instance learns about a new Warmbly version, what the indicator in the admin panel means, and how Update and restart pulls, rebuilds and restarts the stack for you.
---
A self-hosted Warmbly checks for a newer version on its own and shows it in the admin panel. Applying it is one button when the updater runs next to the stack: the new version is fetched, the services that changed are recreated, and the panel reconnects once the backend answers again. Nothing in flight is lost: sending and syncing pause for the restart and resume from the database on the other side.
How "fetched" happens depends on how the instance was installed, and the updater has a mode for each:
| Mode | For | What an update is |
|---|---|---|
| `image` | An install from [`install.sh`](/development/install/) | Pin the new tag in `.env`, `docker compose pull`, recreate. Nothing is built |
| `compose` | A git clone | `git pull`, `docker compose build`, recreate |
| `command` | Anything else | `git pull`, then a build-and-restart script of your own |
## The indicator
The top bar of the **admin panel** (`:5174`) carries a version pill next to the environment label. It reads the running backend's build identity, so it shows exactly what is deployed, not what the checkout says.
| Pill | Means |
|---|---|
| `v1.4.0` (grey) | Up to date. Click it for the details and a manual check |
| `Update to v1.5.0` (amber) | A newer release exists on the configured channel |
| `Update available` (amber) | The checkout is behind its branch (an install that tracks `main`) |
| `Updating` (blue, spinning) | An update job is running; the pill follows it from any tab |
| `Restarting` (blue, spinning) | The backend is coming back after an update |
The same facts sit at the top of **Instance > Setup and health**, and an available update is also a finding there (`update_available`, info severity), so a page that only lists findings still tells you.
## In the dashboard
Every member of a self-hosted workspace sees the same version pill in the **dashboard** header, next to the plan badge: the running version in grey, or "Update to vX.Y.Z" in amber with a pulsing dot once a newer release exists. It is there so nobody has to ask which version the server runs, and so the people who cannot update still know one is waiting.
Who can act on it follows platform admin access, not workspace roles:
- **Members** see a badge. Its tooltip names the version and says to ask a platform admin.
- **Platform admins** click it and get the update dialog: the running and available versions with the release notes, the checkout and updater state, a "Check now" button, and **Update and restart**. That button leads to a confirmation pane that spells out what the update does (pull, rebuild and restart, sending pauses and resumes, migrations apply, the tab reconnects) before anything runs.
While the update runs the dialog shows a progress bar, the step list with the live step highlighted, and the log behind a toggle. When the backend goes away for the restart the dialog says it is reconnecting and keeps polling; the pill in the header turns into a spinner so the job stays visible with the dialog closed, and a reload picks it back up. When the new backend answers, the dialog shows the result and reloads the dashboard after a short countdown, or, if the dialog was closed, a toast reports the new version and every list refreshes.
The dashboard reads `GET /auth/instance` for the version (any member) and the admin endpoints below for the rest (platform admins only, the same permission gates as the admin panel).
## What counts as newer
Two signals, and either one lights the pill:
- **A release.** The newest release of `RELEASES_GITHUB_REPO` on `UPDATE_CHANNEL` (`stable`, or `dev` to include prereleases) is compared with the running build's version. A build stamped `v1.4.0-3-gabc1234` (three commits past the `v1.4.0` tag) is treated as newer than `v1.4.0` and older than `v1.4.1`, so an instance built from `main` is not nagged about the release it already contains.
- **Commits.** When the updater runs, it fetches the remote on `UPDATER_FETCH_INTERVAL` and reports how far the checkout is behind its branch. Any distance counts as an update on an install that tracks a branch.
A build that carries no version (an image built without the build arguments, reported as `dev`) cannot be compared with a release, and only the commit distance applies.
The version comes from the binary itself: the Dockerfiles and `make up` stamp the tag, commit and build time in (`internal/version`), CI does the same for published images, and `warmblyctl status` prints it as the first line.
## Update and restart
The button appears when the updater is reachable and something newer exists. It needs the `manage_settings` admin permission, and every press is an audit row (`upgrade` on `instance`).
On an install from `install.sh`, in image mode:
1. **Pin release.** The chosen tag is written into `WARMBLY_TAG` in the install's `.env`, so the pull, the recreate, and every later `docker compose up` you run by hand all resolve the same images. A pull that fails puts the old tag back rather than leaving `.env` pointing at images that do not exist.
2. **Pull images.** `docker compose pull`.
3. **Restart.** `docker compose up -d --no-build` for every service, minus the updater itself.
4. **Clean up.** `docker image prune -f`, unless `UPDATER_PRUNE=false`.
5. **Wait for backend.** The updater polls `/health` until it answers, for up to six minutes.
On a git checkout, in compose mode:
1. **Fetch.** `git fetch --tags --prune` on the checkout.
2. **Pull.** On a branch, a fast-forward merge of the remote branch. On a checkout pinned to a tag, a checkout of the newest release. A checkout with local modifications is refused rather than merged over; commit or stash them, or set `UPDATER_ALLOW_DIRTY=true`.
3. **Build.** `docker compose build` for every service, with the version stamped in.
4. **Restart.** `docker compose up -d --no-build` for every service the checkout defines plus everything running, minus the updater itself. Compose recreates only the containers whose image or configuration changed, so Postgres, Redis and NATS stay up, and the backend applies migrations as it boots.
5. **Clean up.** `docker image prune -f`, unless `UPDATER_PRUNE=false`.
6. **Wait for backend.** The updater polls `/health` until it answers, for up to six minutes, and marks the job as succeeded or failed.
If the updater's own image changed, it recreates itself last, after the outcome is already on disk, so the new updater reports the finished job.
The dialog streams the log. Close it and the pill keeps following the job; reload the page and the pill picks the job up again. When the backend answers with a new build, a toast says which version is now running and every list refreshes.
Migrations are forward-only, so an update never needs to be rolled back to be safe with data in place. Read [backing up](/development/deployment-guide/#backing-up) anyway: a backup before an update is the one you are glad to have.
<Callout type="warn" title="The updater holds the docker socket">
The updater mounts `/var/run/docker.sock`, which is root on the host. That is why on a clone it lives behind the `updater` compose profile, and why in every install it answers only to a bearer token the backend holds, on the compose network, with no published port. The backend gates the button on an admin permission and audits it. If any of that is more trust than the host should extend, leave the profile off and update by hand.
</Callout>
## Enabling the updater
### An install from install.sh
Already done. The installer writes the updater into the generated compose file in image mode, pointed at the install directory, sharing `INTERNAL_API_TOKEN` with the backend. Nothing to configure.
To update by hand instead, from the install directory:
```bash
docker compose -p warmbly pull && docker compose -p warmbly up -d
```
To leave the button out entirely, delete the `updater` service from `docker-compose.yml` and set `UPDATER_URL=none` in `.env`.
### Docker Compose from a clone
`make up` starts the stack with the `updater` profile, so a stock install already has the button. For a plain `docker compose up`, put this in `.env`:
```bash
COMPOSE_PROFILES=updater
```
The service mounts the checkout at its own host path (`WARMBLY_REPO_DIR`, `$PWD` by default) so the compose file's relative paths resolve the same way inside the container, and shares `INTERNAL_API_TOKEN` with the backend as `UPDATER_TOKEN`. Nothing else to configure.
Files git writes as root are given back to the checkout's owner after each pull, so your own `git pull` on the host keeps working.
To update by hand instead: `make upgrade` (a fast-forward pull, then `make up`). To leave the button out entirely, start with `make up UPDATER=false` and set `UPDATER_URL=none` in `.env`; the panel then only reports.
### Without Docker
On a [bare-metal install](/development/bare-metal/) the updater is a systemd unit that runs `scripts/upgrade-bare-metal.sh` after pulling. The script builds every artifact this host runs as the checkout's owner, then hands off to one privileged installer, `warmbly-install-release`, which copies the artifacts into `/opt/warmbly`, keeps each frontend's `config.js`, restarts the backend first and the rest once it answers.
```bash
cd /opt/warmbly/src
go build -o out/updater ./cmd/updater && sudo install -m 0755 out/updater /opt/warmbly/bin/
sudo install -o root -g root -m 0755 deploy/systemd/warmbly-install-release.sh /usr/local/sbin/warmbly-install-release
sudo install -m 0644 deploy/systemd/warmbly-updater.service /etc/systemd/system/
printf 'UPDATER_TOKEN=%s\n' "$(grep ^INTERNAL_API_TOKEN= /etc/warmbly/warmbly.env | cut -d= -f2-)" | sudo tee /etc/warmbly/updater.env >/dev/null
sudo chmod 0600 /etc/warmbly/updater.env
sudo systemctl daemon-reload && sudo systemctl enable --now warmbly-updater
```
The unit runs as the user who owns the checkout (`deploy` as shipped; edit it). The only root step is the installer, so that user needs exactly one sudoers line, and nothing broader:
```bash
echo "deploy ALL=(root) NOPASSWD: /usr/local/sbin/warmbly-install-release" | sudo tee /etc/sudoers.d/warmbly-upgrade >/dev/null
sudo chmod 0440 /etc/sudoers.d/warmbly-upgrade
```
The installer is root-owned and not writable by that user, takes no arguments, reads only from fixed paths under the checkout and refuses symlinks there, so owning the checkout does not become owning the host. The binaries it installs run as the unprivileged `warmbly` user either way.
Then point the backend at the updater in `warmbly.env` and restart it:
```bash
UPDATER_URL=http://127.0.0.1:8095
```
The same script works by hand: `scripts/upgrade-bare-metal.sh --pull`.
## Configuration
Backend:
| Variable | What it does | Default |
|---|---|---|
| `UPDATE_CHECK_ENABLED` | Polls GitHub Releases and shows a newer version in the panel | `true` |
| `UPDATE_CHECK_INTERVAL` | How often the release check runs; minimum `5m` | `30m` |
| `UPDATE_CHANNEL` | `stable` follows releases; `dev` also offers prereleases | `stable` |
| `RELEASES_GITHUB_REPO` | The `owner/repo` whose releases are Warmbly versions. Point a fork's instance at the fork | `warmbly/warmbly` |
| `RELEASES_GITHUB_TOKEN` | Optional; only raises the GitHub API rate limit | unset |
| `UPDATER_URL` | The updater. Unset, or `none`, leaves the panel report-only | `http://updater:8095` under compose |
| `UPDATER_TOKEN` | Bearer token presented to the updater | `INTERNAL_API_TOKEN` |
Updater:
| Variable | What it does | Default |
|---|---|---|
| `UPDATER_TOKEN` | The token it accepts; falls back to `INTERNAL_API_TOKEN`. Refuses to start without one | unset |
| `UPDATER_MODE` | `image` pins a release tag and pulls; `compose` rebuilds a checkout; `command` runs `UPDATER_COMMAND` | `compose` |
| `UPDATER_COMMAND` | The build-and-restart command for `command` mode | unset |
| `UPDATER_REPO_DIR` | The checkout, or in image mode the install directory holding the compose file and `.env` | working directory |
| `UPDATER_REMOTE` | The git remote to fetch and pull from | `origin` |
| `UPDATER_COMPOSE_PROJECT` | The `-p` the stack was started with | `warmbly` |
| `UPDATER_COMPOSE_PROFILES` | Extra profiles on every compose call, so its own image is rebuilt | `updater` |
| `UPDATER_BACKEND_HEALTH_URL` | Polled after the restart until it answers | `http://backend:8080/health` |
| `UPDATER_FETCH_INTERVAL` | How often it fetches for the commits-behind count | `30m` |
| `UPDATER_PRUNE` | `docker image prune -f` after a successful update | `true` |
| `UPDATER_ALLOW_DIRTY` | Update over local modifications in the checkout | `false` |
| `UPDATER_STATE_DIR` | Where the last job is kept across restarts | `/var/lib/warmbly-updater` |
| `UPDATER_ADDR` | Listen address | `:8095` |
## Endpoints
| Endpoint | Auth | Returns |
|---|---|---|
| `GET /admin/instance/update` | platform admin, `view_analytics` | Running build, latest release, whether an update is available, updater and job state. `?log=1` includes the job log |
| `POST /admin/instance/update/check` | platform admin, `manage_settings` | Runs both checks now and returns the state |
| `POST /admin/instance/update/apply` | platform admin, `manage_settings` | Starts an update job (`{"target": "latest"}` or a release tag) and returns it. The backend restarts as part of it |
The updater's own API (`GET /status`, `POST /check`, `POST /update`) is bearer-authenticated and meant for the backend only.
## Findings
Two [instance health](/development/instance-health/) checks belong to this page:
- `update_available` (info): a newer version exists, with the version and how to apply it.
- `updater_unreachable` (warning): `UPDATER_URL` is set but nothing answers there, so the button cannot work. Start the updater, fix the address, or remove the variable to update by hand.
## Workers on other machines
This page is about the control plane. Remote workers installed from the panel keep their own daily self-update timer and the **Pull latest and restart** action on the worker's page; see [day-2 operations](/development/deployment-guide/#day-2-operations).
+67 -2
View File
@@ -3,6 +3,12 @@ title: warmblyctl
description: The CLI for a Warmbly instance. The operator commands for accounts, health and recovery, and the API commands that let scripts and AI agents drive campaigns, contacts, mailboxes and the inbox with an API key.
---
<Callout type="info" title="There are two CLIs. This is the operator's one.">
`warmblyctl` is for running an instance: accounts, health, recovery, backups. It reads the database directly and is meant to be run inside the backend container.
If you want to use the product from your terminal (campaigns, contacts, mailboxes, the inbox), you want [`warmbly`](/api/cli/): a signed-in, multi-host client you install on your own machine, with `warmbly auth login` instead of an exported key.
</Callout>
`warmblyctl` is the CLI for a Warmbly instance, and it has two halves with two trust models.
The operator commands answer two questions: what state is this install in, and how do I get back in. They read and write the database directly, so they keep working when the sign-in page does not. Their authorization is container or host access, the same trust model as Sentry's `createuser`, Gitea's `admin user create` and authentik's `ak changepassword`, and it is the right one when the identity system is the thing that is broken.
@@ -11,6 +17,8 @@ The [API commands](#the-api-commands) drive a running instance over its public R
The CLI never serves HTTP. The API commands are a client of the already-gated public API, which adds no new surface to your instance.
Those API commands predate the [`warmbly` CLI](/api/cli/) and still work exactly as documented below. For day-to-day product work prefer `warmbly`: it signs in for you, holds a credential per host, prints tables rather than raw JSON, and runs on your laptop rather than inside a container.
## Running it
The binary ships inside the backend image at `/usr/local/bin/warmblyctl`, so it is on the path in every runtime. Running it inside the backend is the documented path because the environment there is already correct.
@@ -55,10 +63,12 @@ A command that would set a password refuses on a non-TTY unless you passed `--pa
| `APP_URL` | every printed link and sign-in hint | Links are built against `https://app.warmbly.com`, which is the hosted service and not your instance |
| `KMS_PROVIDER` and its key | `org export`, `org import` | The command stops: a workspace's sealed values cannot be opened, so an archive would be useless |
| `CREDENTIALS_ENCRYPTION_KEY` | `org export`, `org import` | A warning, and mailbox credentials are neither read nor written. Everything else still moves |
| `CREDENTIALS_ENCRYPTION_KEY`, `KMS_LOCAL_MASTER_KEY` | `backup`, `restore` | `backup` warns and leaves them out of the bundle. `restore` refuses when this host's differ from the bundle's |
| `BLOB_FS_ROOT` | `backup`, `restore` | Defaults to `/data/blobs`, which is where the compose stack mounts it |
| `WARMBLY_API_KEY` | every API command | The command stops and explains where a key comes from |
| `WARMBLY_API_URL` | every API command | Falls back to the instance's own `API_PUBLIC_URL`, then the hosted service |
Inside the backend container all of these are already set, except the API key, which is yours. Outside it, export what the command needs, and match `AUTH_SECRET` to the backend's exactly.
`backup` and `restore` also need `pg_dump` and `psql`, which the backend image ships for exactly this reason. Inside the backend container all of these are already set, except the API key, which is yours. Outside it, export what the command needs, and match `AUTH_SECRET` to the backend's exactly.
## The operator commands
@@ -73,6 +83,8 @@ Inside the backend container all of these are already set, except the API key, w
| [`user revoke-admin`](#user-revoke-admin) | Takes platform admin away from an account |
| [`user disable-2fa`](#user-disable-2fa) | Clears an account's authenticator enrolment |
| [`hash-password`](#hash-password) | Prints an argon2 hash for unattended provisioning |
| [`backup`](#backup) | Writes the whole instance to one restorable bundle |
| [`restore`](#restore) | Restores a bundle onto this instance, replacing everything on it |
| [`org list`](#org-list) | Lists the workspaces on this instance with their id, owner, and size |
| [`org export`](#org-export) | Writes a whole workspace to a portable archive file |
| [`org import`](#org-import) | Applies an archive to a workspace on this instance |
@@ -282,6 +294,57 @@ It prompts when a terminal is attached and reads the pipe when one is not, so th
An argon2 PHC string contains `$` characters. Docker Compose reads those as interpolation, so a bare hash in `.env` silently loses part of itself. Wrap it in single quotes there, and double every `$` if you paste it into `docker-compose.yml` directly.
</Callout>
## backup
```bash
docker compose -p warmbly exec backend warmblyctl backup --out /data/blobs/warmbly.tar.gz
docker compose -p warmbly cp backend:/data/blobs/warmbly.tar.gz ./warmbly.tar.gz \
&& docker compose -p warmbly exec -T backend rm -f /data/blobs/warmbly.tar.gz
```
`/data/blobs` is the one path the container and the host both see, so it is the hand-off. Move the bundle out and delete it: `backup` excludes its own output from the archive, but a bundle left behind is swept into the next one. The delete is chained with `&&` so a failed copy cannot take the only copy with it.
Writes one bundle holding the three things that only restore together: a `pg_dump` of the database, the blob root (message bodies, attachments, avatars), and the encryption keys.
That combination is the whole reason this is a command rather than a documented list of steps. A dump alone restores an instance whose every mailbox credential decrypts to nothing, because the ciphertext is in the database and the key that opens it is in `.env`. The blob root alone restores bodies nothing points at.
| Flag | Does |
|---|---|
| `--out` | Where to write it. Defaults to `warmbly-backup-<timestamp>.tar.gz` in the working directory |
| `--no-keys` | Leaves the encryption keys out. The bundle is then not restorable on its own |
| `--no-blobs` | Database only |
| `--force` | Overwrite an existing output file |
The bundle is written 0600 and holds every mailbox credential on the instance plus the keys that open them. Treat the file as you would the instance itself: encrypted at rest, off the host, and not in a shared drive.
Blobs travel only on the filesystem provider. An instance storing them in S3 keeps them in the bucket, and the bundle says so rather than pretending to be complete.
<Callout title="Schedule it">
[`install.sh --wizard`](/development/install/) writes a `backup.sh` and a systemd timer that runs exactly this, keeps the last N bundles and can copy each one off the host.
</Callout>
## restore
```bash
docker compose -p warmbly exec backend warmblyctl restore --file /data/blobs/warmbly.tar.gz
docker compose -p warmbly restart
```
Empties the schema and replays the bundle into it, so it replaces every organization, user, campaign and mailbox currently on the instance. It prints what the bundle holds and asks you to type `restore` before it does; `--yes` skips that for scripts.
Before anything is written it compares the bundle's `CREDENTIALS_ENCRYPTION_KEY` and `KMS_LOCAL_MASTER_KEY` against this host's, and refuses to continue when they differ, printing the two lines to put in `.env` first. That check is the point of the command: without it a restore looks like it worked, and every mailbox fails to authenticate days later with no error that names the cause.
| Flag | Does |
|---|---|
| `--file` | The bundle. Required |
| `--yes` | Skip the typed confirmation |
| `--no-blobs` | Restore the database only, leaving the blob root alone |
| `--force` | Restore even though the keys differ. Accepts losing every stored mailbox credential |
Blobs are unpacked over the blob root additively: files the bundle knows about are overwritten, anything else is left alone.
[Data control](/development/data-control/#moving-an-instance) walks the whole move to a new host.
## org list
```bash
@@ -344,6 +407,8 @@ Run it with `--dry-run` first. The report is the same preflight the dashboard sh
The whole import runs in one transaction: if any part fails, nothing lands and the workspace is untouched. Members are matched to accounts on this instance by email address, and anyone without one has their rows reassigned to the workspace owner, named in the report before you commit. An archive carries no password material, so it can never create an account here.
`org export` and `org import` are the per-workspace tool and [`backup`](#backup) is the instance-level one. They are not interchangeable: a bundle cannot be applied to one workspace, and a workspace archive cannot restore an instance.
Billing history, plan overrides, worker placement, mailbox sync checkpoints and warmup pool membership are exported for the record but never applied: each belongs to the instance rather than to the workspace. [Export and import](/guides/workspace-export-import/) has the full table.
## The API commands
@@ -405,7 +470,7 @@ Paths are relative to `/v1`. `--data` takes a JSON literal, `-` for stdin, or `@
### For AI agents
The repository ships skills under `skills/` (`warmbly-api` for the product, `warmbly-ops` for instance administration) that teach a coding agent these commands, the conventions above, and the sending-safety rules. Install them the way your agent expects, for example `cp -r skills/warmbly-api ~/.claude/skills/` for Claude Code, or point the agent at the `SKILL.md` directly. An agent given a scoped key and those skills can operate a workspace end to end without touching the database.
The repository ships skills under `skills/` (`warmbly-cli` for the [`warmbly` CLI](/api/cli/), `warmbly-api` for the same product surface through `warmblyctl`, `warmbly-ops` for instance administration, `warmbly-install` for standing an instance up and moving it) that teach a coding agent these commands, the conventions above, and the sending-safety rules. Install them the way your agent expects, for example `cp -r skills/warmbly-api ~/.claude/skills/` for Claude Code, or point the agent at the `SKILL.md` directly. An agent given a scoped key and those skills can operate a workspace end to end without touching the database.
## When Redis is down
+12 -2
View File
@@ -20,8 +20,10 @@ Four rules govern the counts:
- **Tracking must be on.** Opens and clicks need open or link tracking enabled on the campaign, and a tracking host on the install. A [custom tracking domain](/guides/mailboxes/#custom-tracking-domain) per mailbox is optional; without one they go through the shared host and still count. With tracking off they stay at zero while sends and replies still count.
- **Replies are human replies.** Out-of-office and autoresponders never count, never stamp the contact as replied, and never trip stop-on-reply.
- **Bots are filtered.** Crawlers, CLI agents, prefetches, chat link previews, and security gateways that open every link are served normally but never counted. Otherwise one corporate scanner would "click" every link seconds after delivery.
- **Auto-opens are labeled, not hidden.** Privacy proxies like Apple Mail Privacy Protection still count (they confirm delivery) but are tagged and shown separately (`12 auto`). A later real open upgrades them to human. **Auto-opens never trigger opened-based branches or automations.**
- **Bots are filtered.** Crawlers, CLI agents, prefetches, chat link previews, and security gateways that announce themselves are served normally but never counted. The ones that do not announce themselves are caught by what they do: an open or click inside ten seconds of the send (nobody reads that fast; the clock starts when the send is handed to the worker, before the mail has even been delivered) and clicks on two or more links of one email from the same source within five seconds (a scanner walking the message). Otherwise one corporate scanner would "click" every link seconds after delivery.
- **Auto-opens are labeled, not hidden.** Privacy proxies like Apple Mail Privacy Protection, and instant opens, still count (they confirm delivery) but are tagged and shown separately (`12 auto`). A later real open upgrades them to human. **Auto-opens never trigger opened-based branches or automations.**
- **A click is an open.** Opens need the mail client to load images, and many clients block them. A click by the person proves the email was read, so it counts as an open as well, and a lead can never read "clicked, not opened".
- **Auto-clicks are kept but never counted.** A click classified as automated is logged on the contact's activity with the link it hit and the rule that caught it, and the campaign overview shows how many steps had only automated clicks (`3 auto`). It never makes the step clicked, never fires a clicked branch or automation, and never sends a webhook. A person clicking the same link later counts normally. A burst is only recognisable from its second click, so a click that looks human waits for the burst window plus one second (six seconds) before it fires anything (clicked branches, automations, webhooks, the live feed): if a second link follows inside that window, both clicks are relabeled, the stamp is withdrawn and nothing fires.
## Workspace dashboard
@@ -42,6 +44,14 @@ Each campaign reports its own totals: contacts, sent, pending, unique opens and
**Per-step stats** break sent, opens, clicks, replies, and bounces down by sequence step, which is how you find the touch pulling replies and the follow-ups that are dead weight. Daily stats drive the trend chart, and hourly stats show when sends land and get engagement. Campaigns can also be compared side by side on the same metrics.
**Who engaged, and from where** lists the countries, mail clients and devices people opened and clicked from, counting each contact once per bucket. Only human events feed it, so a security scanner's data centre never tops the country list.
### What is recorded per open and click
Every open and every click is kept as its own record, repeats included, with what the request said about itself: the mail client or image proxy when the user agent names one (Gmail, Apple Mail, Outlook), otherwise the browser, operating system and device type; the country, region and city resolved from the source network; for clicks the exact link; and whether it was the person or an automated fetch, with the rule that caught it. The address itself never leaves the tracking service: it publishes the network the address belongs to (the last IPv4 octet zeroed, an IPv6 address cut to its first 48 bits), which is enough for a city-level lookup and cannot single out a host, plus a keyed token of the full address used to tell one source from another, which cannot be turned back into the address. The consumer resolves the network to a location and stores neither. Location needs the GeoLite2 database on the consumer (`GEODB_PATH`); without it the country, region and city stay empty and everything else is still recorded. These records are kept for a year, then pruned; the counts and the first open and click per step live on the lead itself and are not affected.
These records back the engagement rows on a contact's [activity timeline](/guides/contacts-crm/#activity-timeline) and the per-campaign breakdown above.
## Per-mailbox
| Signal | What it tells you |
+26
View File
@@ -34,6 +34,8 @@ Triggers, conditions, and actions only take effect after **Save** (card position
| Trigger | Fires when |
| --- | --- |
| Reply received | A contact replies to a campaign email |
| Contact created | A new contact is added by hand, through the API, by a form or by an automation (bulk file imports and sheet syncs stay quiet, see below) |
| Form submitted | A hosted [form](/guides/forms/) receives a submission |
| Meeting booked / rescheduled / canceled | A Calendly or Cal.com meeting changes |
| Email bounced | A campaign email bounces |
| Unsubscribed | A contact unsubscribes |
@@ -46,6 +48,8 @@ Each trigger carries its own event data (a reply carries contact email, reply in
**Campaign action** never fires by itself; it runs only from a campaign step, still evaluating your conditions.
**Contact created** fires once per new contact, with the contact's fields and its first-touch source, so a condition can branch on `source` (a form, the API, an automation) or on any field. A contact that already existed and was only updated does not fire it. Bulk arrivals are deliberately silent: a file import or a Google Sheets sync of ten thousand rows would otherwise run ten thousand flows and flood every webhook, so those paths never raise it, and neither does a single API request adding more than 100 contacts.
### Inbound webhook
Pick the trigger and **save**, and the editor shows a unique URL. Anything that can send HTTP `POST`s JSON to it.
@@ -93,6 +97,8 @@ Slack, Discord, and webhook actions take an optional message template. Slack and
| Built-in action | What it does |
| --- | --- |
| Create or update contact | Makes a contact from the event's fields, or enriches the one with that email, then tags it and enrols it in a campaign. See [Lead intake](#lead-intake) |
| Add to campaign | Enrols the event's contact in a campaign; a finished campaign restarts through the usual launch checks (a refusal is noted in its activity log and the lead waits), one waiting for leads picks up straight away |
| Add / remove a tag | Adds or removes a contact category |
| Label the email | Applies inbox labels to the replied-on conversation |
| Create a task | Assigned to the workspace owner |
@@ -105,6 +111,26 @@ Slack, Discord, and webhook actions take an optional message template. Slack and
Three constraints worth knowing: **Move the deal stage** does nothing if the contact has no open deal in that pipeline, **Unsubscribe** needs an event carrying a campaign, and **Label the email** works only on a Reply received automation, since it needs a thread to label.
**Add to campaign** and every other contact action need a contact to act on: the event must carry `contact_id` or `contact_email`, which every Warmbly trigger does. On an inbound webhook, put **Create or update contact** first and the contact it writes becomes the event's contact for the steps after it.
### Lead intake
Any system that can send an HTTP request can create contacts in Warmbly without an API client: an **Inbound webhook** trigger followed by a **Create or update contact** action.
1. Pick the Inbound webhook trigger, save, and copy the URL.
2. Add **Create or update contact**. Each field is a template over the JSON the caller sends: an email of `{{.email}}`, a first name of `{{.first_name}}`, a custom field `team_size` set to `{{.answers.team_size}}`. Pick the tags and the campaign the lead lands in.
3. Point the sender at the URL. Zapier's *Webhooks by Zapier*, Make's *HTTP* module, n8n's *HTTP Request* node, a Typeform or Tally webhook, or your own code.
The action matches an existing contact by email, so re-sending the same lead updates it instead of duplicating it. A blank rendered value never erases a field the contact already has. **If the contact already exists** decides whether an existing contact is enriched and enrolled (the default) or left alone. New contacts carry `automation` as their first-touch source, with the automation's name as the detail.
<Callout type="info" title="Facebook, Instagram, LinkedIn and TikTok lead forms">
Meta, LinkedIn and TikTok deliver lead-form submissions to Zapier, Make and n8n in real time. Either route them through the Warmbly app there with its **Create or Update Contact** and **Add to Campaign** actions, or send them to an inbound webhook automation as above. Both end the same way: a contact with the ad's answers as custom fields, tagged and enrolled in the campaign that follows up from your mailbox. See [Zapier](/guides/zapier/#lead-forms-and-other-lead-sources), [Make](/guides/make/#lead-forms-and-other-lead-sources) and [n8n](/guides/n8n/).
</Callout>
Lead-form campaigns produce leads in bursts, and the follow-up still goes out from your mailboxes under their daily caps and spacing. A day of two hundred leads on one mailbox capped at fifty is a four-day send; the campaign's recipients-versus-capacity estimate shows it. Add mailboxes rather than raising caps.
A flow that creates a contact can be the reason a **Contact created** flow runs. That is allowed, and bounded: an event raised by an automation's own action carries how many hops led to it, and after five hops no further automation runs, so two flows cannot feed each other forever.
**Fire event** is the inverse of the inbound webhook: your app subscribes over the websocket with an API key holding `REALTIME_SUBSCRIBE` and receives `{ name, payload }`, so you get events without hosting a public URL. See [Realtime events](/api/realtime/).
### The on-error branch
+54
View File
@@ -0,0 +1,54 @@
---
title: Plans and billing
description: How plans unlock features, how to upgrade from anywhere in the dashboard, and what changes when you switch.
---
Every hosted workspace starts free. The free workspace warms up to 10 mailboxes and links self-hosted instances. Sending from Warmbly Cloud, the unified inbox, contacts, the CRM, automations and integrations turn on with a paid plan.
## Plans
| Plan | Monthly | Annual (per month) | Sends per day | Includes |
| --- | --- | --- | --- | --- |
| Starter | $29 | $23 | 150 | Unlimited warmup, unlimited mailboxes, unified inbox, team invitations, bulk contact operations |
| Grow | $89 | $71 | 3,000 | Everything in Starter |
| Business | $329 | $263 | 15,000 | Everything in Grow, sending kept apart from other customers, webhooks, advanced outreach |
| Enterprise | Custom | Custom | 15,000+ | Everything in Business, dedicated support |
Annual billing is 20% off the monthly price. The same lineup is on the [pricing page](https://warmbly.com/pricing).
Unlimited mailboxes means exactly that, under a fair-use allowance of one mailbox for every send a day the plan includes: Grow holds `3,000`, Business `15,000`, and either can be raised on request. See [mailbox allowance](/guides/mailboxes/#mailbox-allowance).
## Upgrading from a locked feature
When you open something your plan does not include, the dashboard opens a full-screen plan chooser instead of sending you to settings. It names the feature, highlights the plan that unlocks it, and lets you pick monthly or annual billing.
- **No subscription yet.** Choosing a plan opens Stripe Checkout. When you come back, you land on the page you left with the feature unlocked.
- **Already on a paid plan.** Choosing a higher plan switches you in place. The change is prorated and takes effect immediately.
- **Enterprise.** Opens the billing portal so you can reach us about custom volume.
Promo codes can be applied in the chooser before checkout.
Only the workspace owner can change the plan. Other members see the same comparison with a note to ask the owner. Which roles can do what is covered in [Team and roles](/guides/team-roles/).
## Managing your subscription
**Settings > Billing** is owner only and splits into four tabs, each with its own link.
**Overview** is the control panel for the subscription.
- The plan card shows the plan, its status, the price, and the renewal or end date. Change plan opens the same chooser the locked features use.
- Cancel plan schedules the cancellation for the end of the paid period. The plan stays fully active until then, and Resume clears it with one click. Nothing is lost either way: the workspace returns to the free tier and keeps its mailboxes, warmup and settings.
- Usage and limits shows live counts against the limits the server actually enforces, for mailboxes, sends today, contacts, campaigns, team members, attachment storage, sends this period, and API calls. A meter turns amber past 70% and red past 90%. A meter with no cap means that limit is unmetered on your plan. The mailbox meter says where its number comes from (fair use, an approved request, or the free allowance), and the storage meter says so when a plan change left the workspace over its quota: existing attachments keep sending, and new uploads wait until you are back under.
- Banners appear only when something needs a decision: a failed payment, a scheduled cancellation, or a trial about to end.
Need a single limit raised without moving plan? Request an increase from **Settings > Limits**, linked from the usage section, or straight from the connect dialog when it is mailboxes you need. An operator reviews it and the approved limit takes effect on the workspace, including the daily send allowance the sender enforces.
**Plans** compares the lineup as full plan cards, with the best value plan highlighted. On a paid plan each card also prices the switch before you commit, showing what is due today and when the next bill lands. Promo codes and the codes this workspace has redeemed live here too.
**AI and credits** covers the credit balance, top-ups and usage. See [AI credits](/guides/ai-credits/).
**Payment** links out to the Stripe billing portal. Cards and the billing email are held by Stripe and never touch Warmbly, so they are read and changed there. Invoices and PDF receipts live in the portal too.
## Self-hosted deployments
A self-hosted instance running without a billing provider has every feature unlocked and no billing page. See [Warmbly Cloud](/guides/warmbly-cloud/) for how a self-hosted instance links to a hosted workspace.
+46 -5
View File
@@ -11,10 +11,24 @@ You need at least one active mailbox and a contact list. Warm new mailboxes befo
## Create a campaign
**Campaigns** > **New campaign** opens a four-step wizard: **Basics** (name, description), **Schedule** (timezone, sending days, hours), **Sending** (mailbox tags, daily limit per mailbox, stop on reply, open and click tracking, unsubscribe header), and **First email** (subject, body, optional follow-ups). Follow-ups added here are connected in order, each waiting the number of days you set after the previous step; open the Steps tab to rearrange, branch, or change the waits. Only a name is required; the first email can be skipped and written in the full editor on the Steps tab, and everything else can be changed later from settings. Closing a wizard with unsaved edits asks before discarding them.
**Campaigns** > **New campaign** opens a wizard that first asks what you are sending: a **sequence** or a [one-time email](#one-time-emails). For a sequence the steps are **Basics** (name, description), **Schedule** (timezone, sending days, hours), **Sending** (mailbox tags, daily limit per mailbox, stop on reply, open and click tracking, UTM parameters, unsubscribe header), and **First email** (subject, body, optional follow-ups). Follow-ups added here are connected in order, each waiting the number of days you set after the previous step; open the Steps tab to rearrange, branch, or change the waits. Only a name is required; the first email can be skipped and written in the full editor on the Steps tab, and everything else except the campaign type can be changed later from settings. The choice between a sequence and a one-time email is fixed at creation: switching means creating a new campaign. Closing a wizard with unsaved edits asks before discarding them.
The campaign starts as a **draft**. Nothing sends until you start it.
## One-time emails
A one-time email is a campaign with a single message and no follow-ups: an announcement, an update, a single ask to a segment. It is still a campaign underneath, so it sends through your mailbox pool with the same rotation, per-mailbox daily caps, spacing, suppression, tracking and analytics as a sequence. What changes is the flow and the wording.
Pick **One-time email** on the wizard's first step. The steps are then **Basics**, **Email** (subject and body, both required), **Audience** (one or more [segments](/guides/segments/); every current member becomes a lead, and a contact in several of them is emailed once), **Sending** (mailbox tags, daily limit, tracking, unsubscribe header) and **Send**. The last step chooses **Send now** or **Schedule for later** with a date and time, sets the sending days and window, and shows an estimate: how many recipients the segments resolve to, how many mailboxes will send, the pool's daily ceiling, and the day the last send is expected to land. The estimate is the earliest finish under the daily caps, not a promise. A thousand contacts on one mailbox at the default cap is a multi-week send, and the wizard says so before you confirm.
Confirming creates the campaign, links the segments (so the campaign keeps enrolling contacts who join them later, exactly like [linking a segment](/guides/segments/#linking-a-segment-to-a-campaign)), and starts it. A scheduled one waits for its date, then sends inside the window. If the start is refused, for example because the segments are empty or the list fails the [launch check](#the-launch-check-on-your-list), the email is kept as a draft and its page explains why.
In the campaigns list a one-time email carries a **One-time** badge and reads **draft**, **scheduled** (started, waiting for its date), **sending** or **sent**; the **Type** filter shows only sequences or only one-time emails. The Overview tab reports opens, clicks, replies and bounces the same way it does for a sequence. Adding a second email step to a one-time email is refused: create a sequence when you want a reminder later.
## Plain text only
**Settings** > **Plain text only** sends the campaign without an HTML part. Recipients get the plain-text body with the mailbox's plain-text signature, and because open pixels and wrapped links need HTML, open and click tracking are off for that campaign whatever their toggles say. Test emails respect it too, so what lands in your own inbox is what recipients get.
## Senders and rotation
Pick mailboxes in **Sending accounts** three ways, and the first two combine:
@@ -49,6 +63,8 @@ Warmbly is mailbox-first: safe volume is the sum of each mailbox's budget, not o
The campaign daily limit applies as a minimum against the cold cap, so it can lower a mailbox's volume but never raise it above the mailbox's own daily cap (default `50`/day).
Only an email actually handed to a sending worker counts against a mailbox's daily budget and its minimum gap. The scheduler's own wake-ups, a step it deferred because its slot was not due yet, and action or wait steps never do, so a campaign started outside its sending window still has its whole budget when the window opens. When every mailbox on a campaign has used its budget for the day, the campaign stays active and waits for the next day; the activity log notes it once per day. It also waits, rather than pausing, when every mailbox is outside its own hours, resting, or held back by warmup health.
<Callout type="warn" title="Don't push past the defaults casually">
Anything above `50`/day per cold mailbox needs positive reputation signals and a low complaint rate behind it. Adding mailboxes is safer than forcing a few to send more.
</Callout>
@@ -57,11 +73,17 @@ Anything above `50`/day per cold mailbox needs positive reputation signals and a
**Lead flow** throttles new contacts with **Max new leads per day** (`0` for unlimited, up to `1000`). At the cap, follow-ups to in-flight contacts continue and new leads resume tomorrow. You can prioritize new leads over follow-ups, and choose whether to attempt addresses verification flagged risky.
**Keep running for new leads** turns a campaign into an ongoing one: when it runs out of leads it stays **active** and shows **waiting for leads** instead of finishing, then sends the sequence to each lead as they arrive from a linked segment, a form, the API, Zapier, Make, n8n or an automation. Linking a segment turns it on; turn it off in the campaign's preferences to have the campaign finish once every lead is done. A campaign with this on can be started with no leads yet. An end date still ends it.
### Adding leads
The **Leads** tab takes contacts four ways. **From contacts** opens a picker over the workspace contact list: search by name, email or company, filter by category, tick individual people or **Select loaded**, or **Select all matching** to add everyone the search returns (up to 1,000 per batch; repeat for the rest). Contacts already in the campaign are marked as leads and skipped. **Import** runs the file import wizard with this campaign preselected (its Options step can also pin the file into segments), **Sheet sync** attaches a Google Sheet, and **Add lead** creates a single new contact in the campaign. Any workspace member with contact access can add leads to any campaign in the workspace, whoever created it.
**Segments** on the same toolbar links [segments](/guides/segments/) to the campaign as a live audience, up to 20 per campaign. Linking enrols every current member immediately, and contacts who enter a linked segment later are enrolled on their own, within a couple of minutes. An active campaign wakes to send to them, a finished one restarts through the usual launch checks, and a paused one accumulates them for later. Enrolment is additive: a contact who leaves the segment keeps their lead row, and detaching a segment stops future enrolment without touching existing leads. See [linking a segment to a campaign](/guides/segments/#linking-a-segment-to-a-campaign).
**Segments** on the same toolbar links [segments](/guides/segments/) to the campaign as a live audience, up to 20 per campaign. Linking enrols every current member immediately and turns on **Keep running for new leads**, and contacts who enter a linked segment later are enrolled on their own, within a couple of minutes. An active campaign wakes to send to them (a campaign waiting for leads picks up straight away), a finished one restarts through the usual launch checks, and a paused one accumulates them for later. Enrolment is additive: a contact who leaves the segment keeps their lead row, and detaching a segment stops future enrolment without touching existing leads. See [linking a segment to a campaign](/guides/segments/#linking-a-segment-to-a-campaign).
Every linked segment is shown in a **Linked segments** strip under the toolbar, so the audience feeding the list is always visible. Each chip names the segment and shows how many of its members are leads right now (`120/120`); clicking it filters the list to that segment's leads, and the arrow next to it opens the segment. A segment that matches no contacts says **no contacts** instead of a count, and one whose members were removed from the campaign by hand shows how many are **held out** with an **Add back** action, which re-enrols every current member and clears the removals. When a campaign has linked segments but no leads, the empty state explains why (the segments are empty, or their members were removed by hand) rather than offering to link a segment again.
**Export** on the toolbar downloads the leads as CSV, XLSX or JSON. The scope is the campaign: **Every lead** exports the whole list, **Filtered** the leads matching the current filters, **Selected** the ticked rows. The **Campaign-ready** column preset adds each lead's status and whether they opened, clicked or replied. The same export lives on a segment page for its members.
A lead can also be taken out again: the remove button on a lead's row, or **Remove from campaign** in the selection bar when several are ticked, drops the lead from this campaign without deleting the contact. The removal sticks: a linked segment will not re-enrol that contact automatically, even while they still match it, until you add them back yourself. Neither offers to delete the contact itself: inside a campaign the destructive action is leaving the campaign, and deleting a contact from the workspace is done from the contacts list.
@@ -81,7 +103,9 @@ A lead is **Processing** only while steps remain, so a finished campaign reads a
### Who opened, clicked and replied
Next to each lead's status, the Leads list shows three engagement columns: **Opened**, **Clicked** and **Replied**, each with the number of emails in the sequence the person engaged with. A dash means the lead was emailed and has not engaged; the cell is blank for a lead not emailed yet. An open counts only when a person opened the email. Mail clients that fetch every image automatically (Apple Mail Privacy Protection, for example) show as **auto** instead, the same opens the campaign overview reports as automatic, so they never pass for engagement.
Next to each lead's status, the Leads list shows three engagement columns: **Opened**, **Clicked** and **Replied**, each with the number of emails in the sequence the person engaged with. A dash means the lead was emailed and has not engaged; the cell is blank for a lead not emailed yet. An open counts when a person opened the email, or clicked a link in it. Mail clients that fetch every image automatically (Apple Mail Privacy Protection, for example) show as **auto** instead, the same opens the campaign overview reports as automatic, so they never pass for engagement. Clicks are held to the same standard: a link followed within ten seconds of the send, or several links of one email followed within a few seconds of each other, is a security gateway scanning the message, not the recipient. Those clicks are kept on the contact's activity marked **auto**, counted apart on the campaign overview, and never make a lead **Clicked**, never fire a clicked branch or automation, and never send a webhook. See [Link tracking and UTM parameters](#link-tracking-and-utm-parameters).
Opens depend on the recipient's mail client loading images. Outlook, many corporate setups and some privacy-minded clients block them, so a person can read an email and still show no open; the info icon on the column says as much. A click is stronger evidence, so a click by the person always counts as an open too. Open the contact's Activity tab to see each open and click with the mail client and location it came from.
The chips above the list are filters. Click a status chip (**Processing**, **Done**, **Replied**, **Queued**, **Bounced**, **Unsub**) or an engagement chip (**Opened**, **Not opened**, **Clicked**, **Not clicked**, **Replied**, **Not replied**) to show only those leads; click it again to clear. One status and one engagement chip can be active at once and both must match. The numbers on the chips are campaign-wide totals, and filtering happens on the server, so a scope shows every matching lead however long the list is. **Not opened**, **Not clicked** and **Not replied** only cover leads that have been sent at least one email: a queued lead has not had the chance. The same filters live in the **Filters** sheet under **Lead status** and **Engagement**.
@@ -106,6 +130,21 @@ A step counts as sent only once the sending worker has handed it to the mailbox
No contact is emailed the same step twice. Each step is recorded as attempted before the send is handed to a worker, so a crash, a restart, or a database hiccup in the moment between the two cannot make the step look unsent and send it again. The trade-off is a step whose outcome is genuinely unknown, when the worker stops responding mid-send: after 30 minutes with no answer the step is treated as a failed attempt, appears in **Needs attention**, and is retried like any other failure.
## Link tracking and UTM parameters
With **Link tracking** on, every link in the email body becomes its own tracked link, so Warmbly records which link was clicked, not only that one was. Open a contact's **Activity** tab and a click reads **Clicked Pricing**, with the campaign, step and sending mailbox on the row and, expanded, the link's text, its full URL, every UTM parameter it carried, the browser, and whether a person or a scanner followed it. The campaign's live feed names the link the same way, and the `campaign.email_clicked` webhook carries the URL and the link text.
**UTM parameters** (on by default for campaigns created in the dashboard, off for campaigns created through the API unless you send `utm_tracking`) tag every link when the email is sent, so nobody has to add them by hand and clicks show up attributed in your web analytics:
| Parameter | Value |
|-----------|-------|
| `utm_source` | `warmbly`, or the value you set on the campaign |
| `utm_medium` | `email`, or the value you set |
| `utm_campaign` | The campaign name as a slug (`q3_outbound`), or the value you set |
| `utm_content` | The link's own text as a slug (`pricing_page`); a link with no text, such as an image, is numbered `link_1`, `link_2` in order |
A link that already carries one of these keeps the hand-written value; only the missing parameters are added. Existing query strings and fragments are left exactly as written, and the tags go on the real destination, so they survive the tracking redirect. Anchors, `mailto:` and `tel:` links, and anything that is not `http(s)` are never touched. UTM tagging works with link tracking off too: the tags are added and the link is otherwise left as written. Bare URLs in the plain-text part are tagged as well, numbered `link_1`, `link_2` since they have no text. Change the source, medium and campaign values under **Settings** > **Deliverability**; leave a field empty to use the default.
## Scheduling
Campaigns send only inside the **weekly sending windows** you define, in the campaign's timezone. Each day is independent, with different hours or several windows per day, set from presets like `Mon-Fri 9-5` or by dragging on the grid (which is Monday-first).
@@ -122,9 +161,9 @@ Optional **campaign dates** bound when it may send. Leave both blank to run open
The play and pause buttons work from the list row or the detail view. Starting moves the campaign to **active** and begins scheduling inside your windows and limits; pausing stops new scheduling immediately and resumes from where it left off.
A campaign can pause itself: **paused, no accounts** when it loses every sender, **paused, trial expired** when a trial ends, **auto-paused** when a guardrail trips, **needs verification** when address verification has refused every remaining lead (the campaign offers to re-verify them or send anyway; see [address verification](/guides/deliverability/#address-verification)), and plain **paused** if it ever loses its workspace, because unsubscribes, bounces and complaints are checked per workspace and cannot be honoured without one. It moves to **finished** once every contact completes the sequence or its end date passes. Configured to do so, it also stops following up with a contact the moment they reply.
A campaign can pause itself: **paused, no accounts** when it loses every sender or no sender can send under its settings (a sending behaviour profile with no working days), **paused, trial expired** when a trial ends, **auto-paused** when a guardrail trips, **needs verification** when address verification has refused every remaining lead (the campaign offers to re-verify them or send anyway; see [address verification](/guides/deliverability/#address-verification)), and plain **paused** if it ever loses its workspace, because unsubscribes, bounces and complaints are checked per workspace and cannot be honoured without one. It moves to **finished** once every contact completes the sequence or its end date passes, unless **Keep running for new leads** is on: then it stays active and shows **waiting for leads** until the next lead arrives, and only its end date finishes it. Configured to do so, it also stops following up with a contact the moment they reply.
A finished campaign can be started again: after extending or clearing its end date, or adding new leads, pressing play resumes it. If there is genuinely nothing left to send it finishes again immediately with a message saying so.
A finished campaign starts again on its own when a lead is added to it, whether by a linked segment, the API, a form, an integration or an automation, through the same launch checks as pressing play. If a check refuses the restart (the list is a bounce risk, the plan cannot send, too many campaigns are active), the activity log says so and the new leads wait until you fix the cause and press play. Pressing play also works after extending or clearing the end date. If there is genuinely nothing left to send it finishes again immediately with a message saying so.
## Duplicate and delete
@@ -136,6 +175,8 @@ Every campaign has a **⋯** menu, on its row in the list and in the header of i
## Auto-pause guardrails
**Preferences** > **Opt-out line** chooses what this campaign appends after the signature: the workspace default from **Settings > Sending**, a reply-to-opt-out sentence, an unsubscribe link, or nothing. **Unsubscribe header** attaches the RFC 8058 headers alongside. See [unsubscribe and suppression](/guides/unsubscribe/).
**Preferences** > **Auto-pause** stops a campaign the moment its rates leave the band you set, instead of waiting for a mailbox provider to react. Off by default.
| Rule | Direction | Default | Why |
@@ -34,6 +34,12 @@ Indicators are workspace-scoped. Teammates see only your name, avatar, current p
**Events are permission-aware**: a member without inbox access never receives unibox events, and billing events reach only those who can manage billing. See [Team roles](/guides/team-roles/).
## Connection status
A dot in the header reports the live connection: green **Live** when it is healthy, amber **Slow connection** or red **Poor connection** when heartbeats are taking longer than they should, amber **Reconnecting...** while it retries, and red **Disconnected** when it is down. The reading comes from the measured round trip to the realtime service, so amber means your connection genuinely is slow rather than merely busy.
Stored data still catches up while it is down. Warmbly refreshes counts and lists on a slower cycle, and on reconnect it refetches everything that could have drifted. The events themselves are never replayed, so live-only signals such as presence and cursors resume from wherever things stand rather than backfilling what you missed. On a self-hosted install a permanently disconnected dot usually means the realtime service is not running; see [troubleshooting](/development/troubleshooting/).
## Live cursors and chat
On a shared page you see each other's cursors move, tagged with avatar and name, across the contacts table, deal board, unibox, settings, and everywhere else. On app-scrolling pages the cursor tracks content as either person scrolls.
+9 -3
View File
@@ -100,16 +100,22 @@ Every contact records its first-touch source: how it entered the workspace, with
| Import | Created by the file import wizard | The file name |
| Sheet sync | Created by a Google Sheets sync source | The sheet |
| API | Created through the API with an API key | The key's name |
| Form | Submitted a hosted [form](/guides/forms/) | The form |
| Automation | Written by an automation's **Create or update contact** action | The automation |
| AI assistant | Created by the assistant on your behalf | |
| Unknown | Created before Warmbly recorded sources | |
Only new contacts get a source. An import or API call that matches an existing contact updates its fields and leaves the original source in place, so an address that arrived by hand and later turned up in a file still reads as manual.
A new contact is also an event. `contact.created` goes to your [webhooks](/guides/webhooks/) and can start an [automation](/guides/automations/#triggers) with the contact's fields and source, except for bulk arrivals, which stay silent: file imports, sheet syncs, and a single API request adding more than 100 contacts.
## Activity timeline
The **Activity** tab of a contact is one feed, newest first, of everything Warmbly knows about them: every campaign email sent, opened, clicked, replied to or bounced (with the campaign, step, subject and sending mailbox), replies with their classified intent, deliverability and suppression events, notes, meetings, and the contact's lifecycle: when it was created and how, and each time it joined or left a campaign or a category.
Filter chips narrow the feed (**Emails**, **Replies**, **Deliv.**, **Notes**, **Meetings**, **Campaigns**, **Lifecycle**), the search box matches subjects, campaigns, steps, mailboxes, categories and reasons, and the date picker bounds it. Each row stays to one line until you click it; expanded, it shows every detail the event carries. The feed updates live as teammates and the schedulers write to it.
Opens appear once per event, not once per email: a second open from another device is its own row. Opens and clicks show the mail client or browser they came from and the city and country when known (see [what is recorded per open and click](/guides/analytics/#what-is-recorded-per-open-and-click)); expanded, the operating system, device and full location. A click names the link: the row reads **Clicked Pricing** and, expanded, shows the link's text, its full URL, the UTM source, medium, campaign and content it carried, and the browser. Every link in an email is tracked on its own, so two links clicked are two rows. Opens and clicks that came from a machine rather than the person (a mail privacy proxy, a security gateway that follows every link at delivery) carry an **auto** badge, and the expanded row says which rule caught them; see [Link tracking and UTM parameters](/guides/campaigns/#link-tracking-and-utm-parameters).
Filter chips narrow the feed (**Emails**, **Replies**, **Deliv.**, **Notes**, **Meetings**, **Campaigns**, **Lifecycle**), the search box matches subjects, campaigns, steps, mailboxes, categories, reasons and, for clicks, the link's text, URL and UTM values, and the date picker bounds it. Each row stays to one line until you click it; expanded, it shows every detail the event carries. The feed updates live as teammates and the schedulers write to it.
At the top of the tab sits the campaign panel: for each campaign the contact is in, its flow with this contact's progress, the lead status, and what the scheduler will do next. See [Campaigns](/guides/campaigns/) for how the next action is worked out and what its states mean.
@@ -130,9 +136,9 @@ Campaigns can create and advance deals automatically with **Create deal** and **
A suppressed contact receives no further mail. Unsubscribed contacts are skipped by every campaign automatically; toggle subscription on the Details tab, and filter the list by Subscribed or Unsubscribed.
Suppression also happens automatically on a hard bounce or spam complaint, and a sequence can run an **Unsubscribe** action.
Suppression also happens automatically when a recipient unsubscribes or replies asking to stop, on a hard bounce or spam complaint, and when a sequence runs an **Unsubscribe** action. The **Suppression list** tab holds every entry, takes addresses and whole domains by hand, and lets you lift an entry; a suppressed contact's Overview tab shows why and offers the same.
This is the safe default response to a bad signal: stop sending rather than keep collecting bounces and complaints. See [Deliverability](/guides/deliverability/) for the signals behind it.
This is the safe default response to a bad signal: stop sending rather than keep collecting bounces and complaints. See [unsubscribe and suppression](/guides/unsubscribe/) for the full picture and [Deliverability](/guides/deliverability/) for the signals behind it.
## Where to go next
+2 -2
View File
@@ -96,11 +96,11 @@ If a campaign has already been paused because every one of its mailboxes was gat
## Built-in protections
**One-click unsubscribe** (RFC 8058) puts a native Unsubscribe control in the recipient's mail client. It drives complaint rate down, because someone who can opt out cleanly rarely hits "mark as spam" instead, and Google's bulk-sender guidance effectively requires it at volume. Unsubscribes are suppressed automatically.
**An opt-out in every email.** Each campaign email ends with a way to stop hearing from you: by default a plain "just reply and let me know" sentence, whose replies are detected and honoured automatically, or an unsubscribe link if you prefer one. Campaigns can also attach the RFC 8058 `List-Unsubscribe` headers, which put a native Unsubscribe control in mail clients that show one. Either way, someone who can opt out cleanly rarely hits "mark as spam" instead, and every opt-out is suppressed automatically. See [unsubscribe and suppression](/guides/unsubscribe/).
**Pre-send verification** checks every address before a campaign sends to it and skips the undeliverable ones, keeping bad addresses from becoming the hard bounces that drive the `5%` and `10%` thresholds above. There is nothing to run: new contacts are checked in the background within a minute of being added, each contact carries its verdict (a small mark next to the email), and Settings > Sending shows the whole workspace at a glance. See [address verification](#address-verification) below for how verdicts are produced and what to do when one is wrong.
**Suppression** is the safety net: a suppressed recipient is never emailed again by any campaign. Warmbly suppresses automatically on a bounce, a spam complaint, or an unsubscribe, because once someone has bounced or complained the safest response is to stop, not to keep collecting negative signals.
**Suppression** is the safety net: a suppressed recipient is never emailed again by any campaign. Warmbly suppresses automatically on a bounce, a spam complaint, or an unsubscribe, because once someone has bounced or complained the safest response is to stop, not to keep collecting negative signals. The list, with anything you add by hand (addresses or whole domains), is under **Contacts > Suppression list**.
The list is held per workspace, and the check runs against the sending campaign's workspace before every send. If a campaign somehow has no workspace, that check cannot be applied, so the campaign is paused with the reason in its activity feed instead of sending unchecked.
+2
View File
@@ -42,6 +42,8 @@ New reply from {{.contact_email}} on campaign {{.campaign_id}} ({{.intent}})
| Trigger | Variables |
| --- | --- |
| Reply received | `contact_email`, `contact_id`, `campaign_id`, `intent`, `confidence`, `subject`, `snippet` |
| Contact created | `contact_email`, `contact_id`, `first_name`, `last_name`, `company`, `phone`, `subscribed`, `source`, `source_detail`, `custom_fields` (a map: `{{.custom_fields.industry}}`), `campaign_ids`, `category_ids` |
| Form submitted | `contact_email`, `contact_id`, `first_name`, `last_name`, `company`, `phone`, `form_id`, `form_name`, `submission_id`, `source_url`, `campaign_id`, `data` (every answer by its key: `{{.data.team_size}}`) |
| Meeting booked / rescheduled | `invitee_name`, `invitee_email`, `event_name`, `scheduled_for`, `join_url`, `source`, `contact_id` |
| Meeting canceled | Same, without `join_url` |
| Email bounced, Deliverability complaint | `contact_email`, `campaign_id`, `contact_id`, `event_type`, `provider`, `reason` |
+3 -1
View File
@@ -58,7 +58,7 @@ The **Settings** tab controls what a submission does:
- **Success message** is shown after submitting, or set a **redirect URL** to send the visitor to your own thank-you page instead.
- **Add to categories** files every submitted contact under the categories you pick, for example "Website leads".
- **Add to campaign** enrolls new contacts as leads in the campaign you pick. Sending still follows the campaign's own schedule, limits and windows; a form never causes immediate mail.
- **Add to campaign** enrolls new contacts as leads in the campaign you pick. Sending still follows the campaign's own schedule, limits and windows; a form never causes immediate mail. Turn on the campaign's **Keep running for new leads** so it waits between submissions instead of finishing; a finished campaign restarts when a lead arrives, through the usual launch checks, and a refused restart is noted in its activity log while the lead waits.
- **Spam protection** and **allowed embed domains** are covered below.
## Publishing and sharing
@@ -109,6 +109,8 @@ The **Submissions** tab is a table of everyone who engaged, newest activity firs
Each submitted contact carries `form` as its first-touch source, gets a "Submitted a form" entry on its activity timeline, and can be filtered in segments by source. The form's page views, submission count and conversion rate show on the Forms list.
A submission is also the **Form submitted** [automation trigger](/guides/automations/#triggers) and the `form.submitted` webhook event, carrying the form, the contact and every answer by its key, so a flow can tag the contact, open a task or post to Slack the moment it lands.
Submissions also fire the `form.submitted` [webhook](/guides/webhooks/), carrying the answers and the contact id, so you can forward leads to any external system.
## Analytics
+3 -1
View File
@@ -72,7 +72,9 @@ URLs are validated before storage under the same policy as customer webhooks: th
Integrations are most useful as the actions at the end of an [automation](/guides/automations/): post to Slack or Discord, upsert into a CRM, or fan out to Zapier, Make, n8n, or any signed webhook.
The most actionable events are available as triggers: reply received (with intent), email bounced, unsubscribed, meeting booked, rescheduled, or canceled, warmup health changed, and deliverability complaint.
The most actionable events are available as triggers: reply received (with intent), contact created, form submitted, email bounced, unsubscribed, meeting booked, rescheduled, or canceled, warmup health changed, and deliverability complaint.
Data flows in as well as out. An automation's inbound webhook URL plus its **Create or update contact** action turns any push (a lead-form connector in Zapier, Make or n8n, a form tool's webhook, your own code) into a tagged, campaign-enrolled contact. See [Lead intake](/guides/automations/#lead-intake).
Actions run in the background, so a slow third party never blocks the event that triggered it. Failures are recorded against the connection's health and visible in its recent activity rather than retried forever or silently lost.
+54 -3
View File
@@ -41,7 +41,56 @@ With two-factor authentication on, generate an app password in your provider's s
Credentials and both connections are validated when you add the account, so wrong settings fail immediately rather than silently at send time. Tokens and credentials are sealed with envelope encryption before they touch storage.
</Callout>
**Limits**: `200` mailboxes per workspace by default (higher allowances come through a reviewed limit-increase request), and `5` new mailboxes per workspace per day as an abuse guardrail that resets daily. Spread large onboarding batches across days.
## Mailbox allowance
Mailboxes are unlimited on every paid plan. What keeps that honest is a fair-use allowance derived from the plan's sending volume: one mailbox for every send a day the plan includes.
| Plan | Sends per day | Mailboxes |
| --- | --- | --- |
| Free workspace | none | `10` |
| Starter | `150` | `150` |
| Grow | `3,000` | `3,000` |
| Business | `15,000` | `15,000` |
| Enterprise | custom | as many as the volume needs |
That is deliberately far more than safe sending ever needs: at the recommended `30` to `50` sends a day per mailbox, a Business workspace fills its volume with a few hundred mailboxes and still has room for tens of thousands. The allowance exists so that it is never the reason to run a mailbox hotter. A plan whose daily sends are uncapped holds unlimited mailboxes, and a self-hosted instance without billing never counts.
The connect dialog shows the allowance up front: a quiet count while there is room, a warning near the cap, and a clear full state that leads to the request flow instead of letting you type credentials that would be refused. When a connect is refused the answer is the same dialog, not an error toast.
**Getting more.** Two paths, both in the dialog:
- **Move to a bigger plan.** The dialog names the next plan and how many mailboxes it holds; the change is prorated and takes effect immediately.
- **Request an increase.** Keep your plan and ask for a higher allowance with a sentence on what you are sending. An operator reviews it, usually within a business day, and the new allowance applies to the workspace straight away. The dialog shows the open request while it is pending and lets you withdraw it. History lives under **Settings > Limits**.
Nothing is ever removed for being over the allowance. A workspace that moves to a smaller plan keeps every mailbox sending and warming; it simply cannot add more until it is back under, or the allowance is raised.
There is no daily cap on how many mailboxes you connect: a Business workspace can connect thousands in one afternoon, and the [bulk import](#connecting-many-mailboxes-at-once) exists for exactly that.
## Connecting many mailboxes at once
Pick **Bulk import from CSV** in the connect dialog to connect any number of SMTP and IMAP mailboxes from one file. Gmail and Outlook mailboxes sign in one at a time, because each needs its own consent.
**The file.** One mailbox per row. `email`, `smtp_host` and `imap_host` are required, plus a password; everything else has a default.
| Column | Default |
| --- | --- |
| `email` | required |
| `name` | derived from the address (`alex.rivera@` becomes Alex Rivera) |
| `smtp_host`, `imap_host` | required |
| `smtp_port`, `imap_port` | `587` and `993` |
| `smtp_user`, `imap_user` | the address, or a shared `username` column |
| `smtp_password`, `imap_password` | a shared `password` column |
| `smtp_security`, `imap_security` | inferred from the port (`465` and `993` are `tls`, `587` and `143` are `starttls`) |
The dialog offers a template with these headers, and accepts the common spellings (`smtp_server`, `app_password`, `login`, and so on).
**What happens.** The file is read in your browser and checked before anything is sent: rows missing something the connect needs are listed with the reason and left out of the run. The preview also says how many rows fit your allowance; if the file is larger, the first rows that fit are connected and the rest are reported as failed so you can request more and re-upload only those.
The run then streams the rows to the server in small batches. Every credential is verified against its own server before it is saved, the same as a single connect, so a large file takes a few seconds per mailbox; the dialog shows live progress and the mailbox list behind it fills in as they land, for everyone in the workspace. You can stop after the batch in flight. Closing the tab loses nothing that was already connected.
**When it is done** you see how many connected, how many were already here, and how many did not connect, with a reason per row. **Retry failed** runs those rows again without leaving the dialog, passwords still in memory. **Download failed rows** gives you those rows as you uploaded them plus an `error` column, with the password columns left out so no credential lands in a Downloads folder; add them back before uploading the fixed file. Re-uploading is always safe: a mailbox that is already connected is skipped, never doubled.
The same endpoint is available to the API as `POST /emails/onboarding/smtp-imap/bulk`, up to `50` rows per call, answered per row.
## Reconnecting an account
@@ -52,7 +101,7 @@ When the provider stops accepting a mailbox's stored credential (a password chan
A successful reconnect stores the new credential, clears the authentication error, reactivates the mailbox on its existing worker, and it resumes syncing from where it stopped. Nothing else changes: settings, history, warmup progress, and campaign membership all stay.
Reconnecting never counts against the mailbox limit or the daily connect guardrail, so a workspace at its cap can still fix a broken mailbox. A mailbox whose sign-in is held by Warmbly Cloud is reconnected from your cloud workspace instead; the button says so if you try locally.
Reconnecting never counts against the mailbox allowance, so a workspace at its cap can still fix a broken mailbox. A mailbox whose sign-in is held by Warmbly Cloud is reconnected from your cloud workspace instead; the button says so if you try locally.
## What gets synced
@@ -91,7 +140,9 @@ SMTP mailboxes get a **Keep a copy of sent mail** toggle, on by default. SMTP su
Turn it off when your provider already saves its own copy of anything submitted over SMTP, which Gmail, Fastmail and Zoho do, or the folder ends up with two of every message. Gmail and Outlook mailboxes connected with OAuth never show the toggle: their APIs file the copy themselves. Warmup mail is never filed, since it would bury your real sent mail.
The same tab sets the **display name**, **reply-to** (empty uses the mailbox address), **signature** in plain text and HTML, and **tags** for grouping.
The same tab sets the **display name**, **reply-to** (empty uses the mailbox address), **signature** in plain text and HTML, and **tags** for grouping. A new display name is on the From header of the next message the mailbox sends, campaign, reply or warmup alike; nothing needs to be reconnected.
The HTML signature is placed in a block of its own one line below the body, so it arrives without the stack of blank lines above it that Apple Mail and Outlook used to show. Put any extra spacing you want inside the signature itself. The plain-text signature follows the body after a single blank line.
### Custom tracking domain
+11
View File
@@ -51,6 +51,17 @@ Find Contact (by exact email), Find Deal, Find CRM Task, Find Campaign, Find Mai
Pair a search with an action for idempotent flows: Find Contact, then Create or Update Contact.
## Lead forms and other lead sources
Facebook and Instagram Lead Ads, LinkedIn Lead Gen Forms and TikTok Lead Generation all deliver new form submissions to Make in real time, which makes Make the shortest route from an ad to a follow-up sent from your own mailbox.
1. **Trigger:** *Facebook Lead Ads: Watch Leads* (or the LinkedIn or TikTok equivalent), picking the Page and form.
2. **Module:** Warmbly *Create or Update Contact*. Map the form's email and name fields, put every other question into a custom field, and pick the categories. Then *Add to Campaign* with the campaign that follows up. Re-running the same lead updates the contact rather than duplicating it.
Prefer to keep the mapping inside Warmbly? Use the *HTTP: Make a request* module to POST the lead as JSON to an [inbound webhook automation](/guides/automations/#lead-intake) instead. The automation's **Create or update contact** action maps the JSON keys onto contact fields with templates, and the same flow can tag, notify Slack and open a task.
The follow-up still runs through the campaign's mailboxes, so a burst of leads queues under each mailbox's daily cap and spacing rather than going out at once. Cold email tools do not ship a native Meta connector; this route is what their customers use too.
## Good to know
- **Sending respects your limits.** Send Email and Reply in Inbox honor each mailbox's daily cap and spacing, and support smart and scheduled send modes.
+3
View File
@@ -12,6 +12,7 @@
"sequences",
"expressions",
"deliverability",
"unsubscribe",
"advisor",
"---Contacts and inbox---",
"contacts-crm",
@@ -36,7 +37,9 @@
"integrations",
"zapier",
"make",
"n8n",
"---Account and team---",
"billing",
"analytics",
"notifications",
"security",
+37
View File
@@ -0,0 +1,37 @@
---
title: n8n
description: Connect Warmbly to n8n workflows, self-hosted or cloud, with the REST API and webhooks in both directions.
---
n8n has no Warmbly node yet, and it does not need one: its HTTP Request and Webhook nodes cover both directions, and every Warmbly endpoint is documented in the [API reference](/api/endpoints/).
## Connecting
Create a scoped [API key](/api/authentication/) in Settings > API keys with only the scopes the workflow needs (contacts and campaigns for lead intake; add inbox for replies). In n8n, add a **Header Auth** credential with the name `Authorization` and the value `Bearer <your key>`, and attach it to every HTTP Request node that calls Warmbly.
## Warmbly to n8n
To start a workflow from a Warmbly event, add a **Webhook** node, copy its production URL, and either:
- register it as a [webhook endpoint](/guides/webhooks/) subscribed to the events you want (signed with HMAC so the workflow can verify the sender), or
- add an n8n connection on the Integrations page and use it as a **Send a webhook** action inside an [automation](/guides/automations/), which lets you filter and branch before the request leaves Warmbly.
Both deliver the same JSON body: a delivery id, the event type and the full event data.
## n8n to Warmbly
Two routes, depending on where you want the field mapping to live.
**Mapping in n8n.** An HTTP Request node calling `POST /api/v1/contacts` with the contact's fields, `categories` and `campaigns`. The write is an upsert by email, so re-running a workflow updates the contact instead of duplicating it.
**Mapping in Warmbly.** Create an automation with the **Inbound webhook** trigger, copy its URL, and POST the raw payload to it from an HTTP Request node. The automation's **Create or update contact** action maps the JSON keys onto contact fields with templates (`{{.email}}`, `{{.answers.company}}`), tags the contact and enrols it in a campaign, and the same flow can notify Slack or open a task. See [Lead intake](/guides/automations/#lead-intake). This route needs no API key: the URL is the credential.
## Lead forms and other lead sources
n8n's *Facebook Lead Ads Trigger* node fires on a new Facebook or Instagram submission. LinkedIn Lead Gen Forms and TikTok Lead Generation have no dedicated node; point their lead notification webhooks at an n8n *Webhook* node instead. Connect either to one of the routes above and the ad's answers become a tagged, campaign-enrolled contact that your mailbox follows up with. The follow-up still runs under each mailbox's daily cap and spacing, so a burst of leads queues rather than going out at once.
## Good to know
- **Sending respects your limits.** Sends and replies made through the API honor each mailbox's daily cap and spacing.
- **Rate limits** apply per API key; a large backfill should go through the [import endpoints](/api/reference/contacts/#commit-an-import) rather than one request per row.
- For the lowest latency without hosting a URL, subscribe to the [developer websocket](/api/realtime/) instead of a webhook.
+7 -3
View File
@@ -51,7 +51,9 @@ Sequences can pin too: the **Add to segment** and **Remove from segment** action
- **Browse**: a segment page lists its current members with the same table, filters, detail drawer and bulk actions as the contacts page.
- **Add to campaign**: enrols every current member as a lead of the campaign you pick, from the segment page or with **From segment** on a campaign's Leads tab. Contacts already in that campaign are skipped, and a running campaign wakes up to schedule the new leads. This is a snapshot: contacts who join the segment later are not added until you run it again. To keep a campaign fed automatically, [link the segment](#linking-a-segment-to-a-campaign) instead.
- **Link to campaign**: attaches the segment to a campaign as a live audience, so contacts who join the segment later become leads on their own. See [below](#linking-a-segment-to-a-campaign).
- **One-time email**: send one message to the segment without building a sequence. **Campaigns** > **New campaign** > **One-time email** picks the segments, shows how many contacts they resolve to and how long the mailbox pool needs, and sends now or on a date. See [one-time emails](/guides/campaigns/#one-time-emails).
- **Filter**: the contact list's filter bar has a **Segment** pill, and the same scope carries into an export.
- **Export**: **Export** on a segment page downloads its members as CSV, XLSX or JSON, either every member, the ones matching the page's filters, or the selected rows.
- **Duplicate**: the segment menu copies a definition to start a variation from.
- **Search and export**: the contact search and export accept `segment_ids`, so anything that takes a contact filter can be scoped to a segment.
@@ -59,14 +61,16 @@ Sequences can pin too: the **Add to segment** and **Remove from segment** action
Where **Add to campaign** copies today's members once, a linked segment is a live audience: the campaign keeps enrolling whoever the segment says belongs. Manage the links from the **Segments** button on a campaign's Leads tab; a campaign can link up to 20 segments.
Linking enrols every current member as a lead immediately. From then on, any contact who enters the segment, a new contact, an import, a condition edit, a manual pin-in, or simply drifting into a date or engagement condition, is enrolled automatically; a background check picks up drift about every 2 minutes.
Linking enrols every current member as a lead immediately and turns on the campaign's **Keep running for new leads** setting, so the campaign waits for members instead of finishing when it runs out (turn it off in the campaign's preferences if you want it to finish). From then on, any contact who enters the segment, a new contact, an import, a condition edit, a manual pin-in, or simply drifting into a date or engagement condition, is enrolled automatically; a background check picks up drift about every 2 minutes.
The Leads tab shows every linked segment in a **Linked segments** strip: the segment's name, how many of its members are leads (`120/120`), a **no contacts** mark when it currently matches nobody, and a **held out** count when members were removed from the campaign by hand. Clicking a chip filters the list to that segment's leads; **Add back** re-enrols held-out members.
Enrolment is additive. A contact who later falls out of the segment keeps their lead row and their progress; the link only ever adds. Detaching a segment stops future enrolment but leaves the leads it already added in place. Removing a lead from the campaign by hand is respected too: automatic enrolment will not re-add that contact, even while they still match a linked segment, until you add them back yourself.
The campaign reacts to growth by status:
- **Active**: woken up to schedule the new leads straight away.
- **Finished**: restarted automatically, through the same launch checks as pressing play.
- **Active**, including one **waiting for leads**: woken up to schedule the new leads straight away.
- **Finished**: restarted automatically, through the same launch checks as pressing play. A refused restart is noted in the campaign's activity log.
- **Paused** (or draft): the leads accumulate and send when the campaign runs.
A segment linked to a campaign cannot be deleted; detach it from the campaign first.
+5 -1
View File
@@ -20,12 +20,16 @@ Each card has two source dots: the **bottom dot** is a plain "go there next" con
A step has an internal **name** (never seen by recipients), a **subject** with a variable menu for merge fields like `{{.FirstName}}`, and a rich-text **body**.
The Preview tab renders through the real send engine against a sample contact, so merge fields, conditionals, and spintax resolve exactly as they will at send time. Malformed templates (an `{{if}}` with no `{{end}}`) are flagged before you start the campaign. See [Personalization](/guides/expressions/) for everything you can put in copy.
The Preview tab renders through the real send engine, so merge fields, conditionals, and spintax resolve exactly as they will at send time. **Preview as** picks who it renders for: a built-in sample contact, one of the campaign's leads, or any contact you search for, so a custom field your list does not actually have shows up as an unresolved token instead of looking fine. The mailbox picker next to it adds that sender's signature and shows the From name recipients will see. When the campaign is known the preview also appends the opt-out footer and lists the files attached to the campaign. Malformed templates (an `{{if}}` with no `{{end}}`) are flagged before you start the campaign. See [Personalization](/guides/expressions/) for everything you can put in copy.
**Send test** in the preview mails the saved step to an address of your choice through the chosen mailbox, rendered for the chosen contact and carrying the attachments, signature and opt-out footer. Save the step first: the test sends what is stored, not unsaved edits. Opens and clicks on a test are not tracked, and its opt-out link names nobody, so clicking it suppresses no one.
<Callout type="info" title="Templates">
Apply a saved template to a step, or save a step as a reusable one from the composer. Templates carry their subject and body and live in your shared library.
</Callout>
Upload attachments for a step below its composer by dragging and dropping files or clicking the upload area. A file uploaded there is sent with every email from that step and from no other step, and can be removed in the same place. Files attached to the campaign itself rather than to a step, which can only be added through the API, ride every step: they are listed under **Sent with every step** so each step shows everything it carries. Attachments count against your workspace storage limit, shown under **Settings > Billing**; an upload that would pass it is refused with `storage_limit_reached`, and two uploads racing for the last of the quota cannot both get in. Removing a step deletes the files scoped to it.
### Action steps
A step can perform an action instead of sending: add or remove a tag, label email, create a task, create a deal or move its stage, unsubscribe the contact, notify (fires your webhooks and integrations), run an automation, or switch. They connect like email steps and work best at the end of a reply branch, for example creating a deal and notifying your team on a positive reply.
+70
View File
@@ -0,0 +1,70 @@
---
title: Unsubscribe and suppression
description: How recipients opt out of campaign email, what Warmbly does when they do, and how the suppression list works.
---
Every campaign email gives its recipient a way to stop hearing from you, and every opt-out is honoured automatically across the whole workspace. This page covers the three mechanisms, the suppression list behind them, and the settings that shape what a recipient sees.
## What a recipient sees
Warmbly appends an opt-out to every campaign email, after the signature. There are three modes, set for the workspace under **Settings > Sending > Unsubscribe** and overridable per campaign under **Preferences**:
| Mode | What is appended | Default |
| --- | --- | --- |
| Reply to opt out | A plain sentence such as "If this isn't relevant, just reply and let me know and I won't email you again." | Yes |
| Unsubscribe link | A sentence with a real link, for example "Not the right person, or not interested? Unsubscribe" | |
| Nothing | No opt-out in the body | |
The wording of the sentence and of the link text is yours to change.
**Reply to opt out** is the default because it reads as a personal email, which is what cold outreach is. A formal unsubscribe link and footer are the strongest signal a mailbox provider has that a message is bulk marketing, and several deliverability teams report worse placement for cold email that carries one. A reply that asks to stop is detected and honoured automatically (see below), so the plain sentence is a real mechanism, not a courtesy. CAN-SPAM, CASL and the Australian Spam Act all accept a reply as the opt-out method.
**Unsubscribe link** is the right choice when your list skews toward consumers, when your legal team asks for a link, or when your volume is high enough that provider bulk-sender rules apply. The link is unique to the recipient and campaign, signed so it cannot be guessed or altered, and valid for a year after the send. Clicking it opens a plain confirmation page with one button. Nothing happens until the button is pressed, because link scanners and preview fetchers follow every link in an email. The page then offers a way back for anyone who unsubscribed by mistake.
You can also place the link inside your own copy instead of the footer: insert the **Unsubscribe link** variable from the variable menu, or type `{{.UnsubscribeLink}}`. Click tracking never rewrites it, so an opt-out is never counted as a click.
## The List-Unsubscribe header
Independently of the body, each campaign can attach the `List-Unsubscribe` and `List-Unsubscribe-Post` headers (RFC 8058). Mail clients that recognise them show their own **Unsubscribe** control next to the sender's name, and pressing it opts the recipient out with no page at all. The toggle is **Preferences > Unsubscribe header**, on by default.
Two things to know. Gmail only shows its button for mail it already classes as bulk, so a one-to-one style cold email from a Workspace mailbox usually does not get one. And Google and Yahoo require the header only above roughly five thousand messages a day to their consumer inboxes, which no mailbox at Warmbly's default limits approaches. The header costs nothing and is worth leaving on; it is not a substitute for the opt-out line.
## Replies that ask to stop
Warmbly reads every reply to a campaign email. One that asks to stop, in wording such as "unsubscribe", "remove me", "stop emailing me", "opt out" or "do not contact", puts the sender on the suppression list immediately. Only the new text of the reply is read: the quoted history below it carries your own opt-out wording and is ignored. Phrases match on whole words, so "stop by our booth" does not opt anyone out.
A reply that says "not interested" is classed as negative and can stop the sequence, but it does not suppress the contact. Someone who says no today may be a fit next year; someone who says stop is asking not to be contacted at all.
Turn this off with **Settings > Sending > Honour replies that ask to stop** if you would rather handle such replies by hand. Leaving it off with the reply-to-opt-out line in place means promising an opt-out you then honour manually.
## The suppression list
**Contacts > Suppression list** is every address and domain that no campaign in the workspace will email, however it got there:
| Source | How it got there |
| --- | --- |
| Unsubscribed | Clicked the link, pressed the mail client's button, or replied asking to stop |
| Spam complaint | The recipient's provider sent a complaint report for a message you sent |
| Bounced | A permanent delivery failure for the address |
| Added by hand | Someone on the team added it |
| Imported | Pasted in as part of a list |
The list is workspace-wide. An entry stops the address in every campaign, in new campaigns created later, and in the unibox composer, which refuses to send to it. Import a list containing a suppressed address and the contact is created but never mailed. The [launch check](/guides/campaigns/#the-launch-check-on-your-list) counts suppressed leads out of a campaign's deliverable total, so what the campaign says it will send is what it sends.
**Adding entries.** Press **Add** and paste addresses or domains, one per line or as a column from a spreadsheet. A bare domain (`acme.com` or `@acme.com`) suppresses every address at it, which is the usual way to keep customers, partners and your own company out of outreach. An optional reason is kept with each entry.
**Removing entries.** Each entry has a **Remove** action, and a suppressed contact's drawer shows the same. Removing an entry the recipient made themselves, by unsubscribing, complaining or bouncing, is confirmed with a stronger warning and recorded in the audit log with who lifted it and why the address was listed. Removing an address also restores the contact's **Subscribed** flag, so the two never disagree.
**Contacts and the Subscribed flag.** The contact's own **Subscribed** toggle is a second gate: a contact switched off is skipped by every campaign even with no suppression entry. An opt-out sets both, so a contact who unsubscribed reads as unsubscribed everywhere.
Suppression entries travel with a [workspace export](/guides/workspace-export-import/), so a workspace moved to another instance does not re-mail people who already opted out.
## Webhooks and the API
An opt-out fires the `campaign.unsubscribed` [webhook](/guides/webhooks/) with a `source` of `one_click` (the mail client's button), `link` (the link in the email), `reply` (a reply asking to stop) or `action` (a sequence's **Unsubscribe** step).
The suppression list is available over the API as `GET /suppressions`, `POST /suppressions` and `DELETE /suppressions/:id`. See [deliverability and ops](/api/reference/deliverability-ops/#list-the-suppression-list).
## Self-hosting
Unsubscribe links are served by the API process on the origin in `API_PUBLIC_URL`, so that variable must be set to the address recipients can reach. Without it Warmbly cannot mint links: the header is left off, and the link mode falls back to the reply-to-opt-out sentence. Links are signed under `AUTH_SECRET`; rotating it invalidates links in emails already sent.

Some files were not shown because too many files have changed in this diff Show More