starred/warmbly
1
0
Fork 0
mirror of https://github.com/warmbly/warmbly.git synced 2026-09-06 00:01:24 +00:00
Code Issues Packages Projects Releases Wiki Activity

feat: add docs page Deploying without Docker covering backing services, building the Go, Rust, Elixir and Vite artifacts from source, hand-written config.js, warmbly.env and worker.env, systemd, nginx, claiming, native remote workers, upgrades, backups and troubleshooting

Browse Source
This commit is contained in:
Matthew Meszaros
2026-08-29 23:37:22 -07:00
parent b9152ea460
commit cc6783cb0a
1 changed files with 584 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docs/content/docs/development/bare-metal.mdx
+584
View File
@@ -0,0 +1,584 @@
---
title: Deploying without Docker
description: Step-by-step instructions for running Warmbly as native systemd services on one Linux host, built from source, with no containers anywhere.
---
The [self-hosting guide](/development/deployment-guide/) assumes Docker Compose. Nothing in Warmbly needs a container, though: every service is a single binary or a static build that reads its configuration from environment variables. This page walks through building those artifacts from source and running them under systemd on one Linux host, with nginx in front.
It is longer than the compose route because it does by hand what the compose file does for you: installing the backing services, generating secrets, fanning one set of values out to five processes, and serving two static frontends. Read it once end to end before starting; the [quick reference](#quick-reference) at the bottom is enough the second time.
<Mermaid
chart={`
flowchart LR
U["Browser"] --> NG["nginx :443"]
NG --> WEB["web + admin (static files)"]
NG --> API["backend :8080"]
NG --> RT["realtime :4000"]
NG --> TRK["tracking :3000"]
subgraph HOST["One host, systemd"]
API --> PG[("PostgreSQL 16")]
API --> BUS{{"NATS JetStream"}}
API --> RD[("Redis 7")]
BUS --> CONS["consumer"] --> PG
TRK --> BUS
RT --- RD
BUS --> W["worker"]
end
W --> P["Gmail · Microsoft · SMTP"]
`}
/>
## Before you start
| You need | Why |
|----------|-----|
| A Linux host with systemd | The units below; Debian 12, Ubuntu 22.04+ and their relatives are known to work |
| Root or sudo | To install packages, create the service user, and write under `/etc` and `/opt` |
| PostgreSQL 16, Redis 7, NATS 2.10+ | The backing services. Any packaging works: distro packages, upstream repositories, or a managed instance elsewhere |
| Go 1.25 | Builds the backend, consumer, worker, `migrate` and `warmblyctl` |
| Rust (stable, 1.93 or newer) | Builds the tracking service |
| Elixir 1.18 on OTP 26 | Builds the realtime service |
| Node 22 and pnpm | Builds the dashboard and admin panel |
| nginx | Serves the static frontends and terminates TLS for the API, websocket and tracking hosts |
| Five DNS names | `app`, `admin`, `api`, `ws` and `t` under your domain, all pointing at the host |
The toolchains are only needed on the machine that builds. If you would rather not install compilers on the production host, build on another machine of the same architecture and copy `/opt/warmbly` over; nothing below is path-dependent beyond the values you put in the env file.
<Callout type="warn" title="Not a development setup">
For working on Warmbly itself use `make dev`, which runs the Go services natively already and only puts Postgres, Redis and NATS in containers. This page is for running the product for other people, which means real secrets, TLS and a mail relay.
</Callout>
## Install the backing services
<Steps>
<Step>
### PostgreSQL
Install PostgreSQL 16, then create the role and database the backend will own:
```bash
sudo -u postgres psql <<'SQL'
CREATE ROLE warmbly WITH LOGIN PASSWORD 'choose-a-password';
CREATE DATABASE warmbly OWNER warmbly;
SQL
```
The migrations create every table and index, so nothing else is required in the database. Migrations apply automatically on every backend boot.
</Step>
<Step>
### Redis
Install Redis 7 and leave the default configuration: localhost only, no password. Redis holds the rate limiter, the decrypted-key cache and the realtime bridge; none of it is durable state, and a restart loses nothing that matters.
If you enable `requirepass`, put the password in every `REDIS` and `REDIS_URL` value below as `redis://:password@127.0.0.1:6379`.
</Step>
<Step>
### NATS with JetStream
NATS is one static binary. Install it from your distribution or [nats.io](https://docs.nats.io/running-a-nats-system/introduction/installation), then run it with JetStream on and a storage directory:
```bash
sudo useradd --system --home /var/lib/nats --create-home nats
sudo tee /etc/systemd/system/nats.service >/dev/null <<'EOF'
[Unit]
Description=NATS server
After=network-online.target
Wants=network-online.target
[Service]
User=nats
ExecStart=/usr/local/bin/nats-server -js -sd /var/lib/nats -m 8222
Restart=always
[Install]
WantedBy=multi-user.target
EOF
sudo systemctl daemon-reload && sudo systemctl enable --now nats
curl -s http://127.0.0.1:8222/healthz
```
`-js` is the important flag. Without JetStream the backend starts, then fails on its first publish with a message about the stream not existing.
</Step>
</Steps>
## Build from source
<Steps>
<Step>
### Get the code and create the layout
```bash
sudo useradd --system --home /var/lib/warmbly --create-home --shell /usr/sbin/nologin warmbly
sudo mkdir -p /opt/warmbly/bin /opt/warmbly/src /etc/warmbly /var/lib/warmbly/blobs
sudo chown -R warmbly:warmbly /var/lib/warmbly
sudo chmod 0700 /etc/warmbly
sudo git clone https://github.com/warmbly/warmbly /opt/warmbly/src
```
Checkout a release tag rather than `main` when you want a known version: `git -C /opt/warmbly/src checkout vX.Y.Z`. Everything below runs from `/opt/warmbly/src`.
</Step>
<Step>
### Go services
Five binaries come out of one module. Static builds, so the production host needs no Go runtime:
```bash
cd /opt/warmbly/src
export CGO_ENABLED=0
go build -ldflags="-s -w" -o /opt/warmbly/bin/backend ./cmd/backend
go build -ldflags="-s -w" -o /opt/warmbly/bin/consumer ./cmd/consumer
go build -ldflags="-s -w" -o /opt/warmbly/bin/worker ./cmd/worker
go build -ldflags="-s -w" -o /opt/warmbly/bin/migrate ./cmd/migrate
go build -ldflags="-s -w" -o /opt/warmbly/bin/warmblyctl ./cmd/warmblyctl
sudo ln -sf /opt/warmbly/bin/warmblyctl /usr/local/bin/warmblyctl
```
The default build has no Kafka support, which is what you want: NATS is the event bus. Add `-tags kafka` (and `CGO_ENABLED=1` with `librdkafka` installed) only if you are pointing at an existing Kafka cluster.
</Step>
<Step>
### Tracking service (Rust)
```bash
cd /opt/warmbly/src/tracking
cargo build --release
install -m 0755 target/release/tracking /opt/warmbly/bin/tracking
```
The tracking snippet in `static/` is compiled into the binary, so nothing else needs copying.
</Step>
<Step>
### Realtime service (Elixir)
A mix release bundles the Erlang runtime, so the host does not need Elixir installed once the build is done:
```bash
cd /opt/warmbly/src/realtime
mix local.hex --force && mix local.rebar --force
MIX_ENV=prod mix deps.get --only prod
MIX_ENV=prod mix compile
MIX_ENV=prod mix release --overwrite
sudo rm -rf /opt/warmbly/realtime
sudo cp -r _build/prod/rel/realtime /opt/warmbly/realtime
sudo chown -R warmbly:warmbly /opt/warmbly/realtime
```
Build on a machine with the same libc as the host: a release built on Alpine (musl) does not start on Debian (glibc), and the other way round.
</Step>
<Step>
### Dashboard and admin panel
Both are static Vite builds. No URL is baked in at build time; the frontends read `config.js` at runtime, which you write in the next step.
```bash
cd /opt/warmbly/src/web && pnpm install --frozen-lockfile && pnpm build
cd /opt/warmbly/src/admin && pnpm install --frozen-lockfile && pnpm build
sudo rm -rf /opt/warmbly/web /opt/warmbly/admin
sudo cp -r /opt/warmbly/src/web/dist /opt/warmbly/web
sudo cp -r /opt/warmbly/src/admin/dist /opt/warmbly/admin
```
</Step>
<Step>
### Runtime config for the frontends
In the container images an entrypoint renders this file from environment variables. Without containers you write it yourself, once per frontend:
```bash
sudo tee /opt/warmbly/web/config.js >/dev/null <<'EOF'
window.__WARMBLY_ENV__ = {
API_URL: "https://api.example.com",
APP_URL: "https://app.example.com",
TURNSTILE_KEY: ""
};
EOF
sudo tee /opt/warmbly/admin/config.js >/dev/null <<'EOF'
window.__WARMBLY_ENV__ = {
API_URL: "https://api.example.com",
DASHBOARD_URL: "https://app.example.com",
ENV_LABEL: "production",
TURNSTILE_KEY: ""
};
EOF
sudo chmod -R a+rX /opt/warmbly/web /opt/warmbly/admin
```
`TURNSTILE_KEY` is the Cloudflare Turnstile site key and stays empty unless you also set `CAPTCHA_PROVIDER=turnstile` on the backend. The file is read on every page load (nginx serves it with `no-store` below), so changing a URL later never needs a rebuild.
</Step>
</Steps>
## Configure
<Steps>
<Step>
### Generate the secrets
```bash
AUTH_SECRET=$(openssl rand -base64 32)
INTERNAL_API_TOKEN=$(openssl rand -hex 24)
SECRET_KEY_BASE=$(openssl rand -base64 64 | tr -d '\n')
KMS_LOCAL_MASTER_KEY=$(openssl rand -base64 32)
CREDENTIALS_ENCRYPTION_KEY=$(openssl rand -hex 32)
```
<Callout type="warn" title="Back up the last two before you connect a mailbox">
`KMS_LOCAL_MASTER_KEY` and `CREDENTIALS_ENCRYPTION_KEY` seal every stored credential. Losing them is unrecoverable, and a database backup without them cannot be decrypted.
</Callout>
</Step>
<Step>
### Write `/etc/warmbly/warmbly.env`
One file feeds the backend, consumer, tracking and realtime services. Compose maps a few names between services for you (`AUTH_SECRET` to `JWT_SECRET`, `PRIMARY_DB` to `DATABASE_URL`, `REDIS` to `REDIS_URL`); here both spellings sit in the same file with the same value.
```bash
sudo tee /etc/warmbly/warmbly.env >/dev/null <<EOF
# ── Mode ─────────────────────────────────────────────────────────
APP_ENV=prod
DEPLOYMENT_MODE=self_hosted
GIN_MODE=release
AWS_CONFIG_ENABLED=false
# ── Secrets ──────────────────────────────────────────────────────
AUTH_SECRET=${AUTH_SECRET}
JWT_SECRET=${AUTH_SECRET}
INTERNAL_API_TOKEN=${INTERNAL_API_TOKEN}
SECRET_KEY_BASE=${SECRET_KEY_BASE}
KMS_PROVIDER=local
KMS_LOCAL_MASTER_KEY=${KMS_LOCAL_MASTER_KEY}
CREDENTIALS_ENCRYPTION_KEY=${CREDENTIALS_ENCRYPTION_KEY}
# ── Backing services ─────────────────────────────────────────────
PRIMARY_DB=postgres://warmbly:choose-a-password@127.0.0.1:5432/warmbly?sslmode=disable
DATABASE_URL=postgres://warmbly:choose-a-password@127.0.0.1:5432/warmbly?sslmode=disable
REDIS=redis://127.0.0.1:6379
REDIS_URL=redis://127.0.0.1:6379
EVENTBUS_PROVIDER=nats
NATS_URL=nats://127.0.0.1:4222
CODEC_PROVIDER=json
PUBSUB_ENABLED=false
ENCRYPTED_KEYS_PROVIDER=postgres
TASKS_PROVIDER=local
BILLING_PROVIDER=none
CAPTCHA_PROVIDER=none
# ── Storage ──────────────────────────────────────────────────────
BLOB_PROVIDER=filesystem
BLOB_FS_ROOT=/var/lib/warmbly/blobs
BLOB_PUBLIC_BASE_URL=https://api.example.com/public
# ── Where it lives ───────────────────────────────────────────────
API_HOST=127.0.0.1:8080
API_PUBLIC_URL=https://api.example.com
APP_URL=https://app.example.com
CORS_ALLOW_ORIGINS=https://app.example.com,https://admin.example.com
WEBSOCKET_URL=wss://ws.example.com/socket/websocket
TRACKING_DOMAIN=t.example.com
TRUSTED_PROXIES=127.0.0.1/32
# ── Tracking service ─────────────────────────────────────────────
TRACKING_HOST=127.0.0.1
TRACKING_PORT=3000
BACKEND_INTERNAL_URL=http://127.0.0.1:8080
TRACKING_TRUSTED_PROXIES=127.0.0.1/32
# ── Realtime service ─────────────────────────────────────────────
PHX_HOST=ws.example.com
PORT=4000
CHECK_ORIGIN=true
# ── Platform email (resets, invitations, digests) ────────────────
MAIL_TRANSPORT=smtp
SMTP_HOST=smtp.example.com
SMTP_USERNAME=
SMTP_PASSWORD=
SMTP_SECURITY=starttls
EMAIL_NAME=Warmbly
EMAIL_ADDRESS=noreply@example.com
# ── Connecting mailboxes (optional) ──────────────────────────────
BOX_GOOGLE_CLIENT_ID=
BOX_GOOGLE_CLIENT_SECRET=
BOX_OUTLOOK_CLIENT_ID=
BOX_OUTLOOK_CLIENT_SECRET=
EOF
sudo chmod 0600 /etc/warmbly/warmbly.env
sudo chown warmbly:warmbly /etc/warmbly/warmbly.env
```
A few lines differ from the compose defaults on purpose:
- `API_HOST`, `TRACKING_HOST` and the realtime `PORT` sit on `127.0.0.1`, so only nginx reaches them. The realtime service binds all interfaces regardless, so keep `4000` closed at the firewall
- `TRUSTED_PROXIES` and `TRACKING_TRUSTED_PROXIES` name the proxy, so rate limits and audit records see the visitor's address instead of `127.0.0.1`
- `CHECK_ORIGIN=true` makes the websocket refuse browsers that are not on `PHX_HOST`'s origin list
- `GEODB_PATH` is left unset. It only adds a city to sessions and audit rows; set it to a MaxMind `GeoLite2-City.mmdb` if you have one
Every other variable, with its default, is in the [configuration reference](/development/configuration/), and [`deploy/config/env.example`](https://github.com/warmbly/warmbly/blob/main/deploy/config/env.example) is the annotated template this file was cut down from. Mail relay, OAuth clients, single sign-on and the AI provider are configured exactly as in the [self-hosting guide](/development/deployment-guide/#platform-email); only the way values reach the process differs.
</Step>
<Step>
### Write `/etc/warmbly/worker.env`
The worker is deliberately blind to the database: it reaches encrypted keys over the backend's internal API and holds only what it needs to send. Give it its own file:
```bash
sudo tee /etc/warmbly/worker.env >/dev/null <<EOF
APP_ENV=prod
AWS_CONFIG_ENABLED=false
WORKER_ID=$(uuidgen)
WORKER_TIER=shared_premium
EVENTBUS_PROVIDER=nats
NATS_URL=nats://127.0.0.1:4222
CODEC_PROVIDER=json
REDIS=redis://127.0.0.1:6379
ENCRYPTED_KEYS_PROVIDER=http
ENCRYPTED_KEYS_BACKEND_URL=http://127.0.0.1:8080
ENCRYPTED_KEYS_WORKER_TOKEN=${INTERNAL_API_TOKEN}
KMS_PROVIDER=local
KMS_LOCAL_MASTER_KEY=${KMS_LOCAL_MASTER_KEY}
CREDENTIALS_ENCRYPTION_KEY=${CREDENTIALS_ENCRYPTION_KEY}
BLOB_PROVIDER=filesystem
BLOB_FS_ROOT=/var/lib/warmbly/blobs
BOX_GOOGLE_CLIENT_ID=
BOX_GOOGLE_CLIENT_SECRET=
BOX_OUTLOOK_CLIENT_ID=
BOX_OUTLOOK_CLIENT_SECRET=
EOF
sudo chmod 0600 /etc/warmbly/worker.env
sudo chown warmbly:warmbly /etc/warmbly/worker.env
```
`WORKER_ID` is generated once and never changed: it is the worker's identity, and the mailboxes assigned to it follow that UUID. A worker that boots with a fresh id every time leaves the old one behind still holding its mailboxes, and sending stalls until the reconciler notices. The `BOX_*` values must match the backend's, because the worker is what refreshes an expiring OAuth token.
</Step>
</Steps>
## Run it
<Steps>
<Step>
### Install the units
The repository ships one unit per service in [`deploy/systemd/`](https://github.com/warmbly/warmbly/tree/main/deploy/systemd), already pointed at the paths above and locked down to the `warmbly` user:
```bash
sudo cp /opt/warmbly/src/deploy/systemd/warmbly-*.service /etc/systemd/system/
sudo systemctl daemon-reload
```
</Step>
<Step>
### Start the backend first
The backend applies migrations on boot, and everything else expects the schema to exist:
```bash
sudo systemctl enable --now warmbly-backend
sudo journalctl -u warmbly-backend -f
```
Watch for the migrations to finish and `curl http://127.0.0.1:8080/health` to answer `200`. A `prod` boot refuses any of the published development secrets, so a message naming a variable means the env file still has a placeholder in it.
To apply migrations without starting the API, for instance in a deploy script, `sudo -u warmbly env $(grep PRIMARY_DB /etc/warmbly/warmbly.env) /opt/warmbly/bin/migrate` runs the same embedded migrations and exits.
</Step>
<Step>
### Start the rest
```bash
sudo systemctl enable --now warmbly-consumer warmbly-tracking warmbly-realtime warmbly-worker
systemctl status 'warmbly-*' --no-pager
```
Each answers on its own port:
```bash
curl -s -o /dev/null -w '%{http_code}\n' http://127.0.0.1:8080/health # backend
curl -s -o /dev/null -w '%{http_code}\n' http://127.0.0.1:3000/health # tracking
curl -s -o /dev/null -w '%{http_code}\n' http://127.0.0.1:4000/health # realtime
```
The consumer and worker have no port; `journalctl -u warmbly-worker` shows the worker registering and heartbeating, and it appears in the admin panel under Workers once you can sign in.
</Step>
<Step>
### Put nginx in front
[`deploy/nginx/warmbly.conf`](https://github.com/warmbly/warmbly/blob/main/deploy/nginx/warmbly.conf) serves the two static frontends and proxies the API, websocket and tracking hosts. Copy it, replace `example.com`, and obtain certificates:
```bash
sudo cp /opt/warmbly/src/deploy/nginx/warmbly.conf /etc/nginx/sites-available/warmbly.conf
sudo sed -i 's/example\.com/yourdomain.com/g' /etc/nginx/sites-available/warmbly.conf
sudo ln -s /etc/nginx/sites-available/warmbly.conf /etc/nginx/sites-enabled/
sudo certbot --nginx -d app.yourdomain.com -d admin.yourdomain.com -d api.yourdomain.com -d ws.yourdomain.com -d t.yourdomain.com
sudo nginx -t && sudo systemctl reload nginx
```
The frontends are plain files under `/opt/warmbly/web` and `/opt/warmbly/admin` with a history fallback to `index.html`, and `index.html` and `config.js` are served with `Cache-Control: no-store` so a redeploy is picked up on the next load. Any other web server can do the same; the config file has the exact headers.
</Step>
<Step>
### Claim the instance
On its first boot with an empty `users` table the backend prints a single-use setup link to its log:
```bash
sudo journalctl -u warmbly-backend | grep -o 'http[^ ]*/setup?token=[a-f0-9]*' | tail -1
```
Open it at `https://app.yourdomain.com`, pick a password, and you are the owner and platform admin. If the line has scrolled away, `warmblyctl setup-link` prints a fresh one:
```bash
sudo -u warmbly env $(grep -E '^(PRIMARY_DB|REDIS|AUTH_SECRET|APP_URL)=' /etc/warmbly/warmbly.env | xargs) warmblyctl setup-link
```
`warmblyctl` is the same operator CLI the container image ships; it reads `PRIMARY_DB` and friends from the environment, so the `env $(grep ...)` prefix above is how every command on the [warmblyctl reference](/development/warmblyctl/) is run on a bare host. A shell alias saves typing it:
```bash
alias warmblyctl='sudo -u warmbly env $(sudo grep -E "^(PRIMARY_DB|REDIS|AUTH_SECRET|APP_URL|API_PUBLIC_URL|MAIL_TRANSPORT|SMTP_HOST|EMAIL_ADDRESS|EMAIL_NAME)=" /etc/warmbly/warmbly.env | xargs) /opt/warmbly/bin/warmblyctl'
warmblyctl status
```
[First run](/development/first-run/) covers what to do when the database already has accounts, and unattended provisioning through `WARMBLY_BOOTSTRAP_EMAIL`.
</Step>
</Steps>
## Workers on other machines
The point of workers is to spread sending across machine identities, so most installs eventually add a worker on another host. Two routes, neither needing Docker on the control plane:
**The enrollment installer**, which is what the admin panel's Add Worker flow produces, runs the worker as a container on the remote host. It needs Docker only there. The backend serves it at `GET /worker-install.sh` from `WORKER_INSTALLER_PATH`, which the shipped unit already points at the checkout.
**A native worker** is the same binary and env file as above, with the addresses changed. The backend renders a complete env file for an enrollment token, so nothing has to be copied by hand:
```bash
# On the remote host, with /opt/warmbly/bin/worker built for it:
curl -fsS https://api.yourdomain.com/api/v1/workers/enroll \
-H 'Content-Type: application/json' \
-d '{"token":"wmenroll_..."}' | sudo tee /etc/warmbly/worker.env >/dev/null
echo "WORKER_ID=$(uuidgen)" | sudo tee -a /etc/warmbly/worker.env >/dev/null
sudo chmod 0600 /etc/warmbly/worker.env
sudo cp deploy/systemd/warmbly-worker.service /etc/systemd/system/
sudo systemctl daemon-reload && sudo systemctl enable --now warmbly-worker
```
The token is consumed on first use and the response carries the decryption material, so do this over HTTPS only. Before enrolling anything, set `API_PUBLIC_URL`, `NATS_URL` and `REDIS` on the backend to addresses the remote host can reach, and open those ports to it: the rendered file inherits the backend's own values, and `127.0.0.1` does not resolve to your control plane from another machine. Use `BLOB_PROVIDER=s3` with a bucket both sides can reach; a remote worker on the filesystem provider writes to its own disk. The [self-hosting guide](/development/deployment-guide/#remote-workers) has the same caveats in more detail.
Because the worker is not in a container, the admin panel's SSH-driven day-two actions (pull image, restart container) do not apply to it. Manage it with `systemctl` and the update steps below.
## Upgrading
Rebuild the artifacts that changed, then restart. Migrations are forward-only and apply on backend boot, so bring the backend up before the rest:
```bash
cd /opt/warmbly/src && sudo git pull # or checkout a newer tag
# repeat the build steps for the services that changed, then:
sudo systemctl restart warmbly-backend
sudo systemctl restart warmbly-consumer warmbly-tracking warmbly-realtime warmbly-worker
```
Frontend rebuilds overwrite `config.js` when you copy `dist/` over, so rewrite it afterwards (or keep the two files somewhere and copy them back). A build script that does all of it in order is worth writing the second time you upgrade.
## Backups
Three things, all three:
```bash
sudo -u postgres pg_dump warmbly > backup.sql # 1. the database
sudo tar czf blobs.tgz -C /var/lib/warmbly blobs # 2. uploads and stored message bodies
sudo cp /etc/warmbly/*.env somewhere-safe/ # 3. the env files, above all the two encryption keys
```
Restore into an empty database before the backend starts, then start it and it applies only what is newer than the dump. See [restoring](/development/deployment-guide/#restoring); replace the `docker compose exec` commands with plain `psql`.
## Troubleshooting
| Symptom | Cause |
|---------|-------|
| Backend exits at boot naming `AUTH_SECRET` or another variable | `APP_ENV=prod` refuses the published development defaults; the env file still holds a placeholder |
| First publish fails with a stream or JetStream error | `nats-server` runs without `-js` |
| `mkdir /var/lib/warmbly/blobs/...: permission denied` on the first send | `BLOB_FS_ROOT` is not writable by the `warmbly` user, or the unit's `ReadWritePaths` does not cover it |
| Realtime unit starts and exits immediately | A release only listens when `PHX_SERVER=true`; the shipped unit sets it. Otherwise `journalctl -u warmbly-realtime` names the missing variable |
| Realtime raises `JWT_SECRET ... required` | Prod releases read `JWT_SECRET`, `SECRET_KEY_BASE` and `DATABASE_URL`, not the backend's names; keep both spellings in the file |
| Dashboard loads but every request is `403` | `CORS_ALLOW_ORIGINS` does not list the origin the browser is on; it must match the `APP_URL` you wrote into `config.js` |
| Dashboard never goes live, notifications stay silent | `WEBSOCKET_URL` is wrong, or `CHECK_ORIGIN=true` with a `PHX_HOST` that is not the dashboard's host |
| Worker logs `encryptedkeys.http: ... unexpected status 401` | `ENCRYPTED_KEYS_WORKER_TOKEN` does not equal the backend's `INTERNAL_API_TOKEN` |
| Sends fail with `email account not found in worker` | The worker booted with a new `WORKER_ID`; the mailboxes still belong to the old one until the reconciler moves them. Pin the id |
| Opens and clicks are not recorded | `TRACKING_DOMAIN` does not match the host nginx proxies to `:3000`, or `BACKEND_INTERNAL_URL` is unreachable from the tracking service |
| Mailbox connects, then fails about an hour later | The worker env is missing the `BOX_*` OAuth client; only the worker refreshes tokens |
`warmblyctl status` runs the same instance checks the admin panel's System Status page shows and exits non-zero on anything at error severity. The [troubleshooting](/development/troubleshooting/) page covers symptoms that are not specific to running without Docker.
## Quick reference
```
/opt/warmbly/bin/{backend,consumer,worker,tracking,migrate,warmblyctl}
/opt/warmbly/realtime/bin/realtime mix release
/opt/warmbly/web, /opt/warmbly/admin static builds + config.js
/opt/warmbly/src the checkout (worker installer is served from here)
/etc/warmbly/warmbly.env backend, consumer, tracking, realtime
/etc/warmbly/worker.env worker
/var/lib/warmbly/blobs BLOB_FS_ROOT
/etc/systemd/system/warmbly-*.service from deploy/systemd/
/etc/nginx/sites-available/warmbly.conf from deploy/nginx/
```
```bash
systemctl status 'warmbly-*'
journalctl -u warmbly-backend -f
warmblyctl status # with the alias above
warmblyctl setup-link
```
See also: [self-hosting with Docker](/development/deployment-guide/), [configuration reference](/development/configuration/), [warmblyctl](/development/warmblyctl/), [architecture](/development/architecture/).