Files
windmill/docker-compose.yml
T
Ruben FiszelandClaude Opus 5 af15a73b8b chore: move the compose stack to postgres 18 (#10827)
* chore: move the compose stack to postgres 18

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Ha6ovKdT9XRoj5FyVe7fEt

* fix: dump the whole cluster in the postgres 18 upgrade recipe

Windmill creates instance datatable, DuckLake and wm_fork_* databases in the
same cluster as windmill, so a single-database pg_dump followed by removing the
volume loses them silently. Dump the cluster with pg_dumpall instead, which also
carries the roles the RLS policies are granted to, with their passwords.

Also wait on the healthcheck before restoring, stop services generically rather
than by name, and ANALYZE after the restore.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Ha6ovKdT9XRoj5FyVe7fEt

* fix: analyze every restored database and check the restore for errors

ANALYZE is per-database, so the sibling datatable/DuckLake/wm_fork_* databases the
recipe now restores were left with no planner statistics; vacuumdb --all covers
them. psql does not stop on error and the old volume is gone by that point, so
the restore needs an explicit grep rather than a trusted exit code.

Also note that logical replication slots are never dumped, so a Postgres trigger
reading a database in this cluster comes back disabled until it is re-saved.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Ha6ovKdT9XRoj5FyVe7fEt

* fix: drop the bootstrapped windmill database before the restore

POSTGRES_DB creates an empty windmill database, so the dump's own CREATE DATABASE
for it fails and its objects load into the entrypoint's database instead, keeping
the new cluster's encoding and collation rather than the dumped ones. Sibling
databases are created by the dump and so were never affected. Dropping it first
makes the restore reproduce the source cluster exactly, and leaves one expected
error instead of two.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Ha6ovKdT9XRoj5FyVe7fEt

* docs: move the postgres 18 upgrade runbook out of the compose file

A step-by-step runbook in a config file needed corrections in three consecutive
review rounds, which is the argument for keeping it somewhere it can be fixed
once. The comment keeps only the constraint a reader has to know before touching
the mount, plus a link.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Ha6ovKdT9XRoj5FyVe7fEt

* docs: point the postgres 18 upgrade note at windmill.dev

GitHub gists are owned by user accounts, never organisations, so a gist is the
wrong home for the only migration instructions every self-hosted operator gets.
The procedure now lives in the self-host docs page instead.

Depends on windmill-labs/windmilldocs#1704 merging and deploying first.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Ha6ovKdT9XRoj5FyVe7fEt

---------

Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-26 22:14:27 +02:00

239 lines
9.0 KiB
YAML

version: "3.7"
x-logging: &default-logging
driver: "json-file"
options:
max-size: "${LOG_MAX_SIZE:-20m}"
max-file: "${LOG_MAX_FILE:-10}"
compress: "true"
services:
## UPGRADING FROM POSTGRES 16: db_data holds a cluster 18 cannot read, so the
## container exits with an explanatory error rather than coming up blank. Migrating
## means dumping the WHOLE cluster (pg_dumpall), never just the windmill database:
## Windmill keeps datatable, DuckLake and wm_fork_* databases beside it and grants
## its RLS policies to cluster-level roles, and a single-database dump loses both
## silently. Full procedure, and why 16 is still a valid choice until Nov 2028:
## https://www.windmill.dev/docs/advanced/self_host#upgrade-postgresql-to-18
db:
deploy:
# To use an external database, set replicas to 0 and set DATABASE_URL to the external database url in the .env file
replicas: 1
image: postgres:18
shm_size: 1g
restart: unless-stopped
volumes:
# From 18 on the official image keeps the cluster in a major-version
# subdirectory (/var/lib/postgresql/18/docker), so the mount has to be the
# parent directory: that is what lets pg_upgrade see an old and a new
# cluster inside a single mount point. Mounting the pre-18 .../data path
# instead makes the image exit rather than start, which is what turns a
# stale 16 cluster into a loud failure instead of an empty instance.
- db_data:/var/lib/postgresql
expose:
- 5432
environment:
POSTGRES_PASSWORD: changeme
POSTGRES_DB: windmill
healthcheck:
test: ["CMD-SHELL", "pg_isready -U postgres"]
interval: 10s
timeout: 5s
retries: 5
logging: *default-logging
windmill_server:
image: ${WM_IMAGE}
pull_policy: always
deploy:
replicas: 1
restart: unless-stopped
expose:
- 8000
- 2525
environment:
- DATABASE_URL=${DATABASE_URL}
- MODE=server
depends_on:
db:
condition: service_healthy
volumes:
- worker_logs:/tmp/windmill/logs
logging: *default-logging
windmill_worker:
image: ${WM_IMAGE}
pull_policy: always
deploy:
replicas: 3
resources:
limits:
memory: 2048M
# for GB, use syntax '2Gi'
restart: unless-stopped
# Uncomment to enable PID namespace isolation (recommended for security)
# Requires privileged mode for --mount-proc flag
# See: https://www.windmill.dev/docs/advanced/security_isolation
privileged: true
environment:
- DATABASE_URL=${DATABASE_URL}
- MODE=worker
- WORKER_GROUP=default
# If running with non-root/non-windmill UID (e.g., user: "1001:1001"),
# add: - HOME=/tmp
- FAVOR_UNSHARE_PID=true
depends_on:
db:
condition: service_healthy
# to mount the worker folder to debug, KEEP_JOB_DIR=true and mount /tmp/windmill
volumes:
- worker_dependency_cache:/tmp/windmill/cache
- worker_logs:/tmp/windmill/logs
## Sandboxed containers (`# sandbox <image>`) run daemonless via crane + nsjail
## inside the worker itself — no Docker socket or dind sidecar required.
## For the legacy full-compat docker (a bare `# docker`, trusted users only),
## mount the host Docker socket by uncommenting the line below. WARNING: this
## grants user scripts full access to the host Docker daemon (host filesystem
## access and privilege escalation) — only use it if you fully trust all users.
# - /var/run/docker.sock:/var/run/docker.sock
logging: *default-logging
## This worker is specialized for "native" jobs. Native jobs run in-process and thus are much more lightweight than other jobs
windmill_worker_native:
# Use ghcr.io/windmill-labs/windmill-ee:main for the ee
image: ${WM_IMAGE}
pull_policy: always
deploy:
replicas: 1
resources:
limits:
memory: 2048M
# for GB, use syntax '2Gi'
restart: unless-stopped
# Uncomment to enable PID namespace isolation (recommended for security)
# Requires privileged mode for --mount-proc flag
# See: https://www.windmill.dev/docs/advanced/security_isolation
environment:
- DATABASE_URL=${DATABASE_URL}
- MODE=worker
- WORKER_GROUP=native
- NATIVE_MODE=true
- SLEEP_QUEUE=200
depends_on:
db:
condition: service_healthy
volumes:
- worker_logs:/tmp/windmill/logs
logging: *default-logging
# This worker is specialized for reports or scraping jobs. It is assigned the "reports" worker group which has an init script that installs chromium and can be targeted by using the "chromium" worker tag.
# windmill_worker_reports:
# image: ${WM_IMAGE}
# pull_policy: always
# deploy:
# replicas: 1
# resources:
# limits:
# memory: 2048M
# # for GB, use syntax '2Gi'
# restart: unless-stopped
# # Uncomment to enable PID namespace isolation (recommended for security)
# # Requires privileged mode for --mount-proc flag
# # See: https://www.windmill.dev/docs/advanced/security_isolation
# privileged: true
# environment:
# - DATABASE_URL=${DATABASE_URL}
# - MODE=worker
# - WORKER_GROUP=reports
# - FAVOR_UNSHARE_PID=true
# depends_on:
# db:
# condition: service_healthy
# # to mount the worker folder to debug, KEEP_JOB_DIR=true and mount /tmp/windmill
# volumes:
# - worker_dependency_cache:/tmp/windmill/cache
# - worker_logs:/tmp/windmill/logs
# The indexer powers full-text job and log search, an EE feature.
windmill_indexer:
image: ${WM_IMAGE}
pull_policy: always
deploy:
replicas: 0 # set to 1 to enable full-text job and log search
restart: unless-stopped
expose:
- 8002
environment:
- PORT=8002
- DATABASE_URL=${DATABASE_URL}
- MODE=indexer
depends_on:
db:
condition: service_healthy
volumes:
- windmill_index:/tmp/windmill/search
- worker_logs:/tmp/windmill/logs
logging: *default-logging
# Combined extra services: LSP, Multiplayer, and Debugger
# Each service can be enabled/disabled via environment variables:
# - ENABLE_LSP=true (default) - Language Server Protocol for code intelligence
# - ENABLE_MULTIPLAYER=false - Real-time collaboration (Enterprise Edition)
# - ENABLE_DEBUGGER=false - Interactive debugging via DAP WebSocket
windmill_extra:
image: ghcr.io/windmill-labs/windmill-extra:latest
pull_policy: always
restart: unless-stopped
expose:
- 3001 # LSP
- 3002 # Multiplayer
- 3003 # Debugger
environment:
- ENABLE_LSP=true
- ENABLE_MULTIPLAYER=false # Set to true to enable multiplayer (Enterprise Edition)
- ENABLE_DEBUGGER=true # Set to true to enable debugger
- DEBUGGER_PORT=3003 # Debugger service port
- ENABLE_NSJAIL=false # Set to true for nsjail sandboxing (requires privileged: true)
- REQUIRE_SIGNED_DEBUG_REQUESTS=true # Require backend-signed JWT tokens for debug sessions. Do NOT set to false on any internet-reachable deployment: it exposes an unauthenticated code-execution debugger.
- WINDMILL_BASE_URL=http://windmill_server:8000
# - DEBUG_ALLOWED_ORIGINS=https://your-windmill-host # Optional CSWSH hardening: comma-separated allowlist of browser Origins permitted to open debug WebSockets
volumes:
- lsp_cache:/pyls/.cache
# Behind a TLS-intercepting proxy, mount its CA here (as .crt) and it is registered in the
# system trust store before any service starts. That alone does not cover dependency
# installation — see debugger/README.md for the variables it also needs
# - ./corp-ca.crt:/usr/local/share/ca-certificates/corp-ca.crt:ro
logging: *default-logging
caddy:
# Pinned: this image and the ./Caddyfile next to it are version-coupled, so
# they have to move together. Bump docker/caddy-l4.version in the same
# commit as any Caddyfile change.
image: ghcr.io/windmill-labs/caddy-l4:2.11.4-1
restart: unless-stopped
# Configure the mounted Caddyfile and the exposed ports or use another reverse proxy if needed
volumes:
- ./Caddyfile:/etc/caddy/Caddyfile
- caddy_data:/data
# - ./certs:/certs # Provide custom certificate files like cert.pem and key.pem to enable HTTPS - See the corresponding section in the Caddyfile
ports:
# To change the exposed port, simply change 80:80 to <desired_port>:80. No other changes needed
- 80:80
- 25:25
# - 443:443 # Uncomment to enable HTTPS handling by Caddy
environment:
- BASE_URL=":80"
# - BASE_URL=":443" # uncomment and comment line above to enable HTTPS via custom certificate and key files
# - BASE_URL=mydomain.com # Uncomment and comment line above to enable HTTPS handling by Caddy
logging: *default-logging
volumes:
db_data: null
worker_dependency_cache: null
worker_logs: null
worker_memory: null
windmill_index: null
lsp_cache: null
caddy_data: null