Files
windmill/docker-compose.yml
T
Ruben Fiszel 1727271e19 feat: sandboxed daemonless container runtime via '# sandbox <image>' (#9453)
* feat: add sandboxed docker v2 runtime via '# docker <image>'

Run a container image as a subprogram of the job's own nsjail sandbox:
extract the image rootfs with podman (rootless) and run it chrooted inside the
job's nsjail, so the container inherits the job's confinement and is safe under
nsjail / for untrusted code. Selected by '# docker <image>'; a bare '# docker'
keeps the v1 (dind) path untouched.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* feat: default to daemonless docker (drop dind from compose, allow docker on cloud)

docker-compose no longer ships the dind sidecar (v2 is daemonless: podman + nsjail
in the worker); removed the dind service, DOCKER_HOST env, depends_on and volume.
Removed the language-picker guard that blocked Docker scripts on the multi-tenant
platform, now that v2 makes docker safe to run sandboxed.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* feat: select sandboxed container via # sandbox <image>; add pull policy + size guards

- Surface moved from '# docker <image>' to '# sandbox <image>' (groups under the
  sandbox annotation; '# docker' stays v1-only, '# sandbox' stays nsjail-bash).
- SANDBOX_IMAGE_PULL_POLICY (default 'newer') so moving tags don't go stale.
- SANDBOX_IMAGE_MAX_SIZE_MB rejects oversized images before extraction.
- SANDBOX_IMAGE_CACHE_MAX_MB best-effort LRU eviction of podman's image store.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* feat(sandbox): support # volume, honor nsjail tmp instance settings, v2 docker template

- Thread shared_mount into the sandbox container nsjail config so '# volume' mounts
  (and the same-worker /tmp/shared folder) apply inside the container.
- Use resolve_nsjail_tmp_mount_block for the container's /tmp so it honors the same
  nsjail_tmp_backing / nsjail_tmpfs_size_mb instance settings as other nsjail jobs.
- docker-compose comment + the editor's Docker template now use '# sandbox <image>'.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* feat(sandbox): make image size/cache/pull-policy UI instance settings

Convert SANDBOX_IMAGE_* from worker env vars to DB-backed instance settings
(sandbox_image_max_size_mb, sandbox_image_cache_max_mb, sandbox_image_pull_policy),
hot-reloaded via the same mechanism as nsjail_tmpfs_size_mb and configurable in
#superadmin-settings. No worker restart needed.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* feat(sandbox): windmill-managed registry — default registry + private auth

Two new instance settings:
- sandbox_image_default_registry: prepended to unqualified image refs (alpine ->
  <registry>/alpine); fully-qualified refs untouched.
- sandbox_registry_auth: docker/podman auth.json blob written to a per-job authfile
  (0600, removed with the job) and passed to podman --authfile for private registries.
Both hot-reloaded and configurable in #superadmin-settings.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* fix(sandbox): protobuf-safe proto_str escaper, atomic 0600 authfile, registry tests

Addresses local-review P2s: proto_str now emits valid protobuf octal escapes for
control/non-ASCII bytes (not Rust \u{..} that nsjail would reject); the registry
authfile is created 0600 atomically (no world-readable window); add a
registry_qualified table test + a non-ASCII proto_str case.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* fix(sandbox): P0 — deliver image env via nsjail envar:, never the launcher process env

CI review (P0): the image's OCI Env (attacker-controlled keys+values) was applied to
the nsjail launcher process via .envs(), so a hostile image could set LD_PRELOAD/
LD_LIBRARY_PATH/LD_AUDIT on nsjail itself and execute code as the worker outside the
jail. Now the image env is rendered as proto-escaped 'envar:' directives (child-only)
and nsjail's process env carries only windmill-trusted keys (reserved vars + proxy).
Also: warn instead of silently bypassing the size guard on inspect failure; reset the
eviction guard via a Drop guard (no stuck flag on panic/early-return). +render_envars test.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* fix(sandbox): P0 symlink-write escape via rootfs script; P1 redact registry-auth logging

CI review:
- P0 (Codex): the body was written into the image-controlled rootfs as
  .windmill_docker_main.sh via write_file (follows symlinks) — a hostile image could
  plant that path as a symlink to a host file and capture the worker's write before
  nsjail starts. Now the body is passed straight to 'sh -c <body> sh <args>'; no file
  is written into the rootfs at all.
- P1 (Codex): sandbox_registry_auth flowed through the generic setting loader which
  logs the value (raw auth.json credentials). Replaced with a secret-aware reload that
  loads directly and logs only a redacted 'configured=' message.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* fix(sandbox): redact sandbox_registry_auth in instance-settings write log too

The settings API also logs 'Set global setting <key> to <value>' via format_setting_value;
add sandbox_registry_auth to SENSITIVE_SETTINGS so the credential is redacted there as
well as on reload.

* fix(sandbox): don't silently disable cache eviction on podman images parse error

Re-review (cubic/Claude P2): serde_json::from_slice(...).unwrap_or_default() meant any
parse hiccup (e.g. podman omitting Size/Created via omitempty for a zero value, or
schema drift) silently degraded to an empty Vec and disabled eviction with no log.
Now Size/Created are #[serde(default)] (a missing omitempty key -> 0, not a whole-array
parse failure) and a real parse error warns + breaks instead of being swallowed.

---------

Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-05 08:35:51 +00:00

218 lines
7.2 KiB
YAML

version: "3.7"
x-logging: &default-logging
driver: "json-file"
options:
max-size: "${LOG_MAX_SIZE:-20m}"
max-file: "${LOG_MAX_FILE:-10}"
compress: "true"
services:
db:
deploy:
# To use an external database, set replicas to 0 and set DATABASE_URL to the external database url in the .env file
replicas: 1
image: postgres:16
shm_size: 1g
restart: unless-stopped
volumes:
- db_data:/var/lib/postgresql/data
expose:
- 5432
environment:
POSTGRES_PASSWORD: changeme
POSTGRES_DB: windmill
healthcheck:
test: ["CMD-SHELL", "pg_isready -U postgres"]
interval: 10s
timeout: 5s
retries: 5
logging: *default-logging
windmill_server:
image: ${WM_IMAGE}
pull_policy: always
deploy:
replicas: 1
restart: unless-stopped
expose:
- 8000
- 2525
environment:
- DATABASE_URL=${DATABASE_URL}
- MODE=server
depends_on:
db:
condition: service_healthy
volumes:
- worker_logs:/tmp/windmill/logs
logging: *default-logging
windmill_worker:
image: ${WM_IMAGE}
pull_policy: always
deploy:
replicas: 3
resources:
limits:
memory: 2048M
# for GB, use syntax '2Gi'
restart: unless-stopped
# Uncomment to enable PID namespace isolation (recommended for security)
# Requires privileged mode for --mount-proc flag
# See: https://www.windmill.dev/docs/advanced/security_isolation
privileged: true
environment:
- DATABASE_URL=${DATABASE_URL}
- MODE=worker
- WORKER_GROUP=default
# If running with non-root/non-windmill UID (e.g., user: "1001:1001"),
# add: - HOME=/tmp
- FAVOR_UNSHARE_PID=true
depends_on:
db:
condition: service_healthy
# to mount the worker folder to debug, KEEP_JOB_DIR=true and mount /tmp/windmill
volumes:
- worker_dependency_cache:/tmp/windmill/cache
- worker_logs:/tmp/windmill/logs
## Sandboxed containers (`# sandbox <image>`) run daemonless via podman + nsjail
## inside the worker itself — no Docker socket or dind sidecar required.
## For the legacy full-compat docker (a bare `# docker`, trusted users only),
## mount the host Docker socket by uncommenting the line below. WARNING: this
## grants user scripts full access to the host Docker daemon (host filesystem
## access and privilege escalation) — only use it if you fully trust all users.
# - /var/run/docker.sock:/var/run/docker.sock
logging: *default-logging
## This worker is specialized for "native" jobs. Native jobs run in-process and thus are much more lightweight than other jobs
windmill_worker_native:
# Use ghcr.io/windmill-labs/windmill-ee:main for the ee
image: ${WM_IMAGE}
pull_policy: always
deploy:
replicas: 1
resources:
limits:
memory: 2048M
# for GB, use syntax '2Gi'
restart: unless-stopped
# Uncomment to enable PID namespace isolation (recommended for security)
# Requires privileged mode for --mount-proc flag
# See: https://www.windmill.dev/docs/advanced/security_isolation
environment:
- DATABASE_URL=${DATABASE_URL}
- MODE=worker
- WORKER_GROUP=native
- NATIVE_MODE=true
- SLEEP_QUEUE=200
depends_on:
db:
condition: service_healthy
volumes:
- worker_logs:/tmp/windmill/logs
logging: *default-logging
# This worker is specialized for reports or scraping jobs. It is assigned the "reports" worker group which has an init script that installs chromium and can be targeted by using the "chromium" worker tag.
# windmill_worker_reports:
# image: ${WM_IMAGE}
# pull_policy: always
# deploy:
# replicas: 1
# resources:
# limits:
# memory: 2048M
# # for GB, use syntax '2Gi'
# restart: unless-stopped
# # Uncomment to enable PID namespace isolation (recommended for security)
# # Requires privileged mode for --mount-proc flag
# # See: https://www.windmill.dev/docs/advanced/security_isolation
# privileged: true
# environment:
# - DATABASE_URL=${DATABASE_URL}
# - MODE=worker
# - WORKER_GROUP=reports
# - FAVOR_UNSHARE_PID=true
# depends_on:
# db:
# condition: service_healthy
# # to mount the worker folder to debug, KEEP_JOB_DIR=true and mount /tmp/windmill
# volumes:
# - worker_dependency_cache:/tmp/windmill/cache
# - worker_logs:/tmp/windmill/logs
# The indexer powers full-text job and log search, an EE feature.
windmill_indexer:
image: ${WM_IMAGE}
pull_policy: always
deploy:
replicas: 0 # set to 1 to enable full-text job and log search
restart: unless-stopped
expose:
- 8002
environment:
- PORT=8002
- DATABASE_URL=${DATABASE_URL}
- MODE=indexer
depends_on:
db:
condition: service_healthy
volumes:
- windmill_index:/tmp/windmill/search
- worker_logs:/tmp/windmill/logs
logging: *default-logging
# Combined extra services: LSP, Multiplayer, and Debugger
# Each service can be enabled/disabled via environment variables:
# - ENABLE_LSP=true (default) - Language Server Protocol for code intelligence
# - ENABLE_MULTIPLAYER=false - Real-time collaboration (Enterprise Edition)
# - ENABLE_DEBUGGER=false - Interactive debugging via DAP WebSocket
windmill_extra:
image: ghcr.io/windmill-labs/windmill-extra:latest
pull_policy: always
restart: unless-stopped
expose:
- 3001 # LSP
- 3002 # Multiplayer
- 3003 # Debugger
environment:
- ENABLE_LSP=true
- ENABLE_MULTIPLAYER=false # Set to true to enable multiplayer (Enterprise Edition)
- ENABLE_DEBUGGER=true # Set to true to enable debugger
- DEBUGGER_PORT=3003 # Debugger service port
- ENABLE_NSJAIL=false # Set to true for nsjail sandboxing (requires privileged: true)
- REQUIRE_SIGNED_DEBUG_REQUESTS=false # Set to true to require JWT tokens for debug sessions
- WINDMILL_BASE_URL=http://windmill_server:8000
volumes:
- lsp_cache:/pyls/.cache
logging: *default-logging
caddy:
image: ghcr.io/windmill-labs/caddy-l4:latest
restart: unless-stopped
# Configure the mounted Caddyfile and the exposed ports or use another reverse proxy if needed
volumes:
- ./Caddyfile:/etc/caddy/Caddyfile
- caddy_data:/data
# - ./certs:/certs # Provide custom certificate files like cert.pem and key.pem to enable HTTPS - See the corresponding section in the Caddyfile
ports:
# To change the exposed port, simply change 80:80 to <desired_port>:80. No other changes needed
- 80:80
- 25:25
# - 443:443 # Uncomment to enable HTTPS handling by Caddy
environment:
- BASE_URL=":80"
# - BASE_URL=":443" # uncomment and comment line above to enable HTTPS via custom certificate and key files
# - BASE_URL=mydomain.com # Uncomment and comment line above to enable HTTPS handling by Caddy
logging: *default-logging
volumes:
db_data: null
worker_dependency_cache: null
worker_logs: null
worker_memory: null
windmill_index: null
lsp_cache: null
caddy_data: null