diff --git a/README.md b/README.md index cfeb4e65ce..ddc25459f7 100644 --- a/README.md +++ b/README.md @@ -234,9 +234,13 @@ Go to http://localhost - default credentials: `admin@windmill.dev` / `changeme` > etc.) stay root-owned — unreadable by a `docker run --pid=host` container. What this > does **not** do is confine the container's *filesystem* view: the daemon runs outside > the job's nsjail sandbox, so a `# docker` script can still `docker run -v ` -> to reach other concurrent job dirs / caches. So treat docker-capable workers as a -> **trusted-tenant** capability (a big improvement over privileged dind, but not a full -> sandbox) and prefer dedicated/per-workspace docker workers on shared multi-tenant fleets. +> to reach **world-readable** host state: other concurrent job dirs' world-readable files, +> the dependency cache, and world-readable `/proc` (e.g. `-v /proc` → process `cmdline`s / +> args) — though **not** `0400` files like `/proc//environ`, so the worker's env +> secrets stay protected. (Corollary: don't pass secrets as command-line args; windmill +> uses env, not args.) So treat docker-capable workers as a **trusted-tenant** capability +> (a big improvement over privileged dind, but not a full sandbox) and prefer +> dedicated/per-workspace docker workers on shared multi-tenant fleets. > If you enable **nsjail** (it's off by default), per-job podman is **not** auto-provided > for docker jobs — nsjail means "fully sandbox jobs" and the daemon runs outside the jail, > so docker jobs then require an explicit `DOCKER_HOST` or a separate non-nsjail worker group. diff --git a/backend/windmill-worker/src/bash_executor.rs b/backend/windmill-worker/src/bash_executor.rs index 17f91be311..27765445f3 100644 --- a/backend/windmill-worker/src/bash_executor.rs +++ b/backend/windmill-worker/src/bash_executor.rs @@ -654,14 +654,14 @@ fn spawn_docker_storage_monitor( // SECURITY: this is only ever started in NON-nsjail modes (unshare/none) — caller // refuses to auto-provide it under nsjail, since the daemon runs outside any sandbox. // A `# docker` script controls this daemon over the Docker API and can bind-mount any -// path the daemon user can read (e.g. `docker run -v /tmp/windmill/...`), reaching -// other concurrent job dirs / caches — so it is NOT a filesystem sandbox, and -// docker-capable workers stay a TRUSTED-TENANT capability (prefer dedicated/per- +// WORLD-READABLE host path (e.g. `docker run -v /tmp/windmill/...` for other job dirs / +// caches, or `-v /proc` for process cmdlines/args) — so it is NOT a filesystem sandbox, +// and docker-capable workers stay a TRUSTED-TENANT capability (prefer dedicated/per- // workspace docker workers on shared fleets). What IS protected, when the worker runs -// as root (the default) and the daemon is dropped to a non-root uid here: the worker's -// secrets in `/proc//environ` (DATABASE_URL etc.) stay root-owned and -// unreadable by the uid-1000 container even via `docker run --pid=host`, and an escape -// is unprivileged. +// as root (the default) and the daemon is dropped to a non-root uid here: `0400` files +// like the worker's `/proc//environ` (DATABASE_URL etc.) stay root-owned and +// unreadable by the uid-1000 container even via `-v /proc` or `--pid=host` (so don't +// pass secrets as command-line args), and a container escape is unprivileged. #[cfg(feature = "dind")] async fn start_per_job_podman( job_dir: &str, diff --git a/docker-compose.yml b/docker-compose.yml index e0b1480152..a31755f660 100644 --- a/docker-compose.yml +++ b/docker-compose.yml @@ -70,8 +70,11 @@ services: # rootless AND the worker's own secrets in /proc (DATABASE_URL etc.) stay # root-owned, unreadable by a `docker run --pid=host` container. (NB: this does # not confine the container's filesystem view — a `# docker` script can still - # `docker run -v` worker-visible paths, so treat docker workers as trusted-tenant - # / dedicate them on shared fleets.) Cap per-job image storage via the + # `docker run -v` to read WORLD-READABLE host state: other job dirs' world-readable + # files, the dep cache, and world-readable /proc like process cmdlines/args — though + # NOT 0400 files like /proc//environ (env secrets stay safe; don't pass secrets + # as args). So treat docker workers as trusted-tenant / dedicate them on shared + # fleets.) Cap per-job image storage via the # `docker_image_storage_size_mb` instance setting (default 8GB). # If you enable nsjail (DISABLE_NSJAIL=false or job isolation = nsjail), per-job # podman is NOT auto-provided for docker jobs — nsjail signals "fully sandbox jobs"