Files
windmill/backend/windmill-queue/tests
Ruben Fiszel ddec2abbb3 feat(jobs): cap total queued jobs per workspace on cloud (#10218)
* feat(jobs): cap total queued jobs per workspace on cloud

A workspace could flood the queue with an unbounded number of jobs across
many concurrency keys and scripts (or keyless jobs), which the per-key
cap from #10197 does not bound. Add a companion instance-wide ceiling on
a workspace's total queued jobs.

check_workspace_queue_cap rejects a push once the workspace has
WORKSPACE_MAX_QUEUED_JOBS (default 20000, superadmin-configurable, 0 to
disable) jobs queued, cloud-only and runtime-gated on CLOUD_HOSTED like
the per-key cap. It runs on every push, so it applies even to premium
workspaces and catches parallel for-loop floods. Jobs already queued
still drain; only new pushes past the ceiling are rejected, so an
in-flight flow only fails to push further work while at the ceiling.

The setting loader self-gates on CLOUD_HOSTED so it is never loaded off
cloud, from initial load or a settings-change reload. The depth count is
bounded by the cap via LIMIT so a runaway backlog never costs an
unbounded scan on the push path.

* docs(jobs): note the workspace cap is a soft ceiling and the depth helper is count-only

Records the two review points as constraints: the cap does not serialize
admission (a soft ceiling by design, like the per-key cap), and
workspace_queue_depth is pub only for the test, returns a count not job
data, and leaves authorization to the caller.
2026-07-20 22:09:59 +02:00
..