mirror of
https://github.com/windmill-labs/windmill.git
synced 2026-09-12 16:05:43 +00:00
Ruben Fiszel
54217846a9
Existing tests only verified the algorithm's outputs on fabricated activity tables. This adds a real end-to-end simulation that drives 50 concurrent mock workers (async tasks doing the actual pull → mark running → sleep → complete cycle over real v2_job_queue rows) against sustained diverse traffic from one noisy + five victim workspaces. Each scenario runs twice — once with the fairness algorithm disabled (control) and once enabled (treatment) — and asserts the QoL improvement on the victim workspaces is substantial. Setup: - Pool: dedicated 80-connection pool (sqlx::test's 10-conn default serialised the workers and masked the algorithm's effect). - Backlog: 1500 pre-populated noisy queue rows + 200 noisy completions in the rolling window, so workers start saturated and the first fairness refresh already has a dominant-workspace signal to act on. - Sustained load: 4 noisy pushers running flat out (no inter-insert sleep), 3 medium victims (10 jobs/s) + 2 quiet victims (4 jobs/s), each with 60-100 ms job durations, for 5 seconds. - Refresh: 250 ms cadence, mirroring `maybe_refresh_overloaded`. Observed results (stable across 3 consecutive runs): - Control: victim completion rate ~28%, p95 latency ~15s - Treatment: victim completion rate 100%, p95 latency ~200ms - ~65x p95 improvement, ~3.5x completion-rate improvement Headline assertion accepts either ≥5x p95 improvement OR ≥1.5x completion-rate improvement, so the test stays green across CI machines whose relative speeds shift the balance between the two signals. Total runtime ~23s, marked `#[ignore]` to keep the default `cargo test` profile sub-second.
Windmill Backend
This folder holds all backend components, the src/ folder only contains files used to build the "root" binary.
Components
| name | description |
|---|---|
| windmill-api | The API server, exposing functionality to other components and the frontend |
| windmill-audit | Contains audit functionality, allowing different components to record important actions |
| windmill-common | Common code shared by all crates |
| windmill-queue | Contains job & flow queuing functionality, commonly written to by the API server and read from by workers |
| windmill-worker | The worker. Used to process and execute flows & jobs. |
| parsers | Contains code to parse signatures in different langauges. |
Compile sqlx for offline ci
cargo sqlx prepare --workspace -- --bin windmill --features enterprise