mirror of
https://github.com/windmill-labs/windmill.git
synced 2026-08-23 08:00:45 +00:00
Ruben Fiszel
8912e21d15
A customer's top-load query was the job_perms orphan sweep (cleanup_job_perms_orphaned: 6.9s mean, 41s max). The cost is discovery, not deletion (~2.1ms per row deleted): the NOT EXISTS anti-join seq-scans the whole job_perms heap to find a few orphans, and that scan tracks the heap's physical size. job_perms / job_result_stream_v2 get one row per job and are drained only by these per-cycle sweeps, so they churn hard — but the bulk vacuuming_tables() runs only ~hourly, so dead tuples bloat the heap between bulk vacuums. Reclaim right after each sweep instead: VACUUM (SKIP_LOCKED) the swept table when it deleted rows. Plain VACUUM (not FULL) takes only SHARE UPDATE EXCLUSIVE so concurrent job creates/reads proceed; the visibility map skips unchanged pages so repeated runs are cheap; SKIP_LOCKED means HA replicas don't pile up (one vacuums, the rest skip). Benchmarked ~7x: a bloated 268MB job_perms heap swept in 35ms vs 5ms vacuumed. Chosen over an autovacuum reloptions migration so the behavior is explicit and lives with the sweep it pairs with. Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Windmill Backend
This folder holds all backend components, the src/ folder only contains files used to build the "root" binary.
Components
| name | description |
|---|---|
| windmill-api | The API server, exposing functionality to other components and the frontend |
| windmill-audit | Contains audit functionality, allowing different components to record important actions |
| windmill-common | Common code shared by all crates |
| windmill-queue | Contains job & flow queuing functionality, commonly written to by the API server and read from by workers |
| windmill-worker | The worker. Used to process and execute flows & jobs. |
| parsers | Contains code to parse signatures in different langauges. |
Compile sqlx for offline ci
cargo sqlx prepare --workspace -- --bin windmill --features enterprise