mirror of
https://github.com/windmill-labs/windmill.git
synced 2026-08-19 00:02:03 +00:00
pyranota
fadbd73dc6
Experimental and env-gated (defaults off — stock behavior unchanged). Read path is made fully non-blocking on both sides so neither the worker nor the server waits on the stage behind it: Postgres -> [server-side buffer] -> HTTP -> [agent-side buffer] -> worker - batch pull/complete: N jobs per pull, N completions per flush, so the per-job HTTP round-trips and per-job dequeue queries are amortized - server-side prefetch: a single background puller per job-tag-set runs the SKIP LOCKED dequeue ahead of demand and coalesces concurrent agents, moving the PG query off the request critical path - agent-side prefetch: a background refiller keeps the local job buffer deep (high low-watermark + several refills in flight) so the worker loop pops without a network call - empty-pull backoff fix: don't apply the idle long-sleep to a transient empty pull from a prefetching consumer - AGENT_PROF: per-phase profiler + auto-rendered per-job time-budget SVG Single agent goes from ~2.5k to ~125k+ j/s. Multi-agent collapse is reproduced but its root cause is still open (not established to be PG). See benchmarks/agent_worker_scaling.md. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Windmill Backend
This folder holds all backend components, the src/ folder only contains files used to build the "root" binary.
Components
| name | description |
|---|---|
| windmill-api | The API server, exposing functionality to other components and the frontend |
| windmill-audit | Contains audit functionality, allowing different components to record important actions |
| windmill-common | Common code shared by all crates |
| windmill-queue | Contains job & flow queuing functionality, commonly written to by the API server and read from by workers |
| windmill-worker | The worker. Used to process and execute flows & jobs. |
| parsers | Contains code to parse signatures in different langauges. |
Compile sqlx for offline ci
cargo sqlx prepare --workspace -- --bin windmill --features enterprise