Files
Ruben Fiszel e9f016cf66 fix: serialize concurrent python wheel-cache installs to prevent flaky ModuleNotFoundError
The wheel cache (ROOT_CACHE_DIR/python_<v>/<pkg>==<ver>) is shared by every
job on a worker, and uv installs into it with `--reinstall --target <dir>`,
which transiently empties the directory mid-run. Two jobs installing the same
package concurrently (e.g. several WAC tests all importing wmill) clobber each
other's files, and a job that imports from the dir during a concurrent
reinstall sees a partially populated or wiped package -> flaky
ModuleNotFoundError (httpx/wmill).

Serialize installs per target dir with a process-wide keyed lock and re-check
the `.valid.windmill` marker once the lock is held, so a waiter reuses the
freshly populated cache instead of reinstalling over a dir other jobs may be
importing from. Reinstall now only runs when no valid install exists, so no
concurrent importer can observe a half-populated dir.

Reproduced with a cold-wheel-cache loop over the wmill-importing python tests:
8/10 iterations failed before, 0/12 after. Adds a unit test guarding the
per-target lock keying.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-25 14:03:11 +00:00
..

Windmill Worker

The worker. Used to process and execute flows & jobs.

This crate exposes both a library as well as a binary target.