mirror of
https://github.com/windmill-labs/windmill.git
synced 2026-08-25 16:02:11 +00:00
hugocasa
8a667cf9e2
An experiment runs every case of a dataset against one subject and records the
exact case set it executed, so a result set stays reproducible while the dataset
keeps changing.
Each case runs as its own small flow — the agent, then a step per scorer — so a
case keeps the run stamp, history query and trajectory view a single run already
has, and scorers need no orchestration of their own. Results are read back per
step by node id rather than by walking a nested loop's status.
A scorer is any runnable taking (input, output, expected): a script, a flow, or a
reusable agent used as a judge. A judge is prompted with the case and the answer
as one JSON message; a script or flow receives them as named arguments. Scores
accept a bare number, a boolean or {score}.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Windmill API
The API server, exposing functionality to other components and the frontend
This crate exposes both a library as well as a binary target.