mirror of https://github.com/windmill-labs/windmill.git synced 2026-08-24 08:01:38 +00:00

Files

T

hugocasa 39e2f07f3f fix: address round-11 review findings on experiments and scorers

- give scorers the whole case input, not just the message: an answer that came
  from attachments or a replayed conversation could not be judged on it
- accept a judge's boolean and structured {score} answers, including stringified
  ones, and pin every documented scorer shape with a test
- record an experiment for the cases that did launch when a later push fails,
  instead of leaving those jobs running with nothing to attribute them to
- do not capture a preview parent's synthetic runnable_path as a host flow; the
  saved case could not be rerun
- clear the case table before loading a dataset and surface a failed load, so a
  failure cannot leave the previous dataset's cases under the new name
- keep the results table through a refresh of the same experiment
- exclude flow-step jobs from the per-case last-run lookup
- drop case sets from the experiment list, which is only used to pick a run
- report a database failure at the recording lock as itself, not as contention

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

2026-08-11 23:33:16 +02:00