mirror of
https://github.com/windmill-labs/windmill.git
synced 2026-10-13 00:03:00 +00:00
centdixandClaude Opus 4.8
89c46c803e
Add two agent skills under .agents/skills (symlinked into .claude/skills): - ai-chat: guidance for improving the Windmill AI chat / copilot, especially global mode — benchmark before/after with ai_evals, optimize finalContextTokens over cumulative, keep tool params and tool-result payloads minimal (no echoing content the model already has), treat prompts/tool-descriptions as benchmarkable surface. - ai-evals: author and run black-box benchmark cases for the AI generation modes, migrated from ai_evals/AGENTS.md and extended with run mechanics (workspace reuse, reading the summary). ai_evals/AGENTS.md becomes a pointer stub to the skill. Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
15 lines
639 B
Markdown
15 lines
639 B
Markdown
# AI Evals
|
|
|
|
Black-box benchmark cases for the Windmill AI generation modes (`flow`, `app`,
|
|
`script`, `cli`, `global`).
|
|
|
|
**Authoring and running cases is documented in the `ai-evals` skill** — load it
|
|
before adding/changing a case or running a benchmark. Claude Code reads
|
|
`.claude/skills/ai-evals/SKILL.md`; Codex and Pi read
|
|
`.agents/skills/ai-evals/SKILL.md` (same canonical file). Invoke with `/ai-evals` in
|
|
Claude Code, `$ai-evals` in Codex, or `pi --skill ai-evals`.
|
|
|
|
For AI chat / copilot changes that these evals measure, see the `ai-chat` skill.
|
|
|
|
The full case format, fields, and fixture details remain in `ai_evals/README.md`.
|