mirror of
https://github.com/windmill-labs/windmill.git
synced 2026-08-20 16:02:19 +00:00
centdix
b883f9a9d2
* feat: add ai chat schedule and trigger tools * refactor: use zod for ai chat workspace tools * refactor: let ai provide runnable target fields * refactor: generate ai chat workspace tool schemas * fix: add object type to composed tool schemas * fix: avoid top-level trigger schema unions Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * fix: block undeployed workspace ai tools Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * fix: inject ai workspace tool target Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * test: add ai evals for workspace tools Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * test: make workspace tool eval prompts realistic Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * fix: surface workspace tool errors * fix: show workspace tool success details * fix: describe workspace tool path format * fix: clarify workspace path examples * fix: tighten workspace tool validation * fix: align workspace tool prompts * chore: mark generated chat schemas * chore: mark generated cli skills --------- Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>
60 lines
2.3 KiB
YAML
60 lines
2.3 KiB
YAML
- id: script-test1-greet-user
|
|
prompt: |-
|
|
Update the current Bun script so it takes the existing `name` input and returns a plain greeting string like `Hello, Alice!`.
|
|
Do not wrap the result in an object or array.
|
|
Keep it simple and do not add external dependencies.
|
|
initial: ai_evals/fixtures/frontend/script/initial/test1_empty_bun.json
|
|
expected: ai_evals/fixtures/frontend/script/expected/test1_greet_user.json
|
|
judgeChecklist:
|
|
- uses the existing `name` input
|
|
- returns a plain greeting string
|
|
- does not wrap the result in an object or array
|
|
|
|
- id: script-test2-create-current-script-schedule
|
|
prompt: |-
|
|
Update the current Bun script so it takes the existing `name` input and returns a plain greeting string like `Hello, Alice!`.
|
|
Also create an enabled daily schedule named `greet_user_daily` for the current script.
|
|
It should run every day at 09:00 UTC and pass `{ "name": "Alice" }` as args.
|
|
Do not ask me for the script path.
|
|
initial: ai_evals/fixtures/frontend/script/initial/test1_empty_bun.json
|
|
expected: ai_evals/fixtures/frontend/script/expected/test1_greet_user.json
|
|
toolExpect:
|
|
requiredToolsUsed:
|
|
- create_schedule
|
|
toolCallArgs:
|
|
- tool: create_schedule
|
|
field: path
|
|
stringStartsWithAnyOf:
|
|
- f/
|
|
- u/
|
|
stringMustNotStartWithAnyOf:
|
|
- schedules/
|
|
skipJudge: true
|
|
judgeChecklist:
|
|
- uses the existing `name` input
|
|
- returns a plain greeting string
|
|
|
|
- id: script-test3-create-current-script-http-trigger
|
|
prompt: |-
|
|
Update the current Bun script so it takes the existing `name` input and returns a plain greeting string like `Hello, Alice!`.
|
|
Also create a public POST HTTP endpoint named `greet_user_webhook` for the current script.
|
|
Use route path `ai-evals/greet-user` and no authentication.
|
|
Do not ask me for the script path.
|
|
initial: ai_evals/fixtures/frontend/script/initial/test1_empty_bun.json
|
|
expected: ai_evals/fixtures/frontend/script/expected/test1_greet_user.json
|
|
toolExpect:
|
|
requiredToolsUsed:
|
|
- create_trigger
|
|
toolCallArgs:
|
|
- tool: create_trigger
|
|
field: path
|
|
stringStartsWithAnyOf:
|
|
- f/
|
|
- u/
|
|
stringMustNotStartWithAnyOf:
|
|
- schedules/
|
|
skipJudge: true
|
|
judgeChecklist:
|
|
- uses the existing `name` input
|
|
- returns a plain greeting string
|