mirror of
https://github.com/windmill-labs/windmill.git
synced 2026-08-18 16:02:10 +00:00
centdix
5c20d6b4f7
* feat: add global ai chat test tools
* fix: avoid session id in flow test preview
* test: cover global flow preview ids
* test: require script and flow test tools
* fix: harden global flow test fallback
* Revert "fix: harden global flow test fallback"
This reverts commit 97254ef33a.
* fix: fallback from inactive flow test hook
* fix: list nested flow steps in errors
65 lines
2.5 KiB
YAML
65 lines
2.5 KiB
YAML
- id: script-test1-greet-user
|
|
prompt: |-
|
|
Update the current Bun script so it takes the existing `name` input and returns a plain greeting string like `Hello, Alice!`.
|
|
Do not wrap the result in an object or array.
|
|
Keep it simple and do not add external dependencies.
|
|
initial: ai_evals/fixtures/frontend/script/initial/test1_empty_bun.json
|
|
expected: ai_evals/fixtures/frontend/script/expected/test1_greet_user.json
|
|
toolExpect:
|
|
requiredToolsUsed:
|
|
- test_run_script
|
|
judgeChecklist:
|
|
- uses the existing `name` input
|
|
- returns a plain greeting string
|
|
- does not wrap the result in an object or array
|
|
|
|
- id: script-test2-create-current-script-schedule
|
|
prompt: |-
|
|
Update the current Bun script so it takes the existing `name` input and returns a plain greeting string like `Hello, Alice!`.
|
|
Also create an enabled daily schedule named `greet_user_daily` for the current script.
|
|
It should run every day at 09:00 UTC and pass `{ "name": "Alice" }` as args.
|
|
Do not ask me for the script path.
|
|
initial: ai_evals/fixtures/frontend/script/initial/test1_empty_bun.json
|
|
expected: ai_evals/fixtures/frontend/script/expected/test1_greet_user.json
|
|
toolExpect:
|
|
requiredToolsUsed:
|
|
- test_run_script
|
|
- create_schedule
|
|
toolCallArgs:
|
|
- tool: create_schedule
|
|
field: path
|
|
stringStartsWithAnyOf:
|
|
- f/
|
|
- u/
|
|
stringMustNotStartWithAnyOf:
|
|
- schedules/
|
|
skipJudge: true
|
|
judgeChecklist:
|
|
- uses the existing `name` input
|
|
- returns a plain greeting string
|
|
|
|
- id: script-test3-create-current-script-http-trigger
|
|
prompt: |-
|
|
Update the current Bun script so it takes the existing `name` input and returns a plain greeting string like `Hello, Alice!`.
|
|
Also create a public POST HTTP endpoint named `greet_user_webhook` for the current script.
|
|
Use route path `ai-evals/greet-user` and no authentication.
|
|
Do not ask me for the script path.
|
|
initial: ai_evals/fixtures/frontend/script/initial/test1_empty_bun.json
|
|
expected: ai_evals/fixtures/frontend/script/expected/test1_greet_user.json
|
|
toolExpect:
|
|
requiredToolsUsed:
|
|
- test_run_script
|
|
- create_trigger
|
|
toolCallArgs:
|
|
- tool: create_trigger
|
|
field: path
|
|
stringStartsWithAnyOf:
|
|
- f/
|
|
- u/
|
|
stringMustNotStartWithAnyOf:
|
|
- schedules/
|
|
skipJudge: true
|
|
judgeChecklist:
|
|
- uses the existing `name` input
|
|
- returns a plain greeting string
|