test: add a cli eval for a service without a hub resource type

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
hugocasa
2026-10-08 11:34:56 +02:00
co-authored by Claude Opus 5.5
parent 5c208e6098
commit 08ea01acdc
+15
View File
@@ -85,6 +85,21 @@
- updates the step code so the returned greeting uses punctuation
- defaults punctuation to an exclamation mark when omitted
# An in-house API has no hub resource type, so its connection needs a type of the workspace's
# own: a token parameter or a variable read inside the code is the fallback the guidance rules out.
- id: cli-resource-type-custom-api
prompt: |-
Create a Windmill Bun script at `f/evals/get_invoice.ts` that fetches an invoice by id from our internal billing API at https://billing.acme.internal/api/v1. It authenticates with a bearer token.
cliExpect:
requiredSkills:
- write-script-bun
forbiddenExecutedCommands:
- ^wmill sync push
judgeChecklist:
- creates the script at f/evals/get_invoice.ts, fetching an invoice by id with a bearer token
- the script takes the billing API's base URL and token together as one resource-typed parameter, not a plain string token or a variable read inside the code
- defines the resource type for the billing API, as a resource-type file or a schema given in the reply
- id: flow-reuse-existing-script
prompt: |-
There is already a reusable greeting script at `f/lib/format_greeting.ts`.