docs: teach agents to pass a resource as $res:<path> in run arguments (#10927)

* docs: teach agents to pass a resource as $res:<path> in run arguments

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01XjRARL7JA7xm772iJP4mJk

* docs: extend run-argument rule to in-editor chats, fix run-as wording

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01XjRARL7JA7xm772iJP4mJk

* docs: tighten resource run-argument rule after review

- Drop the false rationale that "$var:" only works inside a resource value
  from the write_variable description and its runtime rejection message; keep
  the rule (a variable cannot reference itself).
- MCP resource-argument description: the title fallback renders "No title",
  so say the title is only a label rather than that it can be empty. Guard the
  real-newline fix with asserts in the existing enrichment test.
- Eval: assert the full "$res:f/evals/global/github_main" value as one prefix
  so a wrong path with a right prefix fails.
- resources.md: narrow "a trigger's payload" to its configured static args.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01XjRARL7JA7xm772iJP4mJk

* docs: scope the run-argument rule to global chat, add an exact eval matcher

The ai_evals A/B on the two in-editor modes showed no effect: script mode
sonnet 5/5 both with and without the description, flow mode sonnet 5/5 and
haiku 5/5 on the baseline alone. A flow's input schema already carries
`format: resource-<type>`, so those modes have a signal global mode does not
give. Revert both files to keep the tool schemas free of a description that
buys nothing per iteration; global mode keeps it, where haiku goes 0/5 -> 5/5.

Add `stringEqualsAnyOf` to toolCallArgs and use it for the resource reference:
nothing in the eval resolves the value, so a prefix match accepted a near-miss
path like `$res:f/evals/global/github_main_backup`.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01XjRARL7JA7xm772iJP4mJk

* docs: address cubic review — CLI wording, mock resource getter

- `-d --data` help on all four run/preview commands: give $res: and $var:
  their own clauses instead of a parenthetical that read as if a resource
  were a kind of variable.
- Mock backend: `getBenchmarkResource` now resolves AI-provider seeds as well
  as plain ones, so it agrees with `existsResource` and `listResource` — both
  report either kind, and a case that listed a resource and then read it by
  path got a row it could not fetch.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01XjRARL7JA7xm772iJP4mJk

---------

Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
This commit is contained in:
Guilhem
2026-09-03 11:29:54 +02:00
committed by GitHub
co-authored by Claude Opus 5
parent 582761e37c
commit e39dd7eb12
23 changed files with 399 additions and 41 deletions
+72 -5
View File
@@ -7,6 +7,7 @@ import type {
ListableApp,
ListableResource,
ListableVariable,
Resource,
Script
} from '../../../frontend/src/lib/gen'
import type {
@@ -81,6 +82,15 @@ export interface BenchmarkWorkspaceAiProvider {
isDefault?: boolean
}
/** A plain (non-AI) resource of the benchmark workspace, for cases about referencing a
* credential — passing one as a run argument, say. `value` is what `get_resource` returns. */
export interface BenchmarkWorkspaceResource {
path: string
resource_type: string
value?: Record<string, unknown>
description?: string
}
export interface BenchmarkWorkspaceJob {
/** Stable id so a case prompt can reference a specific run (e.g. for get_job_logs). */
id?: string
@@ -98,6 +108,7 @@ export interface BenchmarkWorkspaceRunnables {
apps?: BenchmarkWorkspaceApp[]
variables?: BenchmarkWorkspaceVariable[]
aiProviders?: BenchmarkWorkspaceAiProvider[]
resources?: BenchmarkWorkspaceResource[]
datatables?: BenchmarkDatatableSeed[]
jobs?: BenchmarkWorkspaceJob[]
}
@@ -284,15 +295,71 @@ export function listBenchmarkAiProviderResources(workspace: string): ListableRes
}))
}
/** The value of a seeded AI provider resource. Only the endpoint fields are modelled — a key is
* never needed, because no eval run calls the provider through this resource. */
/** Plain seeded resources of a benchmark workspace, shaped like `ResourceService.listResource`
* rows. Null when the workspace is not a benchmark one. */
export function listBenchmarkPlainResources(workspace: string): ListableResource[] | null {
const runnables = benchmarkWorkspaceRunnables.get(workspace)
if (!runnables) {
return null
}
return (runnables.resources ?? []).map((seed) => ({
workspace_id: workspace,
path: seed.path,
resource_type: seed.resource_type,
description: seed.description,
value: null,
is_oauth: false,
is_linked: false,
is_refreshed: false,
extra_perms: {},
edited_at: BENCHMARK_TIMESTAMP
}))
}
/** A seeded resource with its value, as `ResourceService.getResource` returns it. Covers both
* seed kinds, so it agrees with `existsResource` and `listResource` — both of those report AI
* providers too, and a case that lists resources and then reads one by path would otherwise get
* a row it cannot fetch. */
export function getBenchmarkResource(workspace: string, path: string): Resource | null {
const runnables = benchmarkWorkspaceRunnables.get(workspace)
const seed = runnables?.resources?.find((entry) => entry.path === path)
if (seed) {
return {
workspace_id: workspace,
path: seed.path,
resource_type: seed.resource_type,
description: seed.description,
value: seed.value ?? {},
is_oauth: false,
extra_perms: {}
} as Resource
}
const provider = runnables?.aiProviders?.find((entry) => entry.path === path)
if (!provider) {
return null
}
return {
workspace_id: workspace,
path: provider.path,
resource_type: provider.kind,
value: getBenchmarkResourceValue(workspace, path) ?? {},
is_oauth: false,
extra_perms: {}
} as Resource
}
/** The value of a seeded resource. For an AI provider only the endpoint fields are modelled — a
* key is never needed, because no eval run calls the provider through this resource. */
export function getBenchmarkResourceValue(
workspace: string,
path: string
): Record<string, unknown> | null {
const seed = benchmarkWorkspaceRunnables
.get(workspace)
?.aiProviders?.find((entry) => entry.path === path)
const runnables = benchmarkWorkspaceRunnables.get(workspace)
const plain = runnables?.resources?.find((entry) => entry.path === path)
if (plain) {
return plain.value ?? {}
}
const seed = runnables?.aiProviders?.find((entry) => entry.path === path)
if (!seed) {
return null
}
@@ -70,7 +70,9 @@ vi.mock('$lib/gen', async () => {
getBenchmarkResourceValue,
getBenchmarkVariableByPath,
hasBenchmarkWorkspace,
getBenchmarkResource,
listBenchmarkAiProviderResources,
listBenchmarkPlainResources,
listBenchmarkApps,
listBenchmarkDatatables,
listBenchmarkDrafts,
@@ -359,18 +361,24 @@ vi.mock('$lib/gen', async () => {
hasBenchmarkWorkspace(data.workspace)
? Boolean(getBenchmarkResourceValue(data.workspace, data.path))
: actual.ResourceService.existsResource(data),
// Only AI provider resources are modelled: they are what an AI agent step references.
listResource: async (data: { workspace: string; resourceType?: string }) => {
if (!hasBenchmarkWorkspace(data.workspace)) {
return actual.ResourceService.listResource(data)
}
const seeded = listBenchmarkAiProviderResources(data.workspace) ?? []
const seeded = [
...(listBenchmarkAiProviderResources(data.workspace) ?? []),
...(listBenchmarkPlainResources(data.workspace) ?? [])
]
const wanted = data.resourceType?.split(',')
return wanted ? seeded.filter((r) => wanted.includes(r.resource_type)) : seeded
},
getResource: async (data: { workspace: string; path: string }) => {
if (hasBenchmarkWorkspace(data.workspace)) {
throw new Error(`Resource "${data.path}" not found in benchmark workspace`)
const resource = getBenchmarkResource(data.workspace, data.path)
if (!resource) {
throw new Error(`Resource "${data.path}" not found in benchmark workspace`)
}
return resource
}
return actual.ResourceService.getResource(data)
},
+34
View File
@@ -2365,3 +2365,37 @@
- the step uses the workspace's anthropic resource f/evals/global/anthropic_main
- the model is the Opus one the user asked for, taken from the models that resource serves
- the diff flow input reaches the agent
# The failure this pins: passing a resource as `{"$res": "<path>"}` (or as a bare path), which
# reaches the script unresolved because the backend only substitutes a string value that itself
# starts with `$res:`. The mock preview echoes args back and reports success, so nothing in the
# loop corrects a wrong shape — the arg form is the whole test.
- id: global-run-arg-resource-reference
prompt: |-
Run `f/evals/global/github_repo_stats` against the `windmill-labs/windmill` repo, passing our
GitHub credentials at `f/evals/global/github_main` as its `gh_auth` input, and tell me whether
it went through.
initial: ai_evals/fixtures/frontend/global/initial/github_repo_stats_script.json
runtime:
maxTurns: 10
validate:
draftCountExactly: 0
toolExpect:
requiredToolsUsed:
- test_run_script
forbiddenToolsUsed:
- write_script
- deploy_workspace_item
toolCallArgs:
# Exact: the mock never resolves the reference, so a near-miss path like
# `$res:f/evals/global/github_main_backup` would otherwise pass.
- tool: test_run_script
field: args.gh_auth
stringEqualsAnyOf:
- "$res:f/evals/global/github_main"
# The judge only sees drafts, and this case makes none — the deliverable is the shape of the
# run argument, checked deterministically above.
skipJudge: true
judgeChecklist:
- runs the existing script rather than rewriting it
- passes the GitHub resource as the bare string $res:f/evals/global/github_main
+7
View File
@@ -160,6 +160,13 @@ export interface ToolCallArgumentRule {
field: string;
stringStartsWithAnyOf?: string[];
stringMustNotStartWithAnyOf?: string[];
/**
* Universal over calls: every recorded call to `tool` must carry `field` as
* exactly one of these strings. Use when a near-miss would still satisfy a
* prefix — a resource reference like `$res:f/a/b` shares its prefix with the
* wrong `$res:f/a/b_backup`, and the mock never resolves it to catch that.
*/
stringEqualsAnyOf?: string[];
/**
* Case-insensitive "contains", existential over calls: at least one recorded
* call to `tool` must have `field` containing one of these substrings. Other
+37
View File
@@ -228,6 +228,43 @@ describe("validateToolExpectations", () => {
});
});
// A resource reference shares its prefix with a wrong sibling path, and the mock
// never resolves it, so only exact matching separates the two.
it("rejects a resource reference whose path merely shares the prefix", () => {
const checks = validateToolExpectations({
run: {
success: true,
actual: {},
assistantMessageCount: 1,
toolCallCount: 1,
toolsUsed: ["test_run_script"],
toolCallDetails: [
{
name: "test_run_script",
arguments: { args: { gh_auth: "$res:f/evals/global/github_main_backup" } },
},
],
skillsInvoked: [],
},
toolExpect: {
toolCallArgs: [
{
tool: "test_run_script",
field: "args.gh_auth",
stringEqualsAnyOf: ["$res:f/evals/global/github_main"],
},
],
},
});
expect(checks).toContainEqual({
name: "test_run_script.args.gh_auth matches an accepted value",
passed: false,
details:
'accepted values: $res:f/evals/global/github_main; values: "$res:f/evals/global/github_main_backup"',
});
});
// The whole point of the same-call rule: the per-field rules are existential over
// calls, so two single-filter pages would satisfy them while never opening the
// combined view the case asks for.
+14
View File
@@ -278,6 +278,20 @@ export function validateToolExpectations(input: {
);
}
if (rule.stringEqualsAnyOf && rule.stringEqualsAnyOf.length > 0) {
const invalidValues = values.filter(
(value) =>
typeof value !== "string" || !rule.stringEqualsAnyOf!.includes(value)
);
checks.push(
check(
`${rule.tool}.${rule.field} matches an accepted value`,
invalidValues.length === 0,
`accepted values: ${rule.stringEqualsAnyOf.join(", ")}; values: ${summarizeToolValues(values)}`
)
);
}
if (rule.stringMustNotStartWithAnyOf && rule.stringMustNotStartWithAnyOf.length > 0) {
const invalidValues = values.filter(
(value) =>
@@ -0,0 +1,37 @@
{
"workspace": {
"resources": [
{
"path": "f/evals/global/github_main",
"resource_type": "github",
"description": "GitHub credentials",
"value": { "token": "$var:f/evals/global/github_token" }
}
],
"scripts": [
{
"path": "f/evals/global/github_repo_stats",
"summary": "Count open issues on a GitHub repository",
"description": "Reads the open issue count for a repository using GitHub credentials.",
"language": "bun",
"schema": {
"$schema": "https://json-schema.org/draft/2020-12/schema",
"type": "object",
"properties": {
"gh_auth": {
"type": "object",
"format": "resource-github",
"description": "GitHub credentials"
},
"repo": {
"type": "string",
"description": "Repository in owner/name form"
}
},
"required": ["gh_auth", "repo"]
},
"content": "type Github = { token: string }\n\nexport async function main(gh_auth: Github, repo: string) {\n const res = await fetch(`https://api.github.com/repos/${repo}/issues?state=open`, {\n headers: { Authorization: `Bearer ${gh_auth.token}` }\n })\n const issues = await res.json()\n return { repo, open_issues: issues.length }\n}\n"
}
]
}
}
+2 -2
View File
@@ -10456,7 +10456,7 @@ paths:
summary: run script by path
operationId: runScriptByPath
x-mcp-tool: true
x-mcp-instructions: "You should first use getScriptByPath to retrieve the script's schema and understand what arguments are expected."
x-mcp-instructions: "You should first use getScriptByPath to retrieve the script's schema and understand what arguments are expected. A resource argument (schema format `resource-<type>`) is the bare string `$res:<path>` as that whole argument value -- not a wrapper object, not a plain path. A variable is `$var:<path>`."
x-mcp-tool-include-query-params: []
tags:
- job
@@ -13814,7 +13814,7 @@ paths:
summary: run flow by path
operationId: runFlowByPath
x-mcp-tool: true
x-mcp-instructions: "You should first use getFlowByPath to retrieve the flow's schema and understand what arguments are expected."
x-mcp-instructions: "You should first use getFlowByPath to retrieve the flow's schema and understand what arguments are expected. A resource argument (schema format `resource-<type>`) is the bare string `$res:<path>` as that whole argument value -- not a wrapper object, not a plain path. A variable is `$var:<path>`."
x-mcp-tool-include-query-params: []
tags:
- job
@@ -878,7 +878,7 @@ is, a different one moves it there and archives the old path"),
EndpointTool {
name: Cow::Borrowed("runScriptByPath"),
description: Cow::Borrowed("run script by path"),
instructions: Cow::Borrowed("You should first use getScriptByPath to retrieve the script's schema and understand what arguments are expected."),
instructions: Cow::Borrowed("You should first use getScriptByPath to retrieve the script's schema and understand what arguments are expected. A resource argument (schema format `resource-<type>`) is the bare string `$res:<path>` as that whole argument value -- not a wrapper object, not a plain path. A variable is `$var:<path>`."),
path: Cow::Borrowed("/w/{workspace}/jobs/run/p/{path}"),
method: Cow::Borrowed("POST"),
path_params_schema: Some(serde_json::json!({
@@ -1419,7 +1419,7 @@ is, a different one moves it there and archives the old path"),
EndpointTool {
name: Cow::Borrowed("runFlowByPath"),
description: Cow::Borrowed("run flow by path"),
instructions: Cow::Borrowed("You should first use getFlowByPath to retrieve the flow's schema and understand what arguments are expected."),
instructions: Cow::Borrowed("You should first use getFlowByPath to retrieve the flow's schema and understand what arguments are expected. A resource argument (schema format `resource-<type>`) is the bare string `$res:<path>` as that whole argument value -- not a wrapper object, not a plain path. A variable is `$var:<path>`."),
path: Cow::Borrowed("/w/{workspace}/jobs/run/f/{path}"),
method: Cow::Borrowed("POST"),
path_params_schema: Some(serde_json::json!({
+7 -3
View File
@@ -101,7 +101,7 @@ fn apply_resource_enrichment(
let resources_count = resource_cache.len();
let description = match resource_type {
Some(rt) => format!(
"This is a resource named `{}` with the following description: `{}`.\\nThe path of the resource should be used to specify the resource.\\n{}",
"This is a resource named `{}` with the following description: `{}`.\nPass it as the bare string `$res:<path>` — the whole value of this argument, never an object wrapper like {{\"$res\": \"<path>\"}} and never a plain path.\n{}",
rt.name,
rt.description.as_deref().unwrap_or("No description"),
if resources_count == 0 {
@@ -138,7 +138,7 @@ fn apply_resource_enrichment(
)
})
.collect::<Vec<String>>()
.join("\\n");
.join("\n");
let prior_description = prop_map
.get("description")
.and_then(Value::as_str)
@@ -147,7 +147,7 @@ fn apply_resource_enrichment(
prop_map.insert(
"description".to_string(),
Value::String(format!(
"{}\\nHere are the available resources, in the format title:path. Title can be empty. Path should be used to specify the resource:\\n{}",
"{}\nHere are the available resources, one per line as `title: $res:path`. The title is only a label; pass the `$res:path` part verbatim as this argument's value:\n{}",
prior_description, resources_description
)),
);
@@ -804,6 +804,10 @@ mod tests {
let desc = node["description"].as_str().unwrap();
assert!(desc.contains("c_aws_account"));
assert!(desc.contains("$res:f/platform/aws_dev"));
// MCP clients render this description verbatim, so the separators must be
// real newlines rather than the two-character escape.
assert!(desc.contains('\n'));
assert!(!desc.contains("\\n"));
}
#[test]
+2 -2
View File
@@ -1144,7 +1144,7 @@ const command = new Command()
.arguments("<path:string>")
.option(
"-d --data <data:string>",
"Inputs specified as a JSON string or a file using @<filename> or stdin using @-."
"Inputs specified as a JSON string or a file using @<filename> or stdin using @-. A resource argument is the bare string $res:<path> as its whole value, and a variable argument is the bare string $var:<path> — not an object wrapper keyed on $res/$var, and not a plain path."
)
.option(
"-s --silent",
@@ -1162,7 +1162,7 @@ const command = new Command()
.arguments("<flow_path:string>")
.option(
"-d --data <data:string>",
"Inputs specified as a JSON string or a file using @<filename> or stdin using @-."
"Inputs specified as a JSON string or a file using @<filename> or stdin using @-. A resource argument is the bare string $res:<path> as its whole value, and a variable argument is the bare string $var:<path> — not an object wrapper keyed on $res/$var, and not a plain path."
)
.option(
"-s --silent",
+2 -2
View File
@@ -2213,7 +2213,7 @@ const command = new Command()
.arguments("<path:file>")
.option(
"-d --data <data:file>",
"Inputs specified as a JSON string or a file using @<filename> or stdin using @-."
"Inputs specified as a JSON string or a file using @<filename> or stdin using @-. A resource argument is the bare string $res:<path> as its whole value, and a variable argument is the bare string $var:<path> — not an object wrapper keyed on $res/$var, and not a plain path."
)
.option(
"-s --silent",
@@ -2231,7 +2231,7 @@ const command = new Command()
.arguments("<path:file>")
.option(
"-d --data <data:file>",
"Inputs specified as a JSON string or a file using @<filename> or stdin using @-."
"Inputs specified as a JSON string or a file using @<filename> or stdin using @-. A resource argument is the bare string $res:<path> as its whole value, and a variable argument is the bare string $var:<path> — not an object wrapper keyed on $res/$var, and not a plain path."
)
.option(
"-s --silent",
+2
View File
@@ -115,6 +115,8 @@ Local previews exist for every entity type and don't deploy:
- \`wmill flow preview <flow_path> -d '<args>'\` — run a local flow.yaml.
- \`wmill app dev\` — live-reload dev server for raw apps.
An argument typed as a resource takes the bare string \`"$res:<path>"\` as its whole value (a variable takes \`"$var:<path>"\`) — never an object wrapper like \`{"$res": "<path>"}\`, and never a plain path. See the \`resources\` skill.
Argument shapes and per-language details live in the \`write-script-<lang>\`, \`write-flow\`, and \`raw-app\` skills.
## Keeping metadata in sync
+41 -4
View File
@@ -5116,6 +5116,8 @@ If the user hasn't already told you to run/test the flow, offer it as a one-sent
If the user already asked to test/run/try the flow in their original request, skip the offer and just execute \`wmill flow preview <path> -d '<args>'\` directly — pick plausible args from the flow's input schema.
An input typed as a resource (\`format: resource-<type>\` in the schema) takes the bare string \`"$res:<path>"\` as its whole value — \`-d '{"db": "$res:f/databases/postgres_prod"}'\`, not \`{"db": {"$res": "..."}}\` and not a plain path. Same for a variable, with \`"$var:<path>"\`. See the \`resources\` skill.
\`wmill flow preview\` is safe to run yourself (it does not deploy). \`wmill generate-metadata\` does not deploy either (it only writes local lock/hash files) but re-resolves deps — offer it and run on agreement, unless the project's \`AGENTS.md\` opts into automatic metadata. After running it, check the regenerated \`.lock\` diff and tell the user which inline-script dependency versions changed, so they can catch an unwanted bump before deploying. Only \`wmill sync push\` deploys; run it only when the user explicitly asks.
### Visual preview
@@ -6188,6 +6190,41 @@ Reference other resources:
}
\`\`\`
## Passing a Resource or Variable as a Run Argument
A script or flow argument typed as a resource (schema \`format: resource-<type>\`) is passed as
the **bare string** \`$res:<path>\` — the whole argument value. Same for a variable, with
\`$var:<path>\`. This applies everywhere job arguments are supplied: \`wmill script run/preview\`,
\`wmill flow run/preview\`, the \`runScriptByPath\` / \`runFlowByPath\` API, a schedule's \`args\`, a
trigger's configured static args.
\`\`\`json
{
"db": "$res:f/databases/postgres_prod",
"api_token": "$var:g/all/api_token"
}
\`\`\`
The reference is resolved when the job runs, under the job's run-as identity the caller for an
ordinary run, but the configured principal for a schedule, a trigger, or a runnable set to run on
behalf of someone else. The run fails if that identity cannot read the referenced resource or
variable.
**Never wrap it in an object.** The resolver only rewrites a JSON value that *is* a string
starting with \`$res:\` / \`$var:\`; keys are never inspected. These are all wrong and are passed
through to the script unchanged:
\`\`\`json
{ "db": { "$res": "f/databases/postgres_prod" } }
{ "db": { "resource": "f/databases/postgres_prod" } }
{ "db": "f/databases/postgres_prod" }
\`\`\`
The string may sit anywhere a string can a top-level argument, a nested object field
(\`{ "gh_auth": { "token": "$var:g/all/gh_token" } }\`), or an array element (array elements are
walked only while nested at most two levels deep, and only for arrays of at most 1000 items).
The prefix must be on the string itself.
## Common Resource Types
### PostgreSQL
@@ -7074,11 +7111,11 @@ flow related commands
- \`flow push <file_path:string> <remote_path:string>\` - push a local flow spec. This overrides any remote versions.
- \`--message <message:string>\` - Deployment message
- \`flow run <path:string>\` - run a flow by path.
- \`-d --data <data:string>\` - Inputs specified as a JSON string or a file using @<filename> or stdin using @-.
- \`-d --data <data:string>\` - Inputs specified as a JSON string or a file using @<filename> or stdin using @-. A resource argument is the bare string $res:<path> as its whole value, and a variable argument is the bare string $var:<path> — not an object wrapper keyed on $res/$var, and not a plain path.
- \`-s --silent\` - Do not ouput anything other then the final output. Useful for scripting.
- \`--tag <tag:string>\` - Override the worker tag the run is dispatched to (e.g. to route it to dev workers instead of the flow's default tag).
- \`flow preview <flow_path:string>\` - preview a local flow without deploying it. Runs the flow definition from local files and uses local PathScripts by default. Pass --step <id> to run only one module in isolation (resolves nested steps inside branchone/branchall/forloopflow/whileloopflow plus the special preprocessor/failure modules; supported step types: rawscript, script, flow).
- \`-d --data <data:string>\` - Inputs specified as a JSON string or a file using @<filename> or stdin using @-.
- \`-d --data <data:string>\` - Inputs specified as a JSON string or a file using @<filename> or stdin using @-. A resource argument is the bare string $res:<path> as its whole value, and a variable argument is the bare string $var:<path> — not an object wrapper keyed on $res/$var, and not a plain path.
- \`-s --silent\` - Do not output anything other then the final output. Useful for scripting.
- \`--remote\` - Use deployed workspace scripts for PathScript steps instead of local files.
- \`--step <step_id:string>\` - Run only the named step instead of the whole flow. Honors --data as the step's args and --remote / local-PathScript resolution the same way the full-flow preview does.
@@ -7474,11 +7511,11 @@ script related commands
- \`--json\` - Output as JSON (for piping to jq)
- \`script show <path:file>\` - show a script's content (alias for get)
- \`script run <path:file>\` - run a script by path
- \`-d --data <data:file>\` - Inputs specified as a JSON string or a file using @<filename> or stdin using @-.
- \`-d --data <data:file>\` - Inputs specified as a JSON string or a file using @<filename> or stdin using @-. A resource argument is the bare string $res:<path> as its whole value, and a variable argument is the bare string $var:<path> — not an object wrapper keyed on $res/$var, and not a plain path.
- \`-s --silent\` - Do not output anything other then the final output. Useful for scripting.
- \`--tag <tag:string>\` - Override the worker tag the run is dispatched to (e.g. to route it to dev workers instead of the script's default tag).
- \`script preview <path:file>\` - preview a local script without deploying it. Supports both regular and codebase scripts.
- \`-d --data <data:file>\` - Inputs specified as a JSON string or a file using @<filename> or stdin using @-.
- \`-d --data <data:file>\` - Inputs specified as a JSON string or a file using @<filename> or stdin using @-. A resource argument is the bare string $res:<path> as its whole value, and a variable argument is the bare string $var:<path> — not an object wrapper keyed on $res/$var, and not a plain path.
- \`-s --silent\` - Do not output anything other than the final output. Useful for scripting.
- \`--tag <tag:string>\` - Override the worker tag the preview is dispatched to (e.g. to route it to dev workers instead of the script's default tag).
- \`script new <path:file> <language:string>\` - create a new script
@@ -645,7 +645,7 @@ const writeVariableSchema = variableRequestSchema.extend({
.string()
.optional()
.describe(
'The value of the variable. Omit it to leave the value alone — required only when creating a new variable, or when changing a secret variable into a non-secret one. Never invent or guess the value of an existing variable: you cannot read it, and a "$var:..." reference is NOT a valid value (that syntax only references a variable from inside a resource). Omitting it keeps whatever the draft already holds, so a value you set earlier in this conversation stays set; discard_local_draft abandons it.'
'The value of the variable. Omit it to leave the value alone — required only when creating a new variable, or when changing a secret variable into a non-secret one. Never invent or guess the value of an existing variable: you cannot read it, and a "$var:..." reference is NOT a valid value (a variable cannot reference itself). Omitting it keeps whatever the draft already holds, so a value you set earlier in this conversation stays set; discard_local_draft abandons it.'
),
is_secret: z
.boolean()
@@ -854,7 +854,9 @@ const testRunArgsSchema = z
.record(z.string(), z.any())
.nullable()
.optional()
.describe('Arguments to pass to the runnable. Omit or pass null when no arguments are needed.')
.describe(
'Arguments to pass to the runnable. Omit or pass null when no arguments are needed. An argument typed as a resource (format "resource-<type>" in the input schema) takes the bare string "$res:<path>" as its whole value — never an object wrapper like {"$res": "<path>"}, and never a plain path, both of which reach the runnable unresolved. Same for a variable, with "$var:<path>". The prefixed string can also sit in a nested field, e.g. {"gh_auth": {"token": "$var:g/all/gh_token"}}.'
)
const backgroundArgSchema = z
.boolean()
@@ -2214,7 +2216,7 @@ function getResourceInstructions(): string {
- Reading a variable returns \`{ type: 'variable', path, summary?, isSecret, isDraft }\` — never its value, secret or not. \`isSecret\` tells you whether the value is encrypted.
- \`write_variable\` takes \`{ path, value?, is_secret?, description?, account?, is_oauth?, expires_at?, labels? }\`. Creating a variable needs \`value\` and \`is_secret\`; editing one needs only the fields you are changing. Omitting \`value\` keeps the stored value, which is the only way to edit a secret variable — you cannot read its value, so passing any \`value\` you did not get from the user destroys it.
- For secret fields in a resource value, do NOT inline the raw secret. Create a Variable first with \`is_secret: true\`, then in the resource value reference it as \`"$var:path/to/variable"\`.
- Reference formats inside resource values: \`$var:g/all/name\` (global), \`$var:u/user/name\` (user), \`$var:f/folder/name\` (folder). Reference another resource with \`$res:path/to/resource\`. These are references FROM a resource value; never store a \`$var:\` string as a variable's own value.
- Reference formats inside resource values: \`$var:g/all/name\` (global), \`$var:u/user/name\` (user), \`$var:f/folder/name\` (folder). Reference another resource with \`$res:path/to/resource\`. The same strings are also how a resource or variable is passed as a run argument (see the run-argument rule in the resource reference below); what they are never valid as is a variable's own value.
- When deploying drafts that depend on each other (e.g., a resource and the variables it references), deploy the variables first.
- Use \`search_resource_types\` to discover valid \`resource_type\` names and their JSON Schemas. Match the resource value to that schema.
- For OAuth resources, the \`is_oauth: true\` flag is managed by Windmill's OAuth flow; global mode generally creates manual resources, not OAuth ones.
@@ -5105,7 +5107,7 @@ function writeVariableDraft(args: WriteVariableArgs, ctx: WriteDraftCtx): Promis
// is always the model echoing the reference syntax back instead of a real value.
if (args.value === `$var:${args.path}`) {
throw new Error(
`"${args.value}" is not a valid value for variable "${args.path}" — it is a self-reference. The "$var:" syntax only references a variable from inside a resource value. Omit value to keep the current one.`
`"${args.value}" is not a valid value for variable "${args.path}" — it is a self-reference. Omit value to keep the current one.`
)
}
return writeDraft(VARIABLE_SPEC, 'variable', args.path, args, ctx, { override: args.override })
+2 -2
View File
@@ -885,7 +885,7 @@ export const mcpEndpointTools: EndpointTool[] = [
{
name: "runScriptByPath",
description: "run script by path",
instructions: "You should first use getScriptByPath to retrieve the script's schema and understand what arguments are expected.",
instructions: "You should first use getScriptByPath to retrieve the script's schema and understand what arguments are expected. A resource argument (schema format `resource-<type>`) is the bare string `$res:<path>` as that whole argument value -- not a wrapper object, not a plain path. A variable is `$var:<path>`.",
path: "/w/{workspace}/jobs/run/p/{path}",
method: "POST",
pathParamsSchema: {
@@ -1426,7 +1426,7 @@ export const mcpEndpointTools: EndpointTool[] = [
{
name: "runFlowByPath",
description: "run flow by path",
instructions: "You should first use getFlowByPath to retrieve the flow's schema and understand what arguments are expected.",
instructions: "You should first use getFlowByPath to retrieve the flow's schema and understand what arguments are expected. A resource argument (schema format `resource-<type>`) is the bare string `$res:<path>` as that whole argument value -- not a wrapper object, not a plain path. A variable is `$var:<path>`.",
path: "/w/{workspace}/jobs/run/f/{path}",
method: "POST",
pathParamsSchema: {
@@ -160,11 +160,11 @@ flow related commands
- `flow push <file_path:string> <remote_path:string>` - push a local flow spec. This overrides any remote versions.
- `--message <message:string>` - Deployment message
- `flow run <path:string>` - run a flow by path.
- `-d --data <data:string>` - Inputs specified as a JSON string or a file using @<filename> or stdin using @-.
- `-d --data <data:string>` - Inputs specified as a JSON string or a file using @<filename> or stdin using @-. A resource argument is the bare string $res:<path> as its whole value, and a variable argument is the bare string $var:<path> — not an object wrapper keyed on $res/$var, and not a plain path.
- `-s --silent` - Do not ouput anything other then the final output. Useful for scripting.
- `--tag <tag:string>` - Override the worker tag the run is dispatched to (e.g. to route it to dev workers instead of the flow's default tag).
- `flow preview <flow_path:string>` - preview a local flow without deploying it. Runs the flow definition from local files and uses local PathScripts by default. Pass --step <id> to run only one module in isolation (resolves nested steps inside branchone/branchall/forloopflow/whileloopflow plus the special preprocessor/failure modules; supported step types: rawscript, script, flow).
- `-d --data <data:string>` - Inputs specified as a JSON string or a file using @<filename> or stdin using @-.
- `-d --data <data:string>` - Inputs specified as a JSON string or a file using @<filename> or stdin using @-. A resource argument is the bare string $res:<path> as its whole value, and a variable argument is the bare string $var:<path> — not an object wrapper keyed on $res/$var, and not a plain path.
- `-s --silent` - Do not output anything other then the final output. Useful for scripting.
- `--remote` - Use deployed workspace scripts for PathScript steps instead of local files.
- `--step <step_id:string>` - Run only the named step instead of the whole flow. Honors --data as the step's args and --remote / local-PathScript resolution the same way the full-flow preview does.
@@ -560,11 +560,11 @@ script related commands
- `--json` - Output as JSON (for piping to jq)
- `script show <path:file>` - show a script's content (alias for get)
- `script run <path:file>` - run a script by path
- `-d --data <data:file>` - Inputs specified as a JSON string or a file using @<filename> or stdin using @-.
- `-d --data <data:file>` - Inputs specified as a JSON string or a file using @<filename> or stdin using @-. A resource argument is the bare string $res:<path> as its whole value, and a variable argument is the bare string $var:<path> — not an object wrapper keyed on $res/$var, and not a plain path.
- `-s --silent` - Do not output anything other then the final output. Useful for scripting.
- `--tag <tag:string>` - Override the worker tag the run is dispatched to (e.g. to route it to dev workers instead of the script's default tag).
- `script preview <path:file>` - preview a local script without deploying it. Supports both regular and codebase scripts.
- `-d --data <data:file>` - Inputs specified as a JSON string or a file using @<filename> or stdin using @-.
- `-d --data <data:file>` - Inputs specified as a JSON string or a file using @<filename> or stdin using @-. A resource argument is the bare string $res:<path> as its whole value, and a variable argument is the bare string $var:<path> — not an object wrapper keyed on $res/$var, and not a plain path.
- `-s --silent` - Do not output anything other than the final output. Useful for scripting.
- `--tag <tag:string>` - Override the worker tag the preview is dispatched to (e.g. to route it to dev workers instead of the script's default tag).
- `script new <path:file> <language:string>` - create a new script
+39 -4
View File
@@ -498,6 +498,41 @@ Reference other resources:
}
\`\`\`
## Passing a Resource or Variable as a Run Argument
A script or flow argument typed as a resource (schema \`format: resource-<type>\`) is passed as
the **bare string** \`$res:<path>\` — the whole argument value. Same for a variable, with
\`$var:<path>\`. This applies everywhere job arguments are supplied: \`wmill script run/preview\`,
\`wmill flow run/preview\`, the \`runScriptByPath\` / \`runFlowByPath\` API, a schedule's \`args\`, a
trigger's configured static args.
\`\`\`json
{
"db": "$res:f/databases/postgres_prod",
"api_token": "$var:g/all/api_token"
}
\`\`\`
The reference is resolved when the job runs, under the job's run-as identity — the caller for an
ordinary run, but the configured principal for a schedule, a trigger, or a runnable set to run on
behalf of someone else. The run fails if that identity cannot read the referenced resource or
variable.
**Never wrap it in an object.** The resolver only rewrites a JSON value that *is* a string
starting with \`$res:\` / \`$var:\`; keys are never inspected. These are all wrong and are passed
through to the script unchanged:
\`\`\`json
{ "db": { "$res": "f/databases/postgres_prod" } }
{ "db": { "resource": "f/databases/postgres_prod" } }
{ "db": "f/databases/postgres_prod" }
\`\`\`
The string may sit anywhere a string can — a top-level argument, a nested object field
(\`{ "gh_auth": { "token": "$var:g/all/gh_token" } }\`), or an array element (array elements are
walked only while nested at most two levels deep, and only for arrays of at most 1000 items).
The prefix must be on the string itself.
## Common Resource Types
### PostgreSQL
@@ -3226,11 +3261,11 @@ flow related commands
- \`flow push <file_path:string> <remote_path:string>\` - push a local flow spec. This overrides any remote versions.
- \`--message <message:string>\` - Deployment message
- \`flow run <path:string>\` - run a flow by path.
- \`-d --data <data:string>\` - Inputs specified as a JSON string or a file using @<filename> or stdin using @-.
- \`-d --data <data:string>\` - Inputs specified as a JSON string or a file using @<filename> or stdin using @-. A resource argument is the bare string $res:<path> as its whole value, and a variable argument is the bare string $var:<path> — not an object wrapper keyed on $res/$var, and not a plain path.
- \`-s --silent\` - Do not ouput anything other then the final output. Useful for scripting.
- \`--tag <tag:string>\` - Override the worker tag the run is dispatched to (e.g. to route it to dev workers instead of the flow's default tag).
- \`flow preview <flow_path:string>\` - preview a local flow without deploying it. Runs the flow definition from local files and uses local PathScripts by default. Pass --step <id> to run only one module in isolation (resolves nested steps inside branchone/branchall/forloopflow/whileloopflow plus the special preprocessor/failure modules; supported step types: rawscript, script, flow).
- \`-d --data <data:string>\` - Inputs specified as a JSON string or a file using @<filename> or stdin using @-.
- \`-d --data <data:string>\` - Inputs specified as a JSON string or a file using @<filename> or stdin using @-. A resource argument is the bare string $res:<path> as its whole value, and a variable argument is the bare string $var:<path> — not an object wrapper keyed on $res/$var, and not a plain path.
- \`-s --silent\` - Do not output anything other then the final output. Useful for scripting.
- \`--remote\` - Use deployed workspace scripts for PathScript steps instead of local files.
- \`--step <step_id:string>\` - Run only the named step instead of the whole flow. Honors --data as the step's args and --remote / local-PathScript resolution the same way the full-flow preview does.
@@ -3626,11 +3661,11 @@ script related commands
- \`--json\` - Output as JSON (for piping to jq)
- \`script show <path:file>\` - show a script's content (alias for get)
- \`script run <path:file>\` - run a script by path
- \`-d --data <data:file>\` - Inputs specified as a JSON string or a file using @<filename> or stdin using @-.
- \`-d --data <data:file>\` - Inputs specified as a JSON string or a file using @<filename> or stdin using @-. A resource argument is the bare string $res:<path> as its whole value, and a variable argument is the bare string $var:<path> — not an object wrapper keyed on $res/$var, and not a plain path.
- \`-s --silent\` - Do not output anything other then the final output. Useful for scripting.
- \`--tag <tag:string>\` - Override the worker tag the run is dispatched to (e.g. to route it to dev workers instead of the script's default tag).
- \`script preview <path:file>\` - preview a local script without deploying it. Supports both regular and codebase scripts.
- \`-d --data <data:file>\` - Inputs specified as a JSON string or a file using @<filename> or stdin using @-.
- \`-d --data <data:file>\` - Inputs specified as a JSON string or a file using @<filename> or stdin using @-. A resource argument is the bare string $res:<path> as its whole value, and a variable argument is the bare string $var:<path> — not an object wrapper keyed on $res/$var, and not a plain path.
- \`-s --silent\` - Do not output anything other than the final output. Useful for scripting.
- \`--tag <tag:string>\` - Override the worker tag the preview is dispatched to (e.g. to route it to dev workers instead of the script's default tag).
- \`script new <path:file> <language:string>\` - create a new script
@@ -165,11 +165,11 @@ flow related commands
- `flow push <file_path:string> <remote_path:string>` - push a local flow spec. This overrides any remote versions.
- `--message <message:string>` - Deployment message
- `flow run <path:string>` - run a flow by path.
- `-d --data <data:string>` - Inputs specified as a JSON string or a file using @<filename> or stdin using @-.
- `-d --data <data:string>` - Inputs specified as a JSON string or a file using @<filename> or stdin using @-. A resource argument is the bare string $res:<path> as its whole value, and a variable argument is the bare string $var:<path> — not an object wrapper keyed on $res/$var, and not a plain path.
- `-s --silent` - Do not ouput anything other then the final output. Useful for scripting.
- `--tag <tag:string>` - Override the worker tag the run is dispatched to (e.g. to route it to dev workers instead of the flow's default tag).
- `flow preview <flow_path:string>` - preview a local flow without deploying it. Runs the flow definition from local files and uses local PathScripts by default. Pass --step <id> to run only one module in isolation (resolves nested steps inside branchone/branchall/forloopflow/whileloopflow plus the special preprocessor/failure modules; supported step types: rawscript, script, flow).
- `-d --data <data:string>` - Inputs specified as a JSON string or a file using @<filename> or stdin using @-.
- `-d --data <data:string>` - Inputs specified as a JSON string or a file using @<filename> or stdin using @-. A resource argument is the bare string $res:<path> as its whole value, and a variable argument is the bare string $var:<path> — not an object wrapper keyed on $res/$var, and not a plain path.
- `-s --silent` - Do not output anything other then the final output. Useful for scripting.
- `--remote` - Use deployed workspace scripts for PathScript steps instead of local files.
- `--step <step_id:string>` - Run only the named step instead of the whole flow. Honors --data as the step's args and --remote / local-PathScript resolution the same way the full-flow preview does.
@@ -565,11 +565,11 @@ script related commands
- `--json` - Output as JSON (for piping to jq)
- `script show <path:file>` - show a script's content (alias for get)
- `script run <path:file>` - run a script by path
- `-d --data <data:file>` - Inputs specified as a JSON string or a file using @<filename> or stdin using @-.
- `-d --data <data:file>` - Inputs specified as a JSON string or a file using @<filename> or stdin using @-. A resource argument is the bare string $res:<path> as its whole value, and a variable argument is the bare string $var:<path> — not an object wrapper keyed on $res/$var, and not a plain path.
- `-s --silent` - Do not output anything other then the final output. Useful for scripting.
- `--tag <tag:string>` - Override the worker tag the run is dispatched to (e.g. to route it to dev workers instead of the script's default tag).
- `script preview <path:file>` - preview a local script without deploying it. Supports both regular and codebase scripts.
- `-d --data <data:file>` - Inputs specified as a JSON string or a file using @<filename> or stdin using @-.
- `-d --data <data:file>` - Inputs specified as a JSON string or a file using @<filename> or stdin using @-. A resource argument is the bare string $res:<path> as its whole value, and a variable argument is the bare string $var:<path> — not an object wrapper keyed on $res/$var, and not a plain path.
- `-s --silent` - Do not output anything other than the final output. Useful for scripting.
- `--tag <tag:string>` - Override the worker tag the preview is dispatched to (e.g. to route it to dev workers instead of the script's default tag).
- `script new <path:file> <language:string>` - create a new script
@@ -64,6 +64,41 @@ Reference other resources:
}
```
## Passing a Resource or Variable as a Run Argument
A script or flow argument typed as a resource (schema `format: resource-<type>`) is passed as
the **bare string** `$res:<path>` — the whole argument value. Same for a variable, with
`$var:<path>`. This applies everywhere job arguments are supplied: `wmill script run/preview`,
`wmill flow run/preview`, the `runScriptByPath` / `runFlowByPath` API, a schedule's `args`, a
trigger's configured static args.
```json
{
"db": "$res:f/databases/postgres_prod",
"api_token": "$var:g/all/api_token"
}
```
The reference is resolved when the job runs, under the job's run-as identity — the caller for an
ordinary run, but the configured principal for a schedule, a trigger, or a runnable set to run on
behalf of someone else. The run fails if that identity cannot read the referenced resource or
variable.
**Never wrap it in an object.** The resolver only rewrites a JSON value that *is* a string
starting with `$res:` / `$var:`; keys are never inspected. These are all wrong and are passed
through to the script unchanged:
```json
{ "db": { "$res": "f/databases/postgres_prod" } }
{ "db": { "resource": "f/databases/postgres_prod" } }
{ "db": "f/databases/postgres_prod" }
```
The string may sit anywhere a string can — a top-level argument, a nested object field
(`{ "gh_auth": { "token": "$var:g/all/gh_token" } }`), or an array element (array elements are
walked only while nested at most two levels deep, and only for arrays of at most 1000 items).
The prefix must be on the string itself.
## Common Resource Types
### PostgreSQL
@@ -77,6 +77,8 @@ If the user hasn't already told you to run/test the flow, offer it as a one-sent
If the user already asked to test/run/try the flow in their original request, skip the offer and just execute `wmill flow preview <path> -d '<args>'` directly — pick plausible args from the flow's input schema.
An input typed as a resource (`format: resource-<type>` in the schema) takes the bare string `"$res:<path>"` as its whole value — `-d '{"db": "$res:f/databases/postgres_prod"}'`, not `{"db": {"$res": "..."}}` and not a plain path. Same for a variable, with `"$var:<path>"`. See the `resources` skill.
`wmill flow preview` is safe to run yourself (it does not deploy). `wmill generate-metadata` does not deploy either (it only writes local lock/hash files) but re-resolves deps — offer it and run on agreement, unless the project's `AGENTS.md` opts into automatic metadata. After running it, check the regenerated `.lock` diff and tell the user which inline-script dependency versions changed, so they can catch an unwanted bump before deploying. Only `wmill sync push` deploys; run it only when the user explicitly asks.
### Visual preview
+2
View File
@@ -72,6 +72,8 @@ If the user hasn't already told you to run/test the flow, offer it as a one-sent
If the user already asked to test/run/try the flow in their original request, skip the offer and just execute `wmill flow preview <path> -d '<args>'` directly — pick plausible args from the flow's input schema.
An input typed as a resource (`format: resource-<type>` in the schema) takes the bare string `"$res:<path>"` as its whole value — `-d '{"db": "$res:f/databases/postgres_prod"}'`, not `{"db": {"$res": "..."}}` and not a plain path. Same for a variable, with `"$var:<path>"`. See the `resources` skill.
`wmill flow preview` is safe to run yourself (it does not deploy). `wmill generate-metadata` does not deploy either (it only writes local lock/hash files) but re-resolves deps — offer it and run on agreement, unless the project's `AGENTS.md` opts into automatic metadata. After running it, check the regenerated `.lock` diff and tell the user which inline-script dependency versions changed, so they can catch an unwanted bump before deploying. Only `wmill sync push` deploys; run it only when the user explicitly asks.
### Visual preview
+35
View File
@@ -59,6 +59,41 @@ Reference other resources:
}
```
## Passing a Resource or Variable as a Run Argument
A script or flow argument typed as a resource (schema `format: resource-<type>`) is passed as
the **bare string** `$res:<path>` — the whole argument value. Same for a variable, with
`$var:<path>`. This applies everywhere job arguments are supplied: `wmill script run/preview`,
`wmill flow run/preview`, the `runScriptByPath` / `runFlowByPath` API, a schedule's `args`, a
trigger's configured static args.
```json
{
"db": "$res:f/databases/postgres_prod",
"api_token": "$var:g/all/api_token"
}
```
The reference is resolved when the job runs, under the job's run-as identity — the caller for an
ordinary run, but the configured principal for a schedule, a trigger, or a runnable set to run on
behalf of someone else. The run fails if that identity cannot read the referenced resource or
variable.
**Never wrap it in an object.** The resolver only rewrites a JSON value that *is* a string
starting with `$res:` / `$var:`; keys are never inspected. These are all wrong and are passed
through to the script unchanged:
```json
{ "db": { "$res": "f/databases/postgres_prod" } }
{ "db": { "resource": "f/databases/postgres_prod" } }
{ "db": "f/databases/postgres_prod" }
```
The string may sit anywhere a string can — a top-level argument, a nested object field
(`{ "gh_auth": { "token": "$var:g/all/gh_token" } }`), or an array element (array elements are
walked only while nested at most two levels deep, and only for arrays of at most 1000 items).
The prefix must be on the string itself.
## Common Resource Types
### PostgreSQL