Files
windmill/cli
Guilhem 574775d50c fix: teach the AI the raw-app job bindings, the SDK reference and the draft/deployed split (#10754)
* feat: teach the AI the raw-app job bindings and the draft/deployed split

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* fix: scope the raw-app deploy advice to the referenced item, and stop kind-conversion from stranding fields

The draft/deployed guidance added in the previous commit was read as "deploy the
app too": the agent asked for both the flow and the app and routed a one-item
dependency through the review-and-deploy page. Only the referenced flow or
script has to exist deployed — the preview runs the app's draft — so the prompts,
the `write_app_runnable` warning and the testing rule now say to offer that one
deploy and leave the app a draft.

`buildPersistedRunnable` spread the existing runnable when rewriting it, so
converting a path runnable to inline left `runType`/`path` behind (and the
reverse left `inlineScript`). `isRunnableByName` matches the inline branch
first, so an app "wired to a flow" silently ran stale inline code.

`test_run_app_runnable` now fills ctx-bound inputs with `$ctx:<prop>` the way
RawAppBackgroundRunner does, so a ctx argument no longer arrives missing.

The SDK-reference rationale claimed WM_TOKEN may be unset, that a missing base
URL falls back to localhost, and that a job token is scoped enough to 403 a
hand-rolled REST call. None of the three is true, and it shipped to every
write-script prompt; the text now only says the client configures itself.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* fix: address review round on the raw-app AI instructions

The eval case could pass on the exact answer it exists to reject. Every
`requiredMentionsAnyOf` alternative but one was flow-agnostic, so "the app must
be deployed" satisfied "must be deployed". All alternatives now name the flow,
and a unit test pins that the app-only phrasing fails.

`instanceLine` asserted "self-hosted Community Edition" outside the browser,
where `isCloudHosted()` reads false and the license store is unset — so every
global eval was told that regardless of what it pointed at. It is now emitted
only under BROWSER.

`assistantExpect.forbiddenMentions` defaulted a missing `assistantText` to "",
which passes every entry forever on a mode whose runner does not report it.
It now fails with that as the reason.

`buildPersistedRunnable` carried `schema` across a retarget, so a path runnable
pointed at a new flow kept the previous item's schema and `genWmillTs` typed
`backend.<key>(args)` from the wrong inputs. It survives only while kind and
path both match.

The SDK header claimed "a function that is not listed below does not exist".
`windmill-client` also exports the generated services, and the Python client
exposes `Windmill.get`/`.post`, so an endpoint without a helper had no legal
move. Each language now names its own escape hatch.

`getAppInstructions` said the attached reference carries the TypeScript SDK even
when `language: "python3"` had swapped in the Python one — on the very sentence
telling the model to make that call.

The kind-conversion comment claimed a hybrid runnable "silently runs stale
inline code". It does not: `isRunnableByName`, `isRunnableByPath`,
`convertPersistedToBackendRunnable` and `rawAppPolicy.processRunnable` all
dispatch on `type` alone. The leftovers contradict the runnable's kind rather
than override it, which is what the comment now says.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* fix: address round-2 review nits on the raw-app AI instructions

`flow is deployed` was satisfied both by "once the flow is deployed, the button
works" and by a hallucinated "done — the flow is deployed", which eval mode makes
impossible and the drafts-only judge cannot see. Every alternative now states an
outstanding obligation, and two more real phrasings ("will need to be deployed")
are accepted so a correct answer is not failed on wording.

Condenses the three comment blocks that ran past the four-line limit in
AGENTS.md, and drops two claims inside them that no longer hold: the
`testRunAppRunnable` doc said it runs a runnable the way the app's own frontend
does (it is the editor preview, which a deployed app's stored policy does not
match), and `undeployedRunnableTargets` described its argument as the write
tool's raw input when the call site passes the persisted runnable.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* fix: report the real cause when a test run fails, and label the app-runnable card

Driving `test_run_app_runnable` in a live session surfaced two defects the
API-level check could not see.

`executeTestRun` built its failure message from `error.message`, which the
generated client leaves as the bare status text while the server's message sits
in `body`. A path runnable aimed at an undeployed flow reported "Not Found"
instead of "Not found: flow not found at name u/admin/current_time" — dropping
the one diagnostic the run exists to produce. `formatToolError`, in the same
file and written for exactly this, now does it. This also applies to
test_run_script and test_run_flow, which had the same loss.

The completion card read "Flow test completed successfully" for an app runnable,
because `contextName` doubles as the jobs-tray kind and a path runnable pointing
at a flow really does queue a flow job. A `completionName` override now names
what ran without changing the kind.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* test: pin the deploy expectation against wrong answers, not just correct ones

`deploying the flow` was satisfied by "done deploying the flow" — a deploy the
agent only claims to have made, which eval mode makes impossible and the
drafts-only judge cannot see. Replaced with the prospective forms, and dropped
the same reading from the workflow variant.

Three review rounds each found this same class of hole in the phrasing list, so
the list is now exercised against the wrong answers themselves rather than
eyeballed: naming the app as what needs deploying, claiming the deploy is
already done, claiming to have deployed the flow, and saying nothing about
deploying all have to fail, while four real correct phrasings have to pass. The
test reads the case out of global.yaml, so a future edit to the alternatives is
checked by it.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* test: drop the tense-neutral deploy alternatives and cover completed claims

A gerund after a preposition carries no tense, so `before`/`after`/`by deploying
the flow` all match a deploy the agent only claims to have made ("after
deploying the flow, I clicked the button and it returns the greeting") just as
the bare gerund did. All three are gone rather than swapped for whichever reads
least badly, and the two completed-deploy phrasings are now negative fixtures.
The remaining alternatives are imperative or obligational, which a claim of
having already deployed cannot satisfy.

Condenses the two comments this list carries: the YAML block to four lines, and
the test's rationale to the durable constraint about substring matching.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* fix: encrypt sensitive inputs when test-running an app runnable

`test_run_app_runnable` sent `force_viewer_static_fields` but not
`force_viewer_sensitive_inputs`, which every other preview path derives from
the runnable's `sensitive` user fields. That list is the only thing driving the
encryption loop in apps.rs, so testing a runnable with a sensitive input wrote
the real value into the job's args in plaintext, readable by anyone with run
access to the workspace.

Verified against a running EE instance. With the list, `api_key` is stored as
`$encrypted:mvqtSRI9…` and the sentinel appears nowhere in the job record;
without it, the sentinel is readable in run details. A non-sensitive field is
left plaintext either way.

The tool claims parity with the editor preview, so it uses that same filter
(`type == 'user' && sensitive`) and omits the field entirely when empty.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
Co-authored-by: Ruben Fiszel <ruben@windmill.dev>
2026-08-20 11:17:33 +02:00
..
2022-11-01 15:53:28 +01:00
2023-02-03 19:49:46 +01:00
2026-02-23 09:09:16 +00:00
2025-12-26 20:32:42 +00:00

Windmill CLI

A simple CLI allowing interactions with windmill from the command line.

You can find more information in Windmill Docs

Installation

Install the wmill CLI tool using npm install -g windmill-cli.

Update to the latest version using wmill upgrade.

Workspaces

To get started run wmill workspace add or use the instructions from the workspace settings.

Running Flows & Scripts

Run a script or flow using wmill flow/script run u/username/path/to/script and pass any inputs using --data + Inputs specified as a JSON string or a file using @ <filename> or stdin using @-.

Curl-style syntax using -d @- for stdin or -d @<filename> is also supported.

Flow Steps and Logs will be streamed during execution automatically.

CLI input example

Pushing Resources, Scripts & More

The CLI can push specifications to a windmill instance. See the examples/ folder for formats.

Switch to a different workspace

wmill workspace switch <workspace_name>

Sync a workspace

Pull

wmill sync pull

Push

wmill sync push

We recommend using the --yaml option to use yaml instead of json as the encoding format. Yaml will be made the default soon.

Pushing individual files

You can push individual resources using wmill <type> push <file_name> \<remote_name\>. This does not require a special folder layout or file name, as this is given at runtime.

Listing

All commands support listing by just not providing a subcommand, ie wmill script will result in a list of scripts. Some allow additional options, learn about this by specifying --help.

User Management

You can add & remove users via wmill user add/remove, and list them using wmill user

Pulling

You can pull the entire workspace using wmill pull

Completion

The CLI comes with completions out of the box via wmill completions <shell>. (Via cliffy)

Bash

To enable bash completions add the following line to your ~/.bashrc:

source <(wmill completions bash)

Fish

To enable fish completions add the following line to your ~/.config/fish/config.fish:

source (wmill completions fish | psub)

Zsh

To enable zsh completions add the following line to your ~/.zshrc:

source <(wmill completions zsh)

Development

AI Guidance Variants

wmill init can now materialize alternate AI guidance bundles without changing the generated defaults in the repo, but this is exposed as internal env-var overrides rather than public CLI flags.

Examples:

WMILL_INIT_AI_SKILLS_SOURCE=/path/to/custom/skills wmill init --use-default
WMILL_INIT_AI_SKILLS_SOURCE=/path/to/custom/skills WMILL_INIT_AI_AGENTS_SOURCE=/path/to/AGENTS.md wmill init --use-default
WMILL_INIT_AI_SKILLS_SOURCE=/path/to/custom/skills WMILL_INIT_AI_CLAUDE_SOURCE=/path/to/CLAUDE.md wmill init --use-default

This is the same guidance-writing path used by the benchmark CLI under ai_evals/, so the benchmark harness and wmill init now generate the same project guidance shape:

  • AGENTS.md
  • CLAUDE.md
  • .agents/skills/*
  • .claude/skills/*

windmill-yaml-validator

wmill lint imports the sibling windmill-yaml-validator package from source rather than from npm, so its schemas always match the OpenAPI specs of the current checkout. bun install regenerates them through this package's preinstall script; run it again after editing openflow.openapi.yaml or backend/windmill-api/openapi.yaml:

npm --prefix ../windmill-yaml-validator run gen

Running Tests

Prerequisites:

  • PostgreSQL running locally (default: postgres://postgres:changeme@localhost:5432)
  • Rust toolchain installed

Run tests locally (full features):

bun test test/

Run tests in CI mode (minimal features, skips EE tests):

CI_MINIMAL_FEATURES=true bun test test/
Variable Description
CI_MINIMAL_FEATURES Set to true to skip EE-dependent tests
DATABASE_URL PostgreSQL connection string
EE_LICENSE_KEY Enterprise license key for EE features