mirror of
https://github.com/windmill-labs/windmill.git
synced 2026-10-14 00:02:55 +00:00
0e807fb1dd
* feat(datatables): put a data table's connection under Postgres roles
A data table backed by the instance database resolved to exactly one Postgres connection,
`custom_instance_user`, for everyone who could reach it at all. There was no way to say
this job reads, that one writes, this one never sees the salaries table.
A data table role is now a real Postgres login on the cluster, defined once for the
instance by a superadmin and named exactly as they named it. A script that declares
`-- role analytics` connects as `analytics`, and Postgres decides what it may touch —
grants are ordinary SQL. Windmill answers only "may this caller ask for this role", from
the tenant lists on the data table entry: `u/alice`, `g/analysts`, `f/finance` or `*`.
A data table with no `permissions` block behaves exactly as before.
Everything that opens a connection on someone's behalf goes through one chokepoint,
`get_datatable_resource_from_db`, which takes the identity explicitly and fails closed when
there is none. The role logs in as itself — never `SET ROLE`, which a script could
`RESET ROLE` its way out of.
A fork's data table entry becomes a pointer at the workspace that governs it rather than a
copy of it. The settings clone used to hand a fork a byte-identical entry naming the
parent's database, which a fork admin could edit to grant themselves `admin` there; a
pointer has nothing local to edit, and its tenants are evaluated as a member of the
governing workspace, by email. `permissions` is stripped from the workspace export and
ignored on import: tenants name principals of one workspace, and a settings push is not
where an access decision should be made.
Operations that see the whole database whatever the roles grant stay with the governing
workspace's admins: editing the roles, a migration that declares none, and opening a
replication stream for a Postgres trigger or capture.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012ti5HyeTikPMYyW8YSdiHR
* fix(datatables): gate the paths that reach a whole database as admin
Auditing what still resolved through the unchecked resolver turned up three that act for a
caller and hand back the admin connection: `resolve_pg_source_checked` (behind schema
export, the full-schema read, database creation, import and the forked-database drop), the
connection test, and the schema snapshot a fork clone takes of its parent. On a data table
under roles each let any workspace member — or a fork admin who is nobody in the governing
workspace — read or copy the whole database whatever its roles grant.
All three now require admin reach on the governing workspace. A dump taken under a
restricted role would be a silently truncated copy rather than an error, so refusing is the
only right answer for the copy paths.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012ti5HyeTikPMYyW8YSdiHR
* fix(datatables): confine roles to the instance database, and stop a fork reaching the parent's bookkeeping
A data table role is a login on Windmill's own Postgres. Nothing stopped a workspace admin
putting a *resource-backed* data table under roles, at which point the executor dialled the
host that resource names — one the admin chose — with the role's real cluster password, and
`CONNECT` is granted to every registered instance database. Both ends now refuse: the
permissions endpoint rejects the save, and the chokepoint refuses to substitute credentials
on a non-instance entry rather than trusting the record it read.
Two more places reached the governing database without answering to it. The initial-migration
generator returned a `pg_dump` of the whole schema to any member. And the migration
rename/delete cascade followed a fork's pointer into the parent, so a fork admin renaming or
removing their own local entry relabelled or wiped the parent's `_wm_migrations` — after
which the parent re-runs every migration from zero. The remote half is now skipped when the
entry resolves into another workspace, which is also just correct: a fork renaming what it
calls a data table changes nothing about the data table.
Also: revoking a tenant now bounces the replication streams of every workspace holding an
entry that resolves here, not only the governing one, so a fork's trigger stops rather than
living on inside its open connection; the instance role catalog and the governing workspace's
tenant lists are no longer returned to someone who cannot edit them; and the tenant rename
dedup collapses non-adjacent duplicates, per role rather than once any role changed.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012ti5HyeTikPMYyW8YSdiHR
* fix(datatables): fail loudly where a role or a pointer can be left half-recorded
Three ways the feature could end up in a state nobody could see or undo.
Creating a role writes the cluster first and the catalog second, but the catalog write was an
`UPDATE` that matched nothing when the instance Postgres settings row was absent — leaving a
live login with a password nobody recorded: invisible to the catalog, un-recreatable because
the name is taken, and un-deletable because there is no entry to delete. It now errors, so
the operation is retryable once the row is restored.
Deleting a workspace only nulls the fork lineage; the data table entries pointing at it are
left resolving to nothing. Sweeping them is not an option — turning a pointer back into a copy
would hand each fork the database outright — so the delete now names the data tables it
stranded, and resolving one says which workspace is missing rather than reporting a data table
this workspace never had.
`InstanceDatatableRole` derived `Debug` while holding a Postgres password; it is now
hand-written so `{:?}` on the catalog cannot put a live credential in a log line.
Adds the two branches the reviews found unpinned: a caller who is not a member of the
governing workspace at all, and `NoIdentity` — the compatibility path for an agent worker that
predates this and sends no job id.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012ti5HyeTikPMYyW8YSdiHR
* fix(datatables): unbreak two operator messages and two comments that described other code
The two strings this branch added for states an operator hits once — the catalog write that
matched nothing, and the delete that stranded a pointer — were collapsed from their multi-line
form with the indentation left in, so both rendered with a fourteen-space gap mid-sentence.
`list_datatables` claimed to report a chain it cannot follow and then dropped it; it does drop
it, and the comment now says why that is the right place to stay quiet. The non-superadmin
check in `edit_datatable_config` was introduced as also covering references, which it does not
and need not: `reference` is overwritten from the stored entry for every caller before the
check runs.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012ti5HyeTikPMYyW8YSdiHR
* fix(datatables): serialize role catalog mutations, and state each helper's authorization contract
The catalog is one JSON document, so create, rename, enable and delete are all
read-modify-write. Two concurrent creates read the same snapshot, both succeed in the
cluster, and the second write drops the first — leaving a live Postgres login with a password
nobody recorded, which is the exact state the delete path exists to prevent. Every mutation
now runs in one transaction holding an advisory lock across the read, the cluster DDL and the
write, so a lost update cannot happen and a failure rolls the whole thing back. The DDL
helpers take that transaction rather than the pool, which is what makes the lock cover them.
Their statements moved off `sqlx::raw_sql`: the simple protocol is only needed for genuinely
multi-statement SQL, and its future is not `Send`, which an axum handler holding the
transaction requires. Each of these is one statement anyway.
The new cross-crate surface now says what callers must do. `read_role_catalog` returns
plaintext credentials; `create`/`rename`/`set_login`/`drop_instance_role` and
`converge_connect_grants` mutate cluster-wide state; `read_datatable_entry` reads a workspace's
raw config. All of them are superadmin-gated by their current handlers, but nothing said so at
the definition, which is where the next caller looks.
Also: the roles table reloads after a failed login toggle instead of leaving it claiming a flip
that did not land; the rename affordance is the design-system `Button`, not a raw one; and
`resolve_datatable_pg_as_caller` drops a `role` parameter no caller ever filled — browsing
resolves as the data table's default until the database manager grows a picker.
Why role passwords stay a plain `String` while the instance user's password beside them is a
`StringOrSecretRef`, asked three times across reviews: that one is a secret ref because an
operator supplies it and may want it from their own backend, while these are minted here and
never entered by anyone, so there is nothing for a ref to point at. Encrypting generated
secrets at rest is a separate change that would take the replication password with it. Now
said at the field.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012ti5HyeTikPMYyW8YSdiHR
* fix(datatables): give the role catalog its own row, out of reach of the config machinery
Putting it inside `custom_instance_pg_databases` was the wrong call, and it cost two ways.
The catalog serializes a generated Postgres password per role, and that row is the
operator-facing instance config, so the passwords reached `get_instance_config` and its YAML
editor — a live cluster credential in a response body, a UI field and any log of either.
Worse in the other direction: `to_settings_map` strips the catalog, so a full-row upsert of
that key writes the row back without it and the catalog is gone, while the cluster keeps every
login it described.
`custom_instance_replication_pwd` is the precedent and says exactly why — a generated secret,
written only by the server, never operator-authored, hidden so the config machinery cannot
read, rewrite or drop it. The catalog is the same thing, so it now has the same shape:
`datatable_roles`, in `HIDDEN_SETTINGS`, `PROTECTED_SETTINGS` and the agent-worker denylist.
No redaction to keep in step with three code paths, and no way for a neighbouring write to
take it out.
Two races on the same shared documents. `edit_datatable_config` read the stored data tables
outside its transaction and then wrote the whole `datatable` document, so a permissions save
committing in between was silently rolled back; it now reads under `FOR UPDATE`. And
`set_datatable_permissions` validated role ids against the catalog before opening its
transaction, so a deletion in between let it write a deleted role back — including as the
default, which every later job then fails on; it now holds the catalog lock and the settings
row across validation and write.
Completes the authorization contracts the previous commit claimed but did not finish:
`read_datatable_entry` (which it named and missed), `resolve_governing_datatable`, whose whole
job is to answer for a workspace the caller may not belong to, and
`converge_connect_grants_with`, which had not inherited its wrapper's.
Also the generic Python SDK reference: `_format_py_params` learned the bare `*` last time, but
`extract_py_functions` is a second formatter and still rendered `datatable(name, role)`, so
code written from that page passed a keyword-only argument positionally.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012ti5HyeTikPMYyW8YSdiHR
* fix(datatables): make the concurrency test pin the handlers, and the contracts describe what is enforced
The concurrency test reimplemented the read-modify-write inline, so deleting the lock from all
three handlers left it green — it pinned Postgres, not the code it was written for. It now
drives `create_datatable_role` twice concurrently and asserts the catalog kept both names.
Checked the way the last one should have been: removing the lock from the handler makes it
fail with "wmtest_a_… is a live cluster login the catalog forgot".
The contracts added last commit were stricter than this PR's own callers, which is worse than
none — the next reader sees a rule already broken and learns to ignore it.
`read_role_catalog` said superadmin-only while two of its four callers are open to any
workspace member, and `converge_connect_grants` said superadmin while
`set_datatable_permissions` reaches it as a workspace admin. Both were fine on substance: the
rule that actually holds is about the credential never reaching a response, log, audit record
or export, not about who may call. They now say that. `read_datatable_entry` gets the same
treatment rather than the one the earlier message claimed for it: it is the primitive every
resolution goes through, so it is deliberately open, and what must not escape is `permissions`
— it names the governing workspace's users, groups and folders.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012ti5HyeTikPMYyW8YSdiHR
* fix(datatables): close the last ways a role or a pointer can be left pointing at nothing
The raw settings readers hand back whatever is in the row, so moving the catalog into its own
`global_settings` key protected the config machinery and left `GET /settings/global/datatable_roles`
and the settings listing returning every live password. Both now filter that one key. The
neighbouring `custom_instance_replication_pwd` has the same shape and is not touched here: it
predates this and widening the fix to it is a decision about an operator workflow, not a
consequence of this change.
Three ways a save could leave something resolving to nothing:
A permissioned data table could be moved to a PostgreSQL resource. The block was carried across
as a server-owned field, the runtime refuses roles on a resource-backed table, so the save
succeeded and every job afterwards failed. Refused instead — turning roles off first is one step,
and it keeps discarding an access decision something somebody chose.
Renaming a governing data table left every fork pointing at the old name: the data table
disappears from their pickers and their jobs stop, with nothing in the renaming workspace to
suggest why. The rename now follows into the pointers in the same transaction.
Deleting one cannot be followed the same way, so it is reported instead — the response names what
it stranded, the way deleting a workspace does, and the fork's own error already says which
workspace is gone.
Also: `ensure_instance_db_grant_options_unchecked` claimed superadmin while the permissions
handler reaches it as a workspace admin (the same class fixed last commit, one instance missed);
the role entry kept an `instance_config_schema` derive it no longer needs; `write_role_catalog`
was the one writer of that table not stamping `updated_at`; and the concurrency test dropped its
roles only on success — a failing run is exactly the one that creates them without recording them.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012ti5HyeTikPMYyW8YSdiHR
* refactor(datatables): put the role catalog in its own table, not in global_settings
Five findings across three rounds were all the same choice. A set of live Postgres credentials
was living in `global_settings`, which has generic read, list, write, config-export and CLI
round-trip paths that know nothing about what they carry: the passwords reached the instance
config and its YAML editor, a full-row upsert of a neighbouring key erased the catalog,
`GET /settings/global/{key}` and the settings listing returned them raw, and this round the
redaction that fixed the last two turned `wmill instance push` into something that wipes every
password — a fix breaking the assumption the previous fix made. `POST /settings/global/datatable_roles`
could also empty it outside the lock.
The approved plan offered a table or `global_settings`, so this is the other option it already
allowed rather than a new design. `datatable_role` is a table: no generic settings path can read
it, list it, export it, write it or round-trip it, so none of the five needs a guard. The
redaction, the hidden/protected/agent-denylist entries and the JSON document all go with it.
One row per role also removes the read-modify-write the concurrency work was about: two
concurrent creates are two inserts, and the unique index on `name` is what settles a collision.
The advisory lock stays for the one window rows do not cover — `CREATE ROLE` is invisible to
another transaction until commit, so without it both creates pass their `pg_roles` check.
Also from this round: rename mappings are checked against the configuration they claim to
describe, since fork pointers are rewritten from them — a caller could otherwise submit
`main -> missing` against an unchanged config and repoint every fork of `main` at a name nothing
has, and `A -> B` plus `B -> C` moved what pointed at `A` all the way to `C`. And the warning
naming forks a delete stranded reached the response but not the screen: both the data table
settings save and the workspace delete now show it.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012ti5HyeTikPMYyW8YSdiHR
* fix(datatables): validate a rename against the save it describes, and re-check under the locks
Three from the round, all about deciding on state that could already have moved.
A permission save resolved the data table and checked it was instance-backed before taking any
lock, then wrote under one. A config save committing in between could move the table onto a
PostgreSQL resource — recreating exactly what the transition guard refuses — or rename it, in
which case the write targeted a key that no longer existed and reported success having changed
nothing. It now re-resolves and re-checks on the locked state.
Rename validation checked that the source existed before and the target existed after, which
still accepts `main -> decoy` against a save that keeps both: every fork of `main` then follows
onto a different data table, silently, because it keeps resolving. The rule is now the actual
old-to-new key transition — a source may only survive if another rename took its name, and a
target may only pre-exist if another rename freed it. That also stops two sources sharing one
target, and it admits a swap, which the previous guard refused: `datatables` is keyed by name, so
a swap cannot be done one save at a time, and refusing it was a regression against main. The
pointer cascade now runs in two passes through a temporary name, the way the migration cascade
one layer down already handles the same shape, so `A -> B` with `B -> C` moves each pointer once
from what it named before the save.
The tenant mutators say what they are for: they write an access decision for any workspace named,
with an arbitrary mutation, and exist for the transaction that frees or renames a principal.
Editing a decision on purpose belongs in the permissions endpoint.
Carried in the same change: the stranded-fork list is a field rather than a phrase to grep out of
a success string; the pointer cascade matches with `EXISTS` instead of a `LIKE` over the whole
document, so a workspace whose pointers name something else is not rewritten to a byte-identical
value under an exclusive lock; and `InstanceDatatableRole` drops the serde derives left over from
the JSON document, one of which would emit `pwd`.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012ti5HyeTikPMYyW8YSdiHR
* fix(datatables): cascade on the leave route that is used, gate migrations before the admin connection, and drop a role atomically
The tenant cascade on leaving went onto `/users/leave`. The UI and the generated client call
`/workspaces/leave` — a different handler in a different crate with the same name — which
deleted the membership and left `u/<username>` in the tenant lists. Leaving and rejoining
therefore restored the access the leave was supposed to end, and a later account taking the
username would have inherited it. The regression test drives the route the client actually
calls; without the fix it fails with "leaving kept the tenant".
The migration endpoints authorized too late. `run_datatable_migrations` opened the data table's
admin connection, created `_wm_migrations` and read it before reaching the per-migration role
check — so with nothing pending, nothing was checked at all. Rollback returned before its check
when nothing was applied, and the status endpoint had none. All three now ask, before any
connection is opened, whether the caller can reach the data table as any role at all; which role
a given migration runs as is still decided per migration, and by the executor after that.
Deleting a role committed the cluster drop and the catalog row, then swept the tenant lists in
separate transactions. A sweep failing part-way left workspaces naming a role nothing can connect
as, while the retry answered `NotFound` because the catalog entry was already gone. The sweep now
runs in the same transaction, so the drop, the row and every tenant list commit together.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012ti5HyeTikPMYyW8YSdiHR
* fix(datatables): refuse to copy a data table that is under roles
pg_dump carries no roles and the import runs with --no-privileges, so a copied
data table arrives owned by the admin connection with no GRANT for any role.
The settings clone brings `permissions` across, so the fork's tenants pass
Windmill's check, connect as the role they were given, and are denied by
Postgres on everything: an entry that reads as configured and answers nothing.
Refuse the copy — in the import endpoint before any data moves, and in the fork
path the CLI takes. Replaying the source's owners and ACLs into the clone is
what lifts this, and is a change of its own. Dropping `permissions` from the
copy instead would be the unsafe half, since the copy holds the parent's rows.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012ti5HyeTikPMYyW8YSdiHR
* fix(datatables): refuse the clone's database too, not only its data
A clone is two endpoints: `create_pg_database` then `import_pg_database`. Only
the second refused a data table under roles, so a fork asking to clone one
created and registered an empty `wm_fork_…` instance database and then failed —
and nothing collects it, since `drop_forked_datatable_databases` only drops
entries carrying `forked_from` and no entry names this one.
Refuse in both, so the clone stops before a database exists.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012ti5HyeTikPMYyW8YSdiHR
* nit worker error msg
* fix pg_dump stuck on version 17 on nix
* fix(datatables): refuse a malformed role annotation instead of ignoring it
`-- Role operator`, `-- role operator;` and `-- role operator -- why` all failed
the annotation parser's exact-match rule, so the query fell through to the data
table's default role and ran, silently, under a login the author did not choose.
Naming a role exists precisely to not do that.
A leading comment whose first word is `role` is now an annotation attempt: the
keyword matches case-insensitively, one trailing `;` is tolerated, and anything
else is an error naming the line. Only callers that already know the target is a
`datatable://` reference ever run this, so ordinary SQL keeps its comments.
Also bumps the dev shell's postgres client to 18 — it trailed the server the dev
database runs, which takes out every data table export, clone and fork-with-data.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012ti5HyeTikPMYyW8YSdiHR
* fix(datatables): refuse a malformed role query string instead of ignoring it
`?Role=analytics`, `?role=` and `?x=1&role=…` all fell through the reference
parser's exact-match rule, so the connection resolved to the data table's default
role and ran under a login the caller never asked for — the URI half of the same
trap as a malformed `-- role` annotation.
The key now matches case-insensitively, and anything else in the query string is
an error naming it; `role` is the only parameter a reference takes. Callers that
only need the entry keep a lenient `datatable_ref_name`, since they never act on
the role. The DuckDB `ATTACH` parser propagates it rather than attaching under
the default.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012ti5HyeTikPMYyW8YSdiHR
* fix(datatables): carry the role annotation into the row_to_json retry
The retry rebuilds its SQL from `pruneComments(code)`, so the leading comment
block never reached the second attempt — and with it the `-- role <name>` line
that decides which login the query runs as. The retry connected as the data
table's default role instead, so a query the first attempt was denied could
succeed on the second, reported as "recovered with the row_to_json fix".
Carry the leading comment block over. The retry itself is unchanged.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BjfMkJyKzodxkobqGZ6Lqb
* chore(datatables): don't mount the roles UI until the ACL editor lands
Enforcement ships first. The permissions drawer is what turns roles on, and the
catalog section is what creates them — both are only useful once there is a way
to grant a role the privileges it needs, which arrives with the ACL editor. Left
mounted they would offer a feature whose other half does not exist.
The two components are complete and reviewed; only their call sites here are
commented out, with a note pointing the follow-up PRs at them.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BjfMkJyKzodxkobqGZ6Lqb
* fix(datatables): honour `-- role: x`, and fix the DuckDB attach test
Two review findings, both real.
`attach_datatable_parses_name_and_role` never compiled: `parse_attach_datatable`
returns `Result<Option<_>>` now and one call site kept a single `unwrap`. Its
`?Role=analytics` case also asserted a refusal, contradicting the parser in the
same commit, which matches the key case-insensitively. Replaced with the cases
that are genuinely malformed, and a positive one for the cased key.
`-- role: analytics` fell through to the default role — the silent fallback the
strict parser exists to remove, for the spelling most likely to be typed. The
keyword now accepts an optional colon, attached or spaced, while a word that
merely starts with it (`rolebased`) is still not an attempt.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BjfMkJyKzodxkobqGZ6Lqb
* fix(datatables): clone a fork's pointer instead of failing after the copy
Forking a fork with cloning left an orphan database. The preflight resolves the
pointer and sees the governing entry, so both endpoints ran and filled the new
database; `apply_forked_datatable` then refused the inherited pointer and rolled
the fork back, stranding a registered `wm_fork_*` that no entry names and whose
name blocks the retry.
Refusing earlier would have been the smaller change, but forking a fork and
cloning worked before pointers existed, so it would trade an orphan for a
regression. Resolve what the pointer names and write the terminal entry the
clone needs: the whole `database` object rather than a patch of its
`resource_path`, since a pointer has none, and `reference` removed with it.
Also accepts `-- role=x` and `-- Role = x`, two more spellings that fell through
to the default role.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BjfMkJyKzodxkobqGZ6Lqb
* fix(datatables): refuse to roll back the catalog while roles exist
The down migration dropped the table and left every role behind: live Postgres
logins whose passwords only that table carried, so after a revert Windmill could
neither use, disable nor delete them, and re-applying could not recreate them
because the names were taken. Cleaning up here is not possible either — dropping
a role means reassigning what it owns in every instance database, and a
migration runs in one — so it now refuses while the catalog is non-empty and
says to delete the roles through instance settings, which does the cluster work.
Also enforces the instance-only invariant the resolved-pointer clone relies on
rather than only asserting it in a comment.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BjfMkJyKzodxkobqGZ6Lqb
* refactor(datatables): settle clonability in one place, before anything is created
A clone is three stages a workspace apart — `create_pg_database`, then
`import_pg_database`, then `apply_forked_datatable` inside the fork transaction.
Only the third can roll back, and `CREATE DATABASE` is not transactional, so any
refusal that lives there strands a registered `wm_fork_*` that no entry names
and whose name blocks the retry.
That orphan has now been fixed three times, most recently reintroduced by a
guard added one commit ago. Patching each new refusal into the first endpoint is
not the fix; having two places that can refuse is. `ensure_datatable_is_clonable`
now answers every reason a copy can be refused and returns what it resolved, and
the stage that writes the entry only does the work.
Also takes an ACCESS EXCLUSIVE lock before the rollback guard counts, so a role
created concurrently cannot slip between the check and the drop.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BjfMkJyKzodxkobqGZ6Lqb
* fix(datatables): let a retried clone reclaim its own leftover database
A clone creates its target database one request before it copies into it, and
the fork that would name it is written a request after that. Any failure in
between — a pg_dump error, a bad restore, a dropped connection, the source's
roles changing mid-flow — left a registered `wm_fork_*` that no entry names,
and every retry then failed on its name. This predates data table roles.
`create_pg_database` now reclaims such a leftover before creating: only a
`wm_fork_*` database Windmill registered as a data table database and that no
data table or ducklake entry names, in any workspace, archived ones included.
The drop never terminates connections, so a clone still copying into it makes
the reclaim fail instead of being cut off. It is limited to callers who
administer the source — reaching it is not enough, since on a data table
without roles every member reaches it — and anyone else gets the refusal an
existing database always got.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* Revert "fix(datatables): let a retried clone reclaim its own leftover database"
This reverts commit 7dd3275a10.
The reclaim tied the caller to the source they administer, but not to the
database it dropped. Between another workspace's import and its final fork
request, that workspace's target is full, registered, unnamed and has no open
connection, so an admin of any instance data table could name it and have it
dropped and recreated empty. The victim's fork would then commit pointing at
the empty copy. Safe reclaim needs durable clone ownership and serialization
with the request that names the database; until then the leftover stays, as it
did before this PR.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* docs(datatables): record the stale clone database as a known limitation
A clone is three requests and `CREATE DATABASE` is not transactional, so a
failure after the first leaves a registered `wm_fork_*` behind, as it did
before data table roles. Accepted for this PR: it is harmless to data and goes
away once the clone is a single server-side operation.
The comment also records why the obvious fix is wrong: reclaiming the leftover
on retry, without durable clone ownership, can drop another workspace's fully
copied database between its import and its final fork request.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* fix(datatables): bounce the streams reading a data table when it is deleted
Deleting a governing data table, or the workspace that holds it, only collected
the fork pointers it stranded, for the warning. A Postgres trigger or capture
already streaming through one of those pointers kept the replication connection
it opened while the pointer still resolved, so it went on dispatching the
governing database's rows after the fork lost access — until its connection
happened to restart. The governing workspace's own streams on a deleted entry
did the same.
Both deletion paths now bounce the affected listeners inside their own
transaction, through the helper a permission change already uses, so a
listener that reconnects re-resolves the entry and finds it gone. The helper is
split so a caller can pass the (workspace, local name) pairs it already holds.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* fix(datatables): keep the fork schema baseline, and bounce streams on every removal
Three fixes from review.
`edit_datatable_config` took `forked_from` wholesale from the stored entry, so
the fork schema diff's save of an advanced baseline was silently discarded and
an applied change was offered again. Whether an entry carries a clone stamp is
still carried from the store, since that is what marks its database droppable,
but the baseline inside it is now taken from the request.
The stranded-pointer warning and the stream bounce ran over the optional
`deleted_datatables` hint, which the settings-sync CLI never sends, so removing
a governing data table through `wmill` bounced nothing. Removals are now derived
from the stored configuration against the saved one.
`delete_workspace` read the pointers to bounce before its transaction, so a fork
committing a pointer during the deletion was missed. The read now happens inside
the transaction, after the workspace row is deleted: a fork's insert key-share
locks that row through its parent foreign key, so it is either seen or fails on
the missing parent.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* refactor(datatables): keep Postgres triggers and data table roles apart
A replication stream reads every row of every table whatever the data table's
roles grant, and its listener checks access only when it connects. Rather than
chase every way access can change and bounce the streams each one affects, a
data table now carries one or the other:
- a Postgres trigger or capture cannot be created on, or connect to, a data
table under roles;
- roles cannot be turned on while an enabled trigger or a live capture reads
the data table, its own or a fork's through its pointer. The refusal names
each one to disable.
This removes the stream bounces on roles edits and on data table and workspace
deletion, and the trigger gate that admitted admins. The fork schema baseline
fix from the same review round is kept.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* fix(datatables): refuse a Postgres trigger on a data table under roles when it is saved
Creating or editing a trigger that points at a data table under roles was
accepted, and its listener then retried the refused connection every 30
seconds forever. The save is now refused, and a trigger that reaches such a
data table anyway (re-enabled, or cloned into a fork) is disabled by its
listener with the reason, as a missing replication slot is.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* fix(datatables): disable a data table role before deleting it
Deleting a role reassigns and drops what it owns in each registered
database on its own connection, and each of those passes commits as it
goes. A database failing part-way left the role enabled in the catalog and
able to log in, but already stripped in the databases reached before it.
The role is now disabled in its own commit first, so a failed delete
leaves a disabled role to retry.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* fix(datatables): serialize roles going on with a stream starting
Turning roles on looked for enabled triggers and live captures once,
without a lock anything starting a stream also took. A trigger enabled in
that window could have its listener connect before roles committed, and a
healthy listener never checks again. Both transitions now serialize on one
advisory lock: roles going on hold it exclusive while they look, and
trigger create, edit and enable, and capture setup and ping hold it shared
while they commit. Either the look sees the stream, or the listener
connects after roles are committed and refuses.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* fix(datatables): wait out live listeners, and resolve stored names containing `?`
Turning roles on counted a trigger as gone once disabled, and a capture
once its client stopped pinging, but the listener keeps its replication
connection until its next heartbeat notices. A trigger or capture whose
listener pinged in the last 15 seconds, the window a server holds a
listener for, now still counts as streaming.
Data table names could contain `?` before they were restricted, and such
entries are still stored. Splitting `?role=` off a reference misread them:
`a?b` became `a` with an unknown parameter, and the clone checks looked at
a different entry than the one copied. An entry stored under the whole
reference is now looked up first, in the Postgres executor, DuckDB ATTACH
and the clone checks. Agent workers cannot read the workspace and keep
the strict parse, which refuses such a name rather than misreading it.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* fix(datatables): warn when a settings sync strands fork pointers
A settings save reported the fork pointers left resolving to nothing only
for the names in `deleted_datatables`, which `wmill sync push` never sends.
The save now works out what it removed from the locked entries, and the
CLI prints the stranded pointers it returns.
Also correct the replication helper's contract: no role or admin check
makes a replication connection safe, so a data table under roles is
refused outright rather than gated as an admin operation.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BjfMkJyKzodxkobqGZ6Lqb
* fix(datatables): refuse a save that drops a data table's roles through an undeclared rename
A data table's roles follow its entry only through a declared rename. A
settings sync sends the whole map and never declares one, so renaming a
data table under roles there read as a delete and a new entry on the same
database: the new entry carried no roles, and every caller connected as
admin. Such a save is now refused, naming both entries.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BjfMkJyKzodxkobqGZ6Lqb
* fix(datatables): no entry without roles may newly reach a database under roles
The previous guard only caught a new name replacing an entry under roles.
A whole-map save could also repoint an existing entry without roles at
that database, or another workspace could point one there, and every
caller of that entry would connect as admin. The rule is now stated on
the saved entries: one that carries no roles and newly points at an
instance database any entry under roles uses, in this workspace or
another, is refused. A declared rename carries its roles and passes.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BjfMkJyKzodxkobqGZ6Lqb
* feat(datatables): move data table role catalog and resolution to the enterprise edition
Roles are an Enterprise Edition feature. The catalog, the Postgres logins,
CONNECT convergence, tenant evaluation and the role half of connection
resolution move to windmill-ee-private. Every public function keeps its path
and signature and forwards through datatable_roles_oss, which re-exports the
enterprise implementation or, without it, refuses.
Without the enterprise edition a data table under roles, or a caller naming a
role, is refused a connection rather than resolved as admin, and the reach and
admin-access checks refuse one under roles. A data table not under roles
resolves as before in every edition, and an instance database keeps the
CONNECT grants it was created with. The catalog lock, the stream lock, the
tenant cascades and the permissions stripping stay in OSS: they only restrict.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BjfMkJyKzodxkobqGZ6Lqb
* feat(datatables): move the data table permissions endpoints to the enterprise edition
The permissions read, save and usable-roles handlers move to
windmill-ee-private; the routes stay registered and, without the enterprise
edition, answer that data table roles are an Enterprise Edition feature.
ensure_governs_datatable and ensure_reaches_datatable keep their paths: the
first refuses, the second passes a data table not under roles.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BjfMkJyKzodxkobqGZ6Lqb
* feat(datatables): move the data table role catalog endpoints to the enterprise edition
The superadmin list, create, update and delete handlers move to
windmill-ee-private. The routes stay registered and, without the enterprise
edition, refuse after authentication.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BjfMkJyKzodxkobqGZ6Lqb
* test(datatables): run the roles tests on the enterprise edition, refusals without it
Each test that exercises roles runs with private and enterprise. Two tests run
without them: every roles route answers the Enterprise refusal, and a data
table saved under roles, or a named role, is refused a connection while one
not under roles resolves as before.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BjfMkJyKzodxkobqGZ6Lqb
* feat(datatables): gate the roles UI mount sites on an enterprise license
Both mount sites are still commented out; the gate travels with them.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BjfMkJyKzodxkobqGZ6Lqb
* test(datatables): run the tenant matcher test on the enterprise edition
The matcher it covers is enterprise code now, so without the enterprise
edition the test hit the stub and failed the default windmill-common run. It
runs with private and enterprise, and a counterpart without them asserts that
no tenant list covers anyone, the wildcard and a workspace admin included.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BjfMkJyKzodxkobqGZ6Lqb
* chore: update ee-repo-ref to a1873dbb67f2302b85ff5362f8387b48eccdb607
This commit updates the EE repository reference after PR #783 was merged in windmill-ee-private.
Previous ee-repo-ref: 5c853e2c20eca6b748415fc0d6862a6ebfb5fec4
New ee-repo-ref: a1873dbb67f2302b85ff5362f8387b48eccdb607
Automated by sync-ee-ref workflow.
* fix(datatables): refuse roles while a same-workspace alias reaches the database
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* fix(datatables): let CE migrations connect as an explicitly named admin
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* fix(datatables): serialize roles going on with aliases saved from other workspaces
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* docs(datatables): note that legacy names with ? cannot be migrated
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* fix(datatables): drop a DuckDB data table secret once its ATTACH has used it
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* perf(datatables): resolve a workspace's data tables per pointer hop, not per entry
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* fix(datatables): hold the parent's settings while a fork points at its data tables
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* chore: update ee-repo-ref to 7e338e4dabf91689bfd7fb0333c6534040b17b59
This commit updates the EE repository reference after PR #787 was merged in windmill-ee-private.
Previous ee-repo-ref: 38d6fcf2aeb39cfdac21814bbdbbcc02911e566a
New ee-repo-ref: 7e338e4dabf91689bfd7fb0333c6534040b17b59
Automated by sync-ee-ref workflow.
---------
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
Co-authored-by: windmill-internal-app[bot] <windmill-internal-app[bot]@users.noreply.github.com>
5559 lines
266 KiB
TypeScript
5559 lines
266 KiB
TypeScript
// Auto-generated by generate.py - DO NOT EDIT
|
|
|
|
export const SCRIPT_BASE = `# Windmill Script Writing Guide
|
|
|
|
## General Principles
|
|
|
|
- Scripts must export a main function (do not call it)
|
|
- Libraries are installed automatically - do not show installation instructions
|
|
- Credentials and configuration are stored in resources and passed as parameters
|
|
- The windmill client (\`wmill\`) provides APIs for interacting with the platform
|
|
|
|
## Function Naming
|
|
|
|
- Main function: \`main\` (or \`preprocessor\` for preprocessor scripts)
|
|
- Must be async for TypeScript variants
|
|
|
|
## Return Values
|
|
|
|
- Scripts can return any JSON-serializable value
|
|
- Return values become available to subsequent flow steps via \`results.step_id\`
|
|
|
|
## Preprocessor Scripts
|
|
|
|
Preprocessor scripts process raw trigger data from various sources (webhook, custom HTTP route, SQS, WebSocket, Kafka, NATS, MQTT, AMQP, Postgres, or email) before passing it to the flow. This separates the trigger logic from the flow logic and keeps the auto-generated UI clean.
|
|
|
|
The returned object determines the parameter values passed to the flow.
|
|
e.g., \`{ b: 1, a: 2 }\` calls the flow with \`a = 2\` and \`b = 1\`, assuming the flow has two inputs called \`a\` and \`b\`.
|
|
|
|
The preprocessor receives a single parameter called \`event\`.
|
|
`;
|
|
|
|
export const FLOW_BASE = `# Windmill Flow Building Guide
|
|
|
|
## OpenFlow Schema
|
|
|
|
The OpenFlow schema (openflow.openapi.yaml) is the source of truth for flow structure. Refer to OPENFLOW_SCHEMA for the complete type definitions.
|
|
|
|
## Reserved Module IDs
|
|
|
|
- \`failure\` - Reserved for failure handler module
|
|
- \`preprocessor\` - Reserved for preprocessor module
|
|
- \`Input\` - Reserved for flow input reference
|
|
|
|
## Hard Structural Rules
|
|
|
|
These are strict Windmill schema rules. Follow them exactly.
|
|
|
|
- \`value.modules\` is only for normal sequential steps
|
|
- \`value.preprocessor_module\` and \`value.failure_module\` are special top-level fields inside \`value\`, not entries in \`value.modules\`
|
|
- If a flow needs a preprocessor, create \`value.preprocessor_module\` with \`id: preprocessor\`
|
|
- If a flow needs a failure handler, create \`value.failure_module\` with \`id: failure\`
|
|
- Do NOT create regular modules inside \`value.modules\` named \`preprocessor\` or \`failure\`
|
|
- \`preprocessor_module\` and \`failure_module\` only support \`script\` or \`rawscript\`
|
|
- \`preprocessor_module\` runs before normal modules and cannot reference \`results.*\`
|
|
- \`failure_module\` can use the \`error\` object with \`error.message\`, \`error.step_id\`, \`error.name\`, and \`error.stack\`
|
|
|
|
Correct shape:
|
|
|
|
\`\`\`yaml
|
|
value:
|
|
preprocessor_module:
|
|
id: preprocessor
|
|
value:
|
|
type: rawscript
|
|
...
|
|
failure_module:
|
|
id: failure
|
|
value:
|
|
type: rawscript
|
|
...
|
|
modules:
|
|
- id: process_event
|
|
value:
|
|
type: rawscript
|
|
...
|
|
\`\`\`
|
|
|
|
Incorrect shape:
|
|
|
|
\`\`\`yaml
|
|
value:
|
|
modules:
|
|
- id: preprocessor
|
|
...
|
|
- id: process_event
|
|
...
|
|
- id: failure
|
|
...
|
|
\`\`\`
|
|
|
|
## Module ID Rules
|
|
|
|
- Must be unique across the entire flow
|
|
- Use underscores, not spaces (e.g., \`fetch_data\` not \`fetch data\`)
|
|
- Use descriptive names that reflect the step's purpose
|
|
|
|
## AI Agent Modules
|
|
|
|
An \`aiagent\` module runs an LLM that can call tools. Each entry of \`value.tools\` is a module-shaped
|
|
object with an extra \`value.tool_type\`: \`flowmodule\` for a script/flow tool, \`mcp\` for an MCP server
|
|
tool, \`websearch\` for web search.
|
|
|
|
\`\`\`json
|
|
{
|
|
"id": "support_agent",
|
|
"summary": "AI agent for customer support",
|
|
"value": {
|
|
"type": "aiagent",
|
|
"input_transforms": {
|
|
"provider": {
|
|
"type": "static",
|
|
"value": { "kind": "openai", "resource": "$res:f/ai_providers/openai", "model": "gpt-4o" }
|
|
},
|
|
"output_type": { "type": "static", "value": "text" },
|
|
"user_message": { "type": "javascript", "expr": "flow_input.query" },
|
|
"system_prompt": { "type": "static", "value": "You are a helpful assistant." }
|
|
},
|
|
"tools": [
|
|
{
|
|
"id": "search_docs",
|
|
"summary": "search_documentation",
|
|
"description": "Search the product documentation. Use it whenever the user asks how a feature works.",
|
|
"value": {
|
|
"tool_type": "flowmodule",
|
|
"type": "rawscript",
|
|
"language": "bun",
|
|
"content": "export async function main(query: string) { return ['doc1', 'doc2']; }",
|
|
"input_transforms": { "query": { "type": "static", "value": "" } }
|
|
}
|
|
}
|
|
]
|
|
}
|
|
}
|
|
\`\`\`
|
|
|
|
- \`provider\` is an object, not a bare resource string: \`{ "kind": <provider kind>,
|
|
"resource": "$res:<path>", "model": <model id> }\`. Required unless the module links to a saved
|
|
agent through \`value.agent\`. Static is right for a flow run from a form; a chat flow wires its
|
|
fields to flow inputs instead — see below
|
|
|
|
### Chat-Mode Flows
|
|
|
|
A flow with \`value.chat_input_enabled: true\` is run from a chat instead of a form: the composer
|
|
sends one message per turn and renders the conversation. It needs a required \`user_message\` string
|
|
input, read by the agent. Any other flow input the composer does not edit itself is asked for
|
|
under Configure inputs.
|
|
|
|
**A static \`provider\` gives a chat that cannot change its model.** Feed it from flow inputs
|
|
instead, either way round: one input carrying the whole object (\`"expr": "flow_input.model_config"\`)
|
|
makes every field editable, or wire it field by field to fix some and expose others. A field the
|
|
chat can write becomes a control in the composer — a provider picker, a model list, a thinking
|
|
control — and a field left static is fixed, with no control drawn for it. \`kind\` is the one
|
|
exception: the composer writes it only together with \`resource\`, since a provider is picked as a
|
|
pair, so a \`kind\` input wired on its own stays askable under Configure inputs and nothing the run
|
|
needs becomes unreachable.
|
|
|
|
\`\`\`json
|
|
{
|
|
"id": "chat_agent",
|
|
"value": {
|
|
"type": "aiagent",
|
|
"input_transforms": {
|
|
"provider": {
|
|
"type": "javascript",
|
|
"expr": "({ kind: 'anthropic', resource: '$res:f/ai/claude', model: flow_input.model, reasoning_effort: flow_input.thinking })"
|
|
},
|
|
"user_message": { "type": "javascript", "expr": "flow_input.user_message" },
|
|
"user_attachments": { "type": "javascript", "expr": "flow_input.files" },
|
|
"memory": { "type": "static", "value": { "kind": "window", "context_length": 10 } },
|
|
"streaming": { "type": "static", "value": true },
|
|
"output_type": { "type": "static", "value": "text" }
|
|
},
|
|
"tools": []
|
|
}
|
|
}
|
|
\`\`\`
|
|
|
|
- Wiring field by field means one object literal whose values are literals or bare \`flow_input.x\`
|
|
references. A spread, a call or a computed key leaves the composer unable to tell which input
|
|
feeds which field, so it offers no control at all — a bare \`flow_input.x\` for the whole object
|
|
is read instead as that one input carrying every field
|
|
- \`memory\` is what lets the agent see earlier turns; without it every message starts from nothing
|
|
- \`streaming\` on makes the answer and its thinking appear token by token instead of all at once
|
|
- \`user_attachments\` points at a flow input typed as an array of s3 objects
|
|
(\`{ "type": "array", "items": { "type": "object", "resourceType": "s3object" } }\`), so files
|
|
sent with a message reach the agent
|
|
- Running one needs a \`memory_id\` **query parameter** — not a flow argument — naming the
|
|
conversation the turn belongs to: a fresh UUID starts one, reusing a UUID continues it. The chat
|
|
supplies it itself; a run driven any other way has to pass it or the server refuses the job
|
|
|
|
### Tool Naming Rules
|
|
|
|
These rules cover \`flowmodule\` tools, the ones the agent calls by name. A \`websearch\` tool's
|
|
\`summary\` is a plain label (\`Web Search\`), and an \`mcp\` tool exposes the MCP server's own tool
|
|
names, so neither is name-checked at all — leave those summaries as they are.
|
|
|
|
- A flowmodule tool's \`summary\` is the **name the agent calls it by**, not a human label. Put the
|
|
human-readable explanation in \`description\`
|
|
- \`summary\` must match \`^[a-zA-Z0-9_]+$\`: letters, numbers and underscores only. No spaces, dashes,
|
|
dots or accents — \`search_documentation\`, never \`Search documentation\`
|
|
- Always set \`summary\`. It must be unique among that agent's tools, and must not be one of the
|
|
reserved ids (\`do\`, \`bg\`, \`ctx\`, \`state\`, \`if\`, \`else\`, \`for\`, \`delete\`, \`while\`, \`new\`, \`in\`,
|
|
\`failure\`, \`preprocessor\`, \`as\`, \`Input\`, \`Result\`, \`Trigger\`)
|
|
- A tool name outside that character set is rejected: flow write tools refuse it, and a flow that
|
|
reaches the worker with one fails every run with \`Invalid tool name\`
|
|
- Tool \`id\` follows the same rules as any module ID — unique across the flow, underscores not spaces
|
|
- \`description\` is optional free text telling the agent when and how to call the tool. Set it
|
|
whenever the name alone does not make that obvious; it overrides the description derived from the
|
|
underlying script
|
|
|
|
## Common Mistakes to Avoid
|
|
|
|
- Missing \`input_transforms\` - Rawscript parameters won't receive values without them
|
|
- Referencing future steps - \`results.step_id\` only works for steps that execute before the current one
|
|
- Duplicate module IDs - Each module ID must be unique in the flow
|
|
- AI agent flowmodule tool names with spaces - \`summary\` is the tool name and only accepts letters, numbers and underscores
|
|
|
|
## Data Flow Between Steps
|
|
|
|
- \`flow_input.property\` - Access flow input parameters
|
|
- \`results.step_id\` - Access output from a previous step only when that step result is in scope
|
|
- \`results.step_id.property\` - Access specific property from a previous step output only when that step result is in scope
|
|
- \`flow_input.iter.value\` - Current iteration value inside a \`forloopflow\`; in a \`whileloopflow\` it is just the iteration index (a plain number, same as \`flow_input.iter.index\`)
|
|
- \`flow_input.iter.index\` - Current loop index when inside a loop (\`forloopflow\` or \`whileloopflow\`)
|
|
|
|
## Loop Structure Rules
|
|
|
|
- For \`whileloopflow\`, break the loop with a module-level \`stop_after_if\`: on the loop module itself, or on an inner step (required when that step carries state via its own \`results\` — see below)
|
|
- \`stop_after_if\` is always a sibling of \`id\` and \`value\` on a flow module — never a direct key of the loop's \`value\` object
|
|
- \`stop_after_all_iters_if\` is for checks after the whole loop finishes, not the normal per-iteration break condition
|
|
- \`flow_input.iter.value\` in a \`whileloopflow\` is just the iteration index (same number as \`flow_input.iter.index\`) — it never carries state, so \`flow_input.iter.value.<field>\` is always undefined and a loop whose stop condition depends on it never terminates
|
|
- To carry state across iterations, a step reads its own previous-iteration result via \`results.<its_own_id>\` with a first-iteration fallback (e.g. \`results.b ?? flow_input.start\`) — but then the loop's \`stop_after_if\` MUST sit on that inner step, not on the loop module: a body that is exactly one plain step with the stop condition on the loop module runs on a fast path where \`results.<step_id>\` is null on every iteration and the loop never terminates (bodies with 2+ steps, or whose single step has its own \`stop_after_if\`, retry or similar, resolve \`results\` across iterations regardless of stop placement)
|
|
- For state that is just a counter, derive it from the index instead (e.g. \`flow_input.iter.index + 1\`) — that works in every configuration, including with \`stop_after_if\` on the loop module
|
|
- If the user asks for a final scalar/object after a loop, add a normal step after the loop that extracts the final value from the loop result instead of returning the whole loop result array
|
|
|
|
Correct \`whileloopflow\` shape:
|
|
|
|
\`\`\`yaml
|
|
- id: loop_until_done
|
|
stop_after_if:
|
|
expr: result.done === true
|
|
skip_if_stopped: false
|
|
value:
|
|
type: whileloopflow
|
|
skip_failures: false
|
|
modules:
|
|
- id: advance_state
|
|
value:
|
|
type: rawscript
|
|
input_transforms:
|
|
count:
|
|
type: javascript
|
|
expr: flow_input.iter.index + 1
|
|
- id: return_final_state
|
|
value:
|
|
type: rawscript
|
|
input_transforms:
|
|
final_state:
|
|
type: javascript
|
|
expr: results.loop_until_done[results.loop_until_done.length - 1]
|
|
\`\`\`
|
|
|
|
Correct \`whileloopflow\` shape carrying state via \`results\` (stop condition on the inner step):
|
|
|
|
\`\`\`yaml
|
|
- id: loop_until_done
|
|
value:
|
|
type: whileloopflow
|
|
skip_failures: false
|
|
modules:
|
|
- id: advance_state
|
|
stop_after_if:
|
|
expr: result.done === true
|
|
skip_if_stopped: false
|
|
value:
|
|
type: rawscript
|
|
input_transforms:
|
|
state:
|
|
type: javascript
|
|
expr: results.advance_state ?? flow_input.initial_state
|
|
\`\`\`
|
|
|
|
Incorrect \`whileloopflow\` patterns:
|
|
|
|
\`\`\`yaml
|
|
- id: loop_until_done
|
|
value:
|
|
type: whileloopflow
|
|
stop_after_if:
|
|
expr: result.done === true
|
|
\`\`\`
|
|
|
|
\`\`\`yaml
|
|
input_transforms:
|
|
state:
|
|
type: javascript
|
|
# iter.value is a number (the iteration index); there is no previous-iteration state
|
|
expr: flow_input.iter.value.count
|
|
\`\`\`
|
|
|
|
\`\`\`yaml
|
|
input_transforms:
|
|
final_state:
|
|
type: javascript
|
|
expr: results.loop_until_done
|
|
\`\`\`
|
|
|
|
## Approval / Suspend Structure
|
|
|
|
An approval step is a normal **script** step (\`type: rawscript\` or \`type: script\`) that is turned into an approval by adding a module-level \`suspend\`. Its script calls \`wmill.getResumeUrls(approver)\` to generate the secret resume/cancel URLs and returns them so they can be sent to the approver(s) (Slack, email, etc.) or approved from the run page.
|
|
|
|
- \`suspend\` belongs on the flow module object itself, as a sibling of \`id\` and \`value\`
|
|
- Never put \`suspend\` inside \`value\`
|
|
- Do NOT use \`type: identity\` for an approval step. An identity step suspends but never produces the resume URLs, so approvers have no link to act on — it is not a functional approval.
|
|
|
|
Correct shape:
|
|
|
|
\`\`\`yaml
|
|
- id: request_approval
|
|
suspend:
|
|
required_events: 1
|
|
resume_form:
|
|
schema:
|
|
type: object
|
|
properties:
|
|
comment:
|
|
type: string
|
|
required: [comment]
|
|
value:
|
|
type: rawscript
|
|
language: bun
|
|
input_transforms:
|
|
approver:
|
|
type: static
|
|
value: ''
|
|
content: |
|
|
import * as wmill from "windmill-client"
|
|
|
|
export async function main(approver?: string) {
|
|
const urls = await wmill.getResumeUrls(approver)
|
|
// send urls.resume / urls.cancel to the approver(s), e.g. via Slack or email
|
|
return urls
|
|
}
|
|
\`\`\`
|
|
|
|
Incorrect shape (suspend misplaced inside \`value\`):
|
|
|
|
\`\`\`yaml
|
|
- id: request_approval
|
|
value:
|
|
type: rawscript
|
|
suspend:
|
|
required_events: 1
|
|
\`\`\`
|
|
|
|
Incorrect shape (identity has no resume URLs — not a real approval):
|
|
|
|
\`\`\`yaml
|
|
- id: request_approval
|
|
suspend:
|
|
required_events: 1
|
|
value:
|
|
type: identity
|
|
\`\`\`
|
|
|
|
## Branch Result Scope Rules
|
|
|
|
- Inside a branch, you may reference earlier outer steps and earlier steps in the same branch
|
|
- Outside a \`branchone\`, do NOT reference ids of steps that only exist inside its branches or default branch. Use \`results.<branchone_module_id>\` instead
|
|
- Outside a \`branchall\`, do NOT reference ids of steps inside its branches. Use \`results.<branchall_module_id>\` instead
|
|
- If downstream steps need a stable shape after a branch, make each branch return the same fields
|
|
- When needed, add a normalization step immediately after the branch and consume \`results.<branch_module_id>\` there
|
|
|
|
Correct after \`branchone\`:
|
|
|
|
\`\`\`yaml
|
|
- id: route_order
|
|
value:
|
|
type: branchone
|
|
...
|
|
- id: send_confirmation
|
|
value:
|
|
input_transforms:
|
|
routed:
|
|
type: javascript
|
|
expr: results.route_order
|
|
\`\`\`
|
|
|
|
Incorrect after \`branchone\`:
|
|
|
|
\`\`\`yaml
|
|
expr: results.create_shipment
|
|
expr: results.create_backorder
|
|
\`\`\`
|
|
|
|
Correct after \`branchall\`:
|
|
|
|
\`\`\`yaml
|
|
- id: enrich_parallel
|
|
value:
|
|
type: branchall
|
|
parallel: true
|
|
...
|
|
- id: combine_data
|
|
value:
|
|
input_transforms:
|
|
enrichments:
|
|
type: javascript
|
|
expr: results.enrich_parallel
|
|
\`\`\`
|
|
|
|
## Input Transforms
|
|
|
|
Every rawscript module needs \`input_transforms\` to map function parameters to values:
|
|
|
|
Static transform (fixed value):
|
|
{"param_name": {"type": "static", "value": "fixed_string"}}
|
|
|
|
JavaScript transform (dynamic expression):
|
|
{"param_name": {"type": "javascript", "expr": "results.previous_step.data"}}
|
|
|
|
## Resource References
|
|
|
|
- For flow inputs: Use type \`"object"\` with format \`"resource-{type}"\` (e.g., \`"resource-postgresql"\`)
|
|
- For step inputs: Use static value \`"$res:path/to/resource"\`
|
|
|
|
## Final Structural Self-Check
|
|
|
|
Before finalizing a flow, verify:
|
|
|
|
- any preprocessor is in \`value.preprocessor_module\`
|
|
- any failure handler is in \`value.failure_module\`
|
|
- any approval step has module-level \`suspend\`
|
|
- no downstream step references inner branch step ids from outside the branch
|
|
- every AI agent flowmodule tool has a unique \`summary\` made only of letters, numbers and underscores
|
|
|
|
## S3 Object Operations
|
|
|
|
Windmill provides built-in support for S3-compatible storage operations.
|
|
|
|
To accept an S3 object as flow input:
|
|
|
|
\`\`\`json
|
|
{
|
|
"type": "object",
|
|
"properties": {
|
|
"file": {
|
|
"type": "object",
|
|
"format": "resource-s3_object",
|
|
"description": "File to process"
|
|
}
|
|
}
|
|
}
|
|
\`\`\`
|
|
|
|
## Using Resources in Flows
|
|
|
|
On Windmill, credentials and configuration are stored in resources. Resource types define the format of the resource.
|
|
|
|
### As Flow Input
|
|
|
|
In the flow schema, set the property type to \`"object"\` with format \`"resource-{type}"\`:
|
|
|
|
\`\`\`json
|
|
{
|
|
"type": "object",
|
|
"properties": {
|
|
"database": {
|
|
"type": "object",
|
|
"format": "resource-postgresql",
|
|
"description": "Database connection"
|
|
}
|
|
}
|
|
}
|
|
\`\`\`
|
|
|
|
### As Step Input (Static Reference)
|
|
|
|
Reference a specific resource using \`$res:\` prefix:
|
|
|
|
\`\`\`json
|
|
{
|
|
"database": {
|
|
"type": "static",
|
|
"value": "$res:f/folder/my_database"
|
|
}
|
|
}
|
|
\`\`\`
|
|
`;
|
|
|
|
export const RESOURCES_BASE = `# Windmill Resources
|
|
|
|
Resources store credentials and configuration for external services.
|
|
|
|
## File Format
|
|
|
|
Resource files use the pattern: \`{path}.resource.json\`
|
|
|
|
Example: \`f/databases/postgres_prod.resource.json\`
|
|
|
|
## Resource Structure
|
|
|
|
\`\`\`json
|
|
{
|
|
"value": {
|
|
"host": "db.example.com",
|
|
"port": 5432,
|
|
"user": "admin",
|
|
"password": "$var:g/all/db_password",
|
|
"dbname": "production"
|
|
},
|
|
"description": "Production PostgreSQL database",
|
|
"resource_type": "postgresql"
|
|
}
|
|
\`\`\`
|
|
|
|
## Required Fields
|
|
|
|
- \`value\` - Object containing the resource configuration
|
|
- \`resource_type\` - Name of the resource type (e.g., "postgresql", "slack")
|
|
|
|
## Variable References
|
|
|
|
Reference variables in resource values:
|
|
|
|
\`\`\`json
|
|
{
|
|
"value": {
|
|
"api_key": "$var:g/all/api_key",
|
|
"secret": "$var:u/admin/secret"
|
|
}
|
|
}
|
|
\`\`\`
|
|
|
|
**Reference formats:**
|
|
- \`$var:g/all/name\` - Global variable
|
|
- \`$var:u/username/name\` - User variable
|
|
- \`$var:f/folder/name\` - Folder variable
|
|
|
|
## Resource References
|
|
|
|
Reference other resources:
|
|
|
|
\`\`\`json
|
|
{
|
|
"value": {
|
|
"database": "$res:f/databases/postgres"
|
|
}
|
|
}
|
|
\`\`\`
|
|
|
|
## Passing a Resource or Variable as a Run Argument
|
|
|
|
A script or flow argument typed as a resource (schema \`format: resource-<type>\`) is passed as
|
|
the **bare string** \`$res:<path>\` — the whole argument value. Same for a variable, with
|
|
\`$var:<path>\`. This applies everywhere job arguments are supplied: \`wmill script run/preview\`,
|
|
\`wmill flow run/preview\`, the \`runScriptByPath\` / \`runFlowByPath\` API, a schedule's \`args\`, a
|
|
trigger's configured static args.
|
|
|
|
\`\`\`json
|
|
{
|
|
"db": "$res:f/databases/postgres_prod",
|
|
"api_token": "$var:g/all/api_token"
|
|
}
|
|
\`\`\`
|
|
|
|
The reference is resolved when the job runs, under the job's run-as identity — the caller for an
|
|
ordinary run, but the configured principal for a schedule, a trigger, or a runnable set to run on
|
|
behalf of someone else. The run fails if that identity cannot read the referenced resource or
|
|
variable.
|
|
|
|
**Never wrap it in an object.** The resolver only rewrites a JSON value that *is* a string
|
|
starting with \`$res:\` / \`$var:\`; keys are never inspected. These are all wrong and are passed
|
|
through to the script unchanged:
|
|
|
|
\`\`\`json
|
|
{ "db": { "$res": "f/databases/postgres_prod" } }
|
|
{ "db": { "resource": "f/databases/postgres_prod" } }
|
|
{ "db": "f/databases/postgres_prod" }
|
|
\`\`\`
|
|
|
|
The string may sit anywhere a string can — a top-level argument, a nested object field
|
|
(\`{ "gh_auth": { "token": "$var:g/all/gh_token" } }\`), or an array element (array elements are
|
|
walked only while nested at most two levels deep, and only for arrays of at most 1000 items).
|
|
The prefix must be on the string itself.
|
|
|
|
## Common Resource Types
|
|
|
|
### PostgreSQL
|
|
\`\`\`json
|
|
{
|
|
"resource_type": "postgresql",
|
|
"value": {
|
|
"host": "localhost",
|
|
"port": 5432,
|
|
"user": "postgres",
|
|
"password": "$var:g/all/pg_password",
|
|
"dbname": "windmill",
|
|
"sslmode": "prefer"
|
|
}
|
|
}
|
|
\`\`\`
|
|
|
|
### MySQL
|
|
\`\`\`json
|
|
{
|
|
"resource_type": "mysql",
|
|
"value": {
|
|
"host": "localhost",
|
|
"port": 3306,
|
|
"user": "root",
|
|
"password": "$var:g/all/mysql_password",
|
|
"database": "myapp"
|
|
}
|
|
}
|
|
\`\`\`
|
|
|
|
### Slack
|
|
\`\`\`json
|
|
{
|
|
"resource_type": "slack",
|
|
"value": {
|
|
"token": "$var:g/all/slack_token"
|
|
}
|
|
}
|
|
\`\`\`
|
|
|
|
### AWS S3
|
|
\`\`\`json
|
|
{
|
|
"resource_type": "s3",
|
|
"value": {
|
|
"bucket": "my-bucket",
|
|
"region": "us-east-1",
|
|
"accessKeyId": "$var:g/all/aws_access_key",
|
|
"secretAccessKey": "$var:g/all/aws_secret_key"
|
|
}
|
|
}
|
|
\`\`\`
|
|
|
|
### HTTP/API
|
|
\`\`\`json
|
|
{
|
|
"resource_type": "http",
|
|
"value": {
|
|
"baseUrl": "https://api.example.com",
|
|
"headers": {
|
|
"Authorization": "Bearer $var:g/all/api_token"
|
|
}
|
|
}
|
|
}
|
|
\`\`\`
|
|
|
|
### Kafka
|
|
\`\`\`json
|
|
{
|
|
"resource_type": "kafka",
|
|
"value": {
|
|
"brokers": "broker1:9092,broker2:9092",
|
|
"sasl_mechanism": "PLAIN",
|
|
"security_protocol": "SASL_SSL",
|
|
"username": "$var:g/all/kafka_user",
|
|
"password": "$var:g/all/kafka_password"
|
|
}
|
|
}
|
|
\`\`\`
|
|
|
|
### NATS
|
|
\`\`\`json
|
|
{
|
|
"resource_type": "nats",
|
|
"value": {
|
|
"servers": ["nats://localhost:4222"],
|
|
"user": "$var:g/all/nats_user",
|
|
"password": "$var:g/all/nats_password"
|
|
}
|
|
}
|
|
\`\`\`
|
|
|
|
### MQTT
|
|
\`\`\`json
|
|
{
|
|
"resource_type": "mqtt",
|
|
"value": {
|
|
"host": "mqtt.example.com",
|
|
"port": 8883,
|
|
"username": "$var:g/all/mqtt_user",
|
|
"password": "$var:g/all/mqtt_password",
|
|
"tls": true
|
|
}
|
|
}
|
|
\`\`\`
|
|
|
|
## Custom Resource Types
|
|
|
|
Create custom resource types with JSON Schema:
|
|
|
|
\`\`\`json
|
|
{
|
|
"name": "custom_api",
|
|
"schema": {
|
|
"type": "object",
|
|
"properties": {
|
|
"base_url": {"type": "string", "format": "uri"},
|
|
"api_key": {"type": "string"},
|
|
"timeout": {"type": "integer", "default": 30}
|
|
},
|
|
"required": ["base_url", "api_key"]
|
|
},
|
|
"description": "Custom API connection"
|
|
}
|
|
\`\`\`
|
|
|
|
Save as: \`custom_api.resource-type.json\`
|
|
|
|
## OAuth Resources
|
|
|
|
OAuth resources are managed through the Windmill UI and marked:
|
|
|
|
\`\`\`json
|
|
{
|
|
"is_oauth": true,
|
|
"account": 123
|
|
}
|
|
\`\`\`
|
|
|
|
OAuth tokens are automatically refreshed by Windmill.
|
|
|
|
## Using Resources in Scripts
|
|
|
|
### TypeScript (Bun/Deno)
|
|
\`\`\`typescript
|
|
export async function main(db: RT.Postgresql) {
|
|
// db contains the resource values
|
|
const { host, port, user, password, dbname } = db;
|
|
}
|
|
\`\`\`
|
|
|
|
### Python
|
|
\`\`\`python
|
|
class postgresql(TypedDict):
|
|
host: str
|
|
port: int
|
|
user: str
|
|
password: str
|
|
dbname: str
|
|
|
|
def main(db: postgresql):
|
|
# db contains the resource values
|
|
pass
|
|
\`\`\`
|
|
|
|
## CLI Commands
|
|
|
|
\`\`\`bash
|
|
# List resources
|
|
wmill resource list
|
|
|
|
# List resource types with schemas
|
|
wmill resource-type list --schema
|
|
|
|
# Get specific resource type schema
|
|
wmill resource-type get postgresql
|
|
|
|
# Deploy resources to the workspace — destructive to remote state, so only run when
|
|
# the user explicitly asks to deploy/publish/push. Depending on how the repo is wired,
|
|
# deploy via \`git push\` or \`wmill sync push\` (see the Deploying section in AGENTS.wmill.md).
|
|
wmill sync push
|
|
\`\`\`
|
|
`;
|
|
|
|
export const RAW_APP_BASE = `# Windmill Raw Apps
|
|
|
|
Raw apps let you build custom frontends with React, Svelte, or Vue that connect to Windmill backend runnables and datatables.
|
|
|
|
## App shape
|
|
|
|
A raw app has three logical parts:
|
|
|
|
- **Frontend** — bundled with esbuild from \`index.tsx\` as the entrypoint. Files include the entrypoint, components (\`App.tsx\`), styles, etc.
|
|
- **Backend runnables** — server-side scripts the frontend calls, each addressed by a unique key.
|
|
- **Data** — optional whitelisted datatables (managed PostgreSQL) that the backend runnables can query. The frontend never queries the database directly; backend runnables are the only bridge.
|
|
|
|
## Frontend
|
|
|
|
### Entrypoint
|
|
|
|
The entrypoint is \`index.tsx\` for React and \`index.ts\` for Svelte and Vue. It is both the bundling entrypoint (the bundler is esbuild) and the **mount** entrypoint: the preview executes the bundle against an empty \`<div id="root">\` and auto-renders nothing, so the entrypoint must mount a top-level \`App\` itself. Keep the UI in \`App.tsx\` / \`App.svelte\` / \`App.vue\` and keep the entrypoint as the mount shim.
|
|
|
|
React (\`index.tsx\`):
|
|
|
|
\`\`\`tsx
|
|
import React from 'react'
|
|
import { createRoot } from 'react-dom/client'
|
|
import App from './App'
|
|
|
|
createRoot(document.getElementById('root')!).render(<App />)
|
|
\`\`\`
|
|
|
|
Svelte (\`index.ts\`): \`mount(App, { target: document.getElementById('root')! })\`. Vue (\`index.ts\`): \`createApp(App).mount('#root')\`.
|
|
|
|
**Never replace the entrypoint with a bare component** (\`export default function App() { ... }\` and no mount call). A component that is defined but never mounted renders a blank screen with **no error thrown** — it never executes, so nothing reaches the console or the error overlay. If an app renders blank, check that the entrypoint still mounts \`App\` into \`#root\`.
|
|
|
|
**Always begin every React file (\`.tsx\`/\`.jsx\`) that uses JSX with \`import React from 'react'\`.** esbuild uses the classic JSX transform, so \`React\` must be in scope wherever JSX appears — a missing import compiles fine but throws \`React is not defined\` at runtime, leaving a blank screen.
|
|
|
|
### Generated bindings (\`wmill.d.ts\` / \`wmill.ts\`)
|
|
|
|
The frontend imports a generated module that mirrors the backend runnables. **Never write to it directly** — it gets regenerated whenever backend runnables change. Modifying it by hand will be overwritten.
|
|
|
|
### Calling backend runnables
|
|
|
|
Import the generated bindings and call the runnable like a function. \`./wmill\` is the **only** way the frontend reaches anything server-side — datatables, workspace items, external services. Never \`fetch\` the Windmill API from frontend code: the bundle holds no token and builds no API URL.
|
|
|
|
| Export | Resolves to | Use it for |
|
|
|---|---|---|
|
|
| \`backend.<key>(args)\` | the runnable's result | the default — run and wait |
|
|
| \`backendAsync.<key>(args)\` | the **job id** (a string) | long-running work you want to track |
|
|
| \`waitJob(jobId)\` | the job's **result** (rejects if the job failed) | awaiting a \`backendAsync\` job |
|
|
| \`getJob(jobId)\` | a \`Job\` (\`{ type, success, result, duration_ms, ... }\`) | polling status without blocking |
|
|
| \`streamJob(jobId, onUpdate?)\` | the final result, calling \`onUpdate\` per chunk | showing output as it is produced |
|
|
|
|
Run and wait — the common case:
|
|
|
|
\`\`\`tsx
|
|
import { backend } from './wmill';
|
|
|
|
const user = await backend.get_user({ user_id: '123' });
|
|
\`\`\`
|
|
|
|
Start a long job, then await it:
|
|
|
|
\`\`\`tsx
|
|
import { backendAsync, waitJob } from './wmill';
|
|
|
|
const jobId = await backendAsync.run_report({ month: '2026-08' }); // a string
|
|
const report = await waitJob(jobId); // the result itself
|
|
\`\`\`
|
|
|
|
Or poll it without blocking, to render progress:
|
|
|
|
\`\`\`tsx
|
|
import { getJob } from './wmill';
|
|
|
|
const job = await getJob(jobId);
|
|
if (job.type === 'CompletedJob') setReport(job.result);
|
|
\`\`\`
|
|
|
|
\`backendAsync\` resolves a job id and nothing else — guard on it before storing or polling. A poll loop started on an \`undefined\` id never completes and shows as a row stuck "running" forever:
|
|
|
|
\`\`\`tsx
|
|
const jobId = await backendAsync.run_report(args);
|
|
if (!jobId) throw new Error('run_report did not start a job');
|
|
\`\`\`
|
|
|
|
**Never hand-write a job-polling runnable.** A backend runnable that calls \`jobs/list\`, or that returns \`getResultMaybe(...)\` for the frontend to poll, reimplements \`backendAsync\` + \`waitJob\` / \`getJob\` / \`streamJob\` — and it is what leads to guessing at base URLs and tokens.
|
|
|
|
### Keeping data out of recorded demos
|
|
|
|
An app can be demoed by recording a session: every interaction becomes a step carrying a snapshot of the page, replayed publicly or on the Hub. Password inputs are masked automatically. Mark anything else that must not appear with \`data-wm-no-record\` — the whole marked subtree is dropped from every snapshot, along with its values and the step's own metadata:
|
|
|
|
\`\`\`tsx
|
|
<label data-wm-no-record>
|
|
Customer SSN <input value={ssn} onChange={onSsn} />
|
|
</label>
|
|
\`\`\`
|
|
|
|
Apply it to customer data, internal notes and anything else a viewer of the demo should not see. It costs nothing when the app is never recorded.
|
|
|
|
### Chat UIs over a flow in chat mode
|
|
|
|
A flow deployed with chat mode on is a chat backend (streaming answer, tool calls, memory, conversation history). Do not drive it through a runnable: add \`windmill-chat\` to \`package.json\` and use it directly, it detects the app's Windmill and credential.
|
|
|
|
\`\`\`tsx
|
|
import { useWindmillChat } from 'windmill-chat/react'
|
|
|
|
const chat = useWindmillChat({ flowPath: 'f/support/assistant' })
|
|
// chat.messages ({ role, content, pending, success, tool? }), chat.status, chat.sendMessage(text), chat.stop()
|
|
\`\`\`
|
|
|
|
\`windmill-chat/ai-sdk\` gives a \`ChatTransport\` for the Vercel AI SDK's \`useChat\`, \`windmill-chat/assistant-ui\` a runtime for assistant-ui. The flow must be deployed, not a draft. A sandboxed app needs \`jobs:run\` in its frontend SDK scopes, plus \`flow_conversations:write\` for the conversation sidebar; without them the chat keeps history in the browser.
|
|
|
|
## Backend runnables
|
|
|
|
Each runnable has a unique key (used to call it from the frontend) and one of four types:
|
|
|
|
| Type | What it is |
|
|
|---|---|
|
|
| \`inline\` | Custom code stored on the app itself. Most common for app-specific logic. |
|
|
| \`script\` | Reference to an existing workspace script by path. |
|
|
| \`flow\` | Reference to an existing workspace flow by path. |
|
|
| \`hubscript\` | Reference to a hub script by path. |
|
|
|
|
### Inline runnables
|
|
|
|
Inline runnables carry their own source code. For file-based raw apps, the runnable language is determined by the backend file extension. The script must expose a \`main\` function as its entrypoint.
|
|
|
|
**TypeScript example** (\`backend/get_user.ts\`):
|
|
|
|
\`\`\`typescript
|
|
import * as wmill from 'windmill-client';
|
|
|
|
export async function main(user_id: string) {
|
|
const sql = wmill.datatable();
|
|
const user = await sql\`SELECT * FROM users WHERE id = \${user_id}\`.fetchOne();
|
|
return user;
|
|
}
|
|
\`\`\`
|
|
|
|
**Python example** (\`backend/get_user.py\`):
|
|
|
|
\`\`\`python
|
|
import wmill
|
|
|
|
def main(user_id: str):
|
|
db = wmill.datatable()
|
|
user = db.query('SELECT * FROM users WHERE id = $1', user_id).fetch_one()
|
|
return user
|
|
\`\`\`
|
|
|
|
#### The \`wmill\` client is already authenticated
|
|
|
|
An inline runnable runs as an ordinary Windmill job. \`import * as wmill from 'windmill-client'\` (TypeScript) and \`import wmill\` (Python) are already pointed at this instance and this workspace — there is nothing to configure.
|
|
|
|
**Don't read \`WM_TOKEN\` or \`BASE_INTERNAL_URL\` and build an API URL to \`fetch\`.** The client's own \`setClient\` already reads exactly those, and it also sets the credentials mode a raw app needs (\`WM_RAW_APP\` suppresses credentials, because a sandboxed bundle calls the API from an opaque origin that can never pair with \`Access-Control-Allow-Origin: *\`). Rebuilding that by hand drops the parts you can't see. Use \`wmill.*\` for everything Windmill, and \`fetch\` only for third-party APIs.
|
|
|
|
Prefer the \`wmill\` functions that appear in the SDK reference; for an endpoint none of them covers, the generated service classes (\`JobService\`, \`ScriptService\`, ...) are importable from \`windmill-client\`. What is not available is a name you guessed at: \`getBaseUrl\` and \`getWorkspaceToken\` are inventions, not API.
|
|
|
|
### Path runnables (script / flow / hubscript)
|
|
|
|
When \`type\` is \`script\`, \`flow\`, or \`hubscript\`, the runnable just stores a \`path\` to an existing workspace or hub item — no inline code. The referenced item's input/output schema becomes the runnable's surface.
|
|
|
|
### Draft code vs deployed code
|
|
|
|
This decides whether an app works before anything is deployed:
|
|
|
|
- **Inline runnables run the app's current code.** The editor sends the runnable's source with each request, so an inline runnable works in the preview with nothing deployed.
|
|
- **Path runnables (\`script\` / \`flow\` / \`hubscript\`) run the DEPLOYED item at that path.** So do \`wmill.runFlow\`, \`wmill.runFlowAsync\` and \`wmill.runScriptByPath\` called from inside a runnable. A draft — including a draft you just created — does not exist for them.
|
|
|
|
So an app wired to a flow you just wrote does nothing until **that flow is deployed**. The app itself does NOT have to be deployed for this: the preview runs the app's draft, so the referenced flow is the only thing that has to exist deployed.
|
|
|
|
That makes the fix a one-item deploy, not a release. Offer to deploy exactly the referenced flow or script and leave the app a draft the user keeps testing in the preview — do not push the whole change set through the review-and-deploy page, and do not ask the user to deploy the app, unless they said they want to ship it.
|
|
|
|
Do NOT quietly reimplement the flow inside an inline runnable to dodge the deployment: that leaves the user with two copies of the same logic and an app that ignores the flow they asked for. Inline the logic only when the user actually wants it inline.
|
|
|
|
Prefer a **path runnable of type \`flow\`** over an inline runnable that calls \`wmill.runFlowAsync\`. The path runnable gives the frontend the flow's real input schema and works with \`backend\` / \`backendAsync\` / \`waitJob\` like any other runnable; a hand-written wrapper gives up all of that.
|
|
|
|
### Static inputs
|
|
|
|
\`staticInputs\` is an optional \`Record<string, any>\` for arguments not overridable from the frontend. Useful with path runnables to pre-fill some args while leaving the rest to the frontend caller.
|
|
|
|
## Data Tables
|
|
|
|
Data tables are PostgreSQL databases managed by Windmill. Backend runnables query them via the \`wmill\` client; the frontend never queries them directly.
|
|
|
|
### Critical rules
|
|
|
|
1. **Whitelisted tables only**: a runnable can only query tables listed in the app's \`data.tables\` config. Tables not in this list are not accessible.
|
|
2. **Add tables before using**: queries against unlisted tables fail at runtime. When you introduce a new table, register it in \`data.tables\` first.
|
|
3. **Use the configured datatable/schema**: the app's \`data\` config sets the default datatable and schema; reference them consistently across runnables.
|
|
|
|
### Querying in TypeScript (Bun/Deno)
|
|
|
|
\`\`\`typescript
|
|
import * as wmill from 'windmill-client';
|
|
|
|
export async function main(user_id: string) {
|
|
const sql = wmill.datatable(); // Or: wmill.datatable('other_datatable')
|
|
|
|
// Parameterized queries (safe from SQL injection)
|
|
const user = await sql\`SELECT * FROM users WHERE id = \${user_id}\`.fetchOne();
|
|
const users = await sql\`SELECT * FROM users WHERE active = \${true}\`.fetch();
|
|
|
|
// Insert/Update
|
|
await sql\`INSERT INTO users (name, email) VALUES (\${name}, \${email})\`.execute();
|
|
await sql\`UPDATE users SET name = \${newName} WHERE id = \${user_id}\`.execute();
|
|
|
|
return user;
|
|
}
|
|
\`\`\`
|
|
|
|
### Querying in Python
|
|
|
|
\`\`\`python
|
|
import wmill
|
|
|
|
def main(user_id: str):
|
|
db = wmill.datatable() # Or: wmill.datatable('other_datatable')
|
|
|
|
# Use $1, $2, etc. for parameters
|
|
user = db.query('SELECT * FROM users WHERE id = $1', user_id).fetch_one()
|
|
users = db.query('SELECT * FROM users WHERE active = $1', True).fetch()
|
|
|
|
# Insert/Update
|
|
db.query('INSERT INTO users (name, email) VALUES ($1, $2)', name, email).execute()
|
|
db.query('UPDATE users SET name = $1 WHERE id = $2', new_name, user_id).execute()
|
|
|
|
return user
|
|
\`\`\`
|
|
|
|
## Best Practices
|
|
|
|
1. **Check existing tables** before creating new ones — reuse beats schema growth.
|
|
2. **Use parameterized queries** — never concatenate user input into SQL.
|
|
3. **Terminate every datatable statement** — the tagged template and \`db.query(...)\` only build a statement. It runs when you call \`fetch\` / \`fetchOne\` / \`fetchOneScalar\` / \`execute\` (\`fetch\` / \`fetch_one\` / \`fetch_one_scalar\` / \`execute\` in Python). An INSERT or UPDATE without one writes nothing and raises nothing. Awaiting the statement itself is a no-op — it is not a promise.
|
|
4. **Keep runnables focused** — one function per runnable; small surface area.
|
|
5. **Use descriptive keys** — \`get_user\`, not \`a\`.
|
|
6. **Always whitelist tables** — adding a runnable that queries a new table requires the table to be in \`data.tables\` first.
|
|
7. **Mark sensitive UI with \`data-wm-no-record\`** — it is what keeps that data out of a recorded demo; passwords are handled for you.
|
|
8. **Reach for \`backendAsync\` + \`waitJob\`** for long work — never a hand-written job-polling runnable.
|
|
9. **Deploy what a path runnable points at** — a path runnable aimed at a draft fails at runtime; tell the user what needs deploying.
|
|
10. **Use \`windmill-chat\` for a chat over a chat-mode flow** — never a runnable that runs the flow and polls its stream.
|
|
`;
|
|
|
|
export const PIPELINE_BASE = `# Data pipeline authoring
|
|
|
|
A **data pipeline** is NOT a flow. A flow is one runnable that orchestrates steps internally. A data pipeline is a set of **independent scripts**, each deployed on its own, that form a DAG by reading and writing shared **storage assets** (DuckLake tables, data tables, S3 objects, volumes, resources) and by declaring execution **triggers**. The pipeline is visualized and edited at \`/pipeline/<folder>\`; every node is a normal workspace script that happens to carry pipeline annotations. When the user asks for a "data pipeline" (or to "ingest / transform / materialize" data across steps), build pipeline-annotated scripts — do NOT build a flow.
|
|
|
|
## Default to DuckDB + DuckLake
|
|
|
|
A pipeline node that produces a table should almost always be a **\`duckdb\`** node that materializes its output into a **DuckLake** table with \`-- materialize ducklake://<name>/<table>\` (in a DuckDB node the annotation uses SQL \`--\` comment syntax; write the body as a bare \`SELECT\` and let the runtime do the write). DuckLake is the default lakehouse store for pipelines and is the shape the pipeline editor is built around, so prefer it unless the work specifically calls for something else:
|
|
|
|
- \`postgresql\` / data tables — only for row-level, OLTP-style mutations against an existing Postgres data table (frequent single-row upserts/updates, transactional reads that an app queries live).
|
|
- \`bun\` / \`python3\` — only for non-tabular work that doesn't map to SQL: calling an external API, wrangling files, arbitrary glue. When such a node still produces tabular data for downstream steps, land it in DuckLake (write it with the wmill SDK / ducklake helpers) rather than inventing a parallel store.
|
|
|
|
Do not spread a pipeline across postgres, S3, and DuckLake when one DuckLake lake would do; a consistent DuckLake lakehouse is the goal.
|
|
|
|
## Storage prerequisites
|
|
|
|
A DuckLake pipeline only runs once the workspace has **object storage** (S3 / Azure Blob / GCS) **and a DuckLake catalog** configured — DuckLake tables and \`s3://\` assets can't be materialized or read without it. Check with the \`list_ducklakes\` tool before you build (it returns the configured DuckLake catalogs, or none). Drafting the annotated scripts does not require storage, but the pipeline can't ingest, materialize, or read its assets until it exists. So if \`list_ducklakes\` returns none (or the user hits "storage not configured" errors), say so and give the right next step **by role**:
|
|
|
|
- a workspace **admin** sets it up in Workspace settings → Object Storage (add an S3/Azure/GCS storage), then adds a DuckLake catalog on top of it;
|
|
- anyone **without admin rights** should ask a workspace admin to configure object storage + a DuckLake catalog.
|
|
|
|
Never hand back a DuckLake pipeline that cannot run without flagging the missing storage and pointing to who sets it up.
|
|
|
|
## What makes a script a pipeline node
|
|
|
|
A script joins the pipeline when its source begins with the \`pipeline\` annotation as a top-of-file comment, **written in the script's own comment syntax** — \`//\` for TS/JS (bun), \`--\` for SQL (DuckDB/Postgres), \`#\` for Python/Bash. So it's \`-- pipeline\` in a DuckDB node, \`# pipeline\` in a Python node, \`// pipeline\` in a bun node. Every annotation below uses that same prefix (the \`//\` shown is the TS form). All other wiring is expressed as annotation comments near the top of the file:
|
|
|
|
- \`// on <ref>\` — declares an execution-DAG **input** (what triggers/feeds this node). \`<ref>\` is either:
|
|
- an **asset URI** (the node runs when that asset is produced upstream): \`ducklake://main/orders\`, \`datatable://main/users\`, \`$res:f/folder/my_resource\`, \`volume://name/path\`, or an S3 object (see the S3 storage-form rule below).
|
|
- a **native trigger kind**: \`schedule\`, \`webhook\`, \`email\`, \`kafka\`, \`mqtt\`, \`amqp\`, \`nats\`, \`postgres\`, \`sqs\`, \`gcp\`, or \`data_upload\` (a user-uploaded S3 file). For these the actual trigger row (cron, topic, …) is created separately; the annotation only declares the binding. **\`data_upload\` is special**: there is no trigger row — the node instead declares an **\`S3Object\` input parameter** fed by the auto-generated upload picker; it never hard-codes a key. Any language can be the \`data_upload\` node:
|
|
- Python (has \`import wmill\`): \`def main(file: wmill.S3Object):\` then \`wmill.load_s3_file(file)\`; TS (has \`import * as wmill from "windmill-client"\`): \`export async function main(file: wmill.S3Object)\`. Qualify the type as \`wmill.S3Object\` (or add \`from wmill import S3Object\` / \`import { S3Object } from "windmill-client"\`) — a bare \`S3Object\` is undefined.
|
|
- **DuckDB** takes the s3object arg via a \`-- $<name> (s3object)\` declaration and reads it directly, so a single DuckDB node can ingest **and** materialize:
|
|
\`\`\`
|
|
-- pipeline
|
|
-- on data_upload
|
|
-- materialize ducklake://main/raw_uploads
|
|
-- $file (s3object)
|
|
SELECT * FROM read_csv($file)
|
|
\`\`\`
|
|
- **Outputs** are inferred from what the body writes — a \`CREATE TABLE\`, a \`wmill.writeS3File(...)\`, a DuckLake/datatable write. To declare a managed output explicitly, use \`// materialize <asset-uri>\`.
|
|
- Optional badges: \`// partitioned <daily|hourly|weekly|monthly|dynamic>\`, \`// freshness <duration>\` (e.g. \`1h\`), \`// tag <worker-tag>\`, \`// retry <count> [delay]\`, \`// data_test <kind> ...\` (managed DuckLake targets only — deploy rejects it beside a \`dbt://\` one).
|
|
|
|
## S3 object wiring (storage form matters)
|
|
|
|
An \`s3://\` URI's first slashes select the **storage**, not part of the key — get this wrong and the producer/consumer edge silently won't connect:
|
|
|
|
- \`s3:///<key>\` (**triple** slash, empty first segment) = the **default** workspace storage. A downstream node reading or triggering on that object uses \`s3:///<key>\` — e.g. DuckDB \`-- on s3:///orders/2024.parquet\` and \`read_parquet('s3:///orders/2024.parquet')\`.
|
|
- \`s3://<storage>/<key>\` (**double** slash, non-empty first segment) = a **named secondary** storage called \`<storage>\` — so \`s3://ingest/x\` means storage \`ingest\`, key \`x\`, NOT key \`ingest/x\`. Only use this when the object genuinely lives in a configured secondary storage; never invent a bucket/storage name for a default-storage object (it breaks the edge).
|
|
|
|
To make the producer side visible to lineage, a Python/TS node MUST pass the **\`S3Object\` form**, not a bare key string: Python \`wmill.write_s3_file(wmill.S3Object(s3="<key>"), data)\` (or the import-free dict \`{"s3": "<key>"}\`), TS \`wmill.writeS3File({ s3: "<key>" }, data)\`. That records the default-storage asset \`/<key>\`, which a downstream \`s3:///<key>\` reader connects to (same key both sides). A bare \`write_s3_file("<key>", ...)\` records **no** asset and produces **no** edge. Add \`storage="<name>"\` only for a named secondary storage.
|
|
|
|
The key must be a **string literal** — the graph parser is static and cannot follow a variable, f-string, or computed path, so \`write_s3_file(wmill.S3Object(s3=key_var), ...)\` records no edge. Inline the literal (\`s3="events/user_events.parquet"\`) on both the writing and reading node. The same rule applies to every asset URI in an annotation or SDK call (\`ducklake://\`, \`datatable://\`, \`s3://\`): write them literally, not via a variable.
|
|
|
|
## Materialize (the managed output)
|
|
|
|
> **A MANAGED \`// materialize\` is DuckDB-only**, and its target must be a DuckLake table (\`ducklake://<name>/<table>\`). Deploy **rejects** a \`ducklake://\` \`// materialize\` on any other language (\`python3\`, \`bun\`, \`postgresql\`). For a non-DuckDB node writing the lake, do **not** use \`// materialize\` — write the output via the SDK (\`wmill.writeS3File(...)\`, ducklake helpers, …) and let it be inferred. Use \`duckdb\` when a node should materialize a DuckLake table.
|
|
>
|
|
> The one target ANY language may declare (except a dbt script, whose writes come from its manifest) is a **warehouse relation**: \`// materialize manual dbt://<warehouse>/<schema>/<name>\`, where \`<warehouse>\` is a warehouse the workspace configures under Settings → dbt. \`manual\` is the only mode it has — nothing generates warehouse DDL, so the node issues its own write (a postgresql \`CREATE TABLE\` / \`INSERT\`, an SDK load, …) and the annotation records the outcome. Use it on an ingestion node whose output a dbt project reads as a \`source\`: the declared relation and the dbt model land on ONE graph node, and a downstream \`// on dbt://<warehouse>/<schema>/<name>\` fires when the ingestion node completes.
|
|
|
|
A managed \`// materialize ducklake://<name>/<table>\` tells the runtime to write the node's output table **for you**: write the body as a single \`SELECT\` and the runtime wraps it in the create/replace — do **not** also write your own \`CREATE TABLE\` / \`INSERT\`. (The opposite holds for the \`dbt://\` target above: there the node writes its own DDL and the strategies below do not apply.) Write strategy:
|
|
|
|
- no option → **replace** the whole table each run (full refresh; the only mode whose output columns may change);
|
|
- \`// materialize <uri> append\` → INSERT-append rows (incremental);
|
|
- \`// materialize <uri> key=<col>\` → merge/upsert on \`<col>\`.
|
|
|
|
\`// materialize manual <uri>\` opts **out** of managed writes — the script writes its own DDL and the annotation only records the output asset for lineage.
|
|
|
|
\`materialize\` pairs with partitioning for incremental pipelines: a \`// partitioned <daily|hourly|weekly|monthly|dynamic>\` node runs **once per partition** (append/merge into a fixed-schema table), and the \`{partition}\` token inside any asset URI is substituted with the current partition value at run time.
|
|
|
|
\`materialize\` is an output **declaration** on a node — not a command. There is no "materialize run".
|
|
|
|
## How to build one in chat
|
|
|
|
1. Put every node in the **same folder**: \`f/<folder>/<name>\`. The folder is the pipeline.
|
|
2. Author each node as a **script draft** with \`write_script\` (or \`edit_script\`). Default to \`duckdb\` materializing into DuckLake (see "Default to DuckDB + DuckLake" above); pick \`postgresql\`, \`bun\`, or \`python3\` only when that section says the work calls for it.
|
|
3. Start each body with \`// pipeline\`, then the \`// on\` input declarations, then the transform that writes the output.
|
|
4. **Chain nodes by asset URI**: read an upstream node's output asset, then \`// on <that-same-uri>\` in the downstream node so the edge forms. Reuse exact asset paths from existing nodes rather than inventing parallel ones.
|
|
5. Leave nodes as drafts unless the user asks to deploy. A pipeline only "runs" once its scripts are deployed and their triggers exist.
|
|
|
|
When the user already has the \`/pipeline/<folder>\` editor open, prefer the dedicated \`build_pipeline_node\` / \`edit_pipeline_node\` tools (they stage reviewable, canvas-highlighted proposals). Outside the editor, use the standard script-draft tools with the annotations above.
|
|
|
|
## Example (DuckDB → DuckLake, scheduled ingest + downstream transform)
|
|
|
|
Node \`f/sales/orders_ingest\` (runs on a schedule, materializes a DuckLake table):
|
|
|
|
\`\`\`sql
|
|
-- pipeline
|
|
-- on schedule
|
|
-- materialize ducklake://main/orders
|
|
SELECT * FROM read_csv('s3:///raw/orders/*.csv')
|
|
\`\`\`
|
|
|
|
Node \`f/sales/orders_daily\` (runs when \`orders\` is produced, writes a rollup):
|
|
|
|
\`\`\`sql
|
|
-- pipeline
|
|
-- on ducklake://main/orders
|
|
-- materialize ducklake://main/orders_daily
|
|
SELECT date_trunc('day', ts) AS day, count(*) AS n
|
|
FROM ducklake.main.orders GROUP BY 1
|
|
\`\`\`
|
|
`;
|
|
|
|
export const WORKFLOW_AS_CODE_BASE = `# Windmill Workflow-as-Code Writing Guide
|
|
|
|
## Scope
|
|
|
|
Use this guide when writing or modifying Windmill Workflow-as-Code (WAC) scripts.
|
|
WAC is authored as a Windmill script and deployed with the normal script workflow. It is not an OpenFlow YAML flow.
|
|
|
|
Supported WAC authoring targets:
|
|
- Bun TypeScript scripts that import from \`windmill-client\`
|
|
- Python 3 scripts that import from \`wmill\`
|
|
|
|
## File Shape
|
|
|
|
Bun TypeScript:
|
|
|
|
\`\`\`typescript
|
|
import {
|
|
task,
|
|
taskScript,
|
|
taskFlow,
|
|
step,
|
|
sleep,
|
|
waitForApproval,
|
|
getApprovalUrls,
|
|
parallel,
|
|
workflow,
|
|
} from "windmill-client";
|
|
|
|
const process = task(async (x: string): Promise<string> => {
|
|
return \`processed: \${x}\`;
|
|
});
|
|
|
|
export const main = workflow(async (x: string) => {
|
|
const result = await process(x);
|
|
return { result };
|
|
});
|
|
\`\`\`
|
|
|
|
Python:
|
|
|
|
\`\`\`python
|
|
from wmill import task, task_script, task_flow, step, sleep, wait_for_approval, get_approval_urls, parallel, workflow
|
|
|
|
@task()
|
|
async def process(x: str) -> str:
|
|
return f"processed: {x}"
|
|
|
|
@workflow
|
|
async def main(x: str):
|
|
result = await process(x)
|
|
return {"result": result}
|
|
\`\`\`
|
|
|
|
Rules:
|
|
- Do not call \`main\`.
|
|
- Bun TypeScript should export the workflow entrypoint, preferably \`export const main = workflow(async (...) => { ... })\`.
|
|
- Python must use \`@workflow\` on an async top-level function, usually \`main\`.
|
|
- Define task functions and \`taskScript\`/\`task_script\` or \`taskFlow\`/\`task_flow\` assignments at module top level with stable names.
|
|
- Use the exact SDK names. Do not alias \`workflow\`, \`task\`, \`taskScript\`, \`taskFlow\`, \`step\`, \`sleep\`, \`waitForApproval\`, \`task_script\`, \`task_flow\`, or \`wait_for_approval\`; the WAC parser recognizes these names directly.
|
|
|
|
## Checkpoint And Replay Model
|
|
|
|
The parent workflow may rerun from the top after any suspension, retry, approval, or child task completion. Completed durable steps are replayed from the checkpoint.
|
|
|
|
Put every side effect or non-deterministic value behind a durable WAC boundary:
|
|
- Use \`task()\` / \`@task()\` for substantial work that should run as its own child job.
|
|
- Use \`taskScript()\` / \`task_script()\` for an existing script or a relative module file.
|
|
- Use \`taskFlow()\` / \`task_flow()\` for an existing Windmill flow.
|
|
- Use \`step(name, fn)\` for lightweight inline work whose result must be checkpointed.
|
|
- Use \`sleep(seconds)\` for server-side sleeps that do not hold a worker.
|
|
- Use \`waitForApproval()\` / \`wait_for_approval()\` for external approval suspension.
|
|
|
|
Never put API calls, database writes, notifications, random values, timestamps, or irreversible changes directly in the top-level workflow body. The workflow body can be rerun. Put those operations in a task or in \`step()\`.
|
|
|
|
Branching on task or step results is safe because those results are checkpointed. Branching on current time, random data, environment reads, or external state is unsafe unless the value is first captured with \`step()\`.
|
|
|
|
## Tasks
|
|
|
|
Use \`task()\` / \`@task()\` for inline functions that become workflow steps:
|
|
|
|
\`\`\`typescript
|
|
const enrich = task(async (customerId: string) => {
|
|
return await fetchCustomer(customerId);
|
|
});
|
|
\`\`\`
|
|
|
|
\`\`\`python
|
|
@task(timeout=600, tag="etl")
|
|
async def enrich(customer_id: str):
|
|
return await fetch_customer(customer_id)
|
|
\`\`\`
|
|
|
|
In TypeScript, prefer assigning each task to a named top-level const. In Python, prefer top-level async functions decorated with \`@task()\` or \`@task\`.
|
|
|
|
For existing scripts:
|
|
|
|
\`\`\`typescript
|
|
const helper = taskScript("./helper.ts");
|
|
const existing = taskScript("f/data/extract", { timeout: 600 });
|
|
const value = await helper({ input: x });
|
|
\`\`\`
|
|
|
|
\`\`\`python
|
|
helper = task_script("./helper.py")
|
|
existing = task_script("f/data/extract", timeout=600)
|
|
value = await helper(input=x)
|
|
\`\`\`
|
|
|
|
For existing flows:
|
|
|
|
\`\`\`typescript
|
|
const pipeline = taskFlow("f/etl/pipeline");
|
|
const output = await pipeline({ input: data });
|
|
\`\`\`
|
|
|
|
\`\`\`python
|
|
pipeline = task_flow("f/etl/pipeline")
|
|
output = await pipeline(input=data)
|
|
\`\`\`
|
|
|
|
## Inline Steps
|
|
|
|
Use \`step()\` for lightweight inline values that must not change during replay:
|
|
|
|
\`\`\`typescript
|
|
const startedAt = await step("started_at", () => new Date().toISOString());
|
|
\`\`\`
|
|
|
|
\`\`\`python
|
|
from datetime import datetime
|
|
|
|
started_at = await step("started_at", lambda: datetime.now().isoformat())
|
|
\`\`\`
|
|
|
|
Use stable, descriptive step names. Do not generate step names dynamically.
|
|
|
|
## Parallelism
|
|
|
|
To run independent work in parallel, start task promises/coroutines before awaiting them together:
|
|
|
|
\`\`\`typescript
|
|
const [a, b] = await Promise.all([process("a"), process("b")]);
|
|
const many = await parallel(items, process, { concurrency: 5 });
|
|
\`\`\`
|
|
|
|
\`\`\`python
|
|
import asyncio
|
|
|
|
a, b = await asyncio.gather(process("a"), process("b"))
|
|
many = await parallel(items, process, concurrency=5)
|
|
\`\`\`
|
|
|
|
Only parallelize independent steps. Do not read the result of a task before it is awaited.
|
|
|
|
## Approvals
|
|
|
|
Name the approval step and generate its URLs inside \`step()\` before sending them.
|
|
\`getApprovalUrls\` / \`get_approval_urls\` returns the URLs bound to that step, the same
|
|
ones its built-in approve/reject buttons use:
|
|
|
|
\`\`\`typescript
|
|
const urls = await step("urls", () => getApprovalUrls("manager"));
|
|
await step("notify", () => sendApprovalEmail(urls.resume, urls.cancel));
|
|
const approval = await waitForApproval({ key: "manager", timeout: 3600 });
|
|
\`\`\`
|
|
|
|
\`\`\`python
|
|
urls = await step("urls", lambda: get_approval_urls("manager"))
|
|
await step("notify", lambda: send_approval_email(urls["resume"], urls["cancel"]))
|
|
approval = await wait_for_approval(key="manager", timeout=3600)
|
|
\`\`\`
|
|
|
|
With several approvals in one workflow, give each its own key so each notification
|
|
resumes its own step. Keys must be unique — reusing one raises an error rather than
|
|
silently renaming the step. A minted URL only resumes while its own step is awaiting
|
|
approval; used at any other moment it is rejected rather than resuming the wrong one. \`getResumeUrls()\` / \`get_resume_urls()\` still works but signs a
|
|
random nonce, so its URLs are not tied to any particular approval step.
|
|
|
|
\`selfApproval: false\` and \`self_approval=False\` are Enterprise-only approval behavior. Do not use them unless the user asks for that behavior.
|
|
|
|
## Error Handling
|
|
|
|
Let task errors fail the workflow unless the user asks for recovery logic.
|
|
|
|
Python: \`except Exception\` is safe around WAC calls because internal suspension inherits from \`BaseException\`. Avoid bare \`except:\` in workflow code. If the user asks for recovery logic around failed child work, catch \`TaskError\` from \`wmill\` for task failures.
|
|
|
|
TypeScript: avoid broad \`try/catch\` around WAC SDK calls. The SDK uses an internal suspension error during initial dispatch; catching it can break workflow suspension. If a broad catch is unavoidable, rethrow internal suspension errors before handling business errors.
|
|
|
|
A caught failure reads the same whether it came from a task or from a \`step()\`, and the same in the round that ran the failing body as in every round replaying it. It carries \`step_key\`, \`child_job_id\` (absent for a \`step()\`, which runs in the workflow job and has no child job), a \`message\` that is the failure's own message, and \`result\` = \`{"error": {"name", "message", "stack"?, "extra"?}}\`. \`name\`, \`message\` and \`stack\` are the fields that read the same whichever side failed; \`name\` and \`message\` are always there, \`stack\` only when the failure had a traceback to give. \`extra\` carries the failure's own custom fields (an exception's attributes, an error's properties) and is best-effort: it is absent when there were none, and a task can report entries a step does not, so read it defensively and don't branch on its absence. \`extra\` is dropped when it is too large to keep in the checkpoint, and \`extra_omitted: true\` says so — absent \`extra\` with no \`extra_omitted\` means the failure simply had no custom fields. Branch on those, not on the original exception type: the workflow body re-runs from the top every round and a replay rebuilds the failure from the checkpoint, so nothing outside that record survives. Python raises \`TaskError\`; TypeScript throws an \`Error\` named \`TaskError\` carrying the same fields. Nothing is chained onto \`__cause__\` / \`cause\` — the traceback is in \`result.error.stack\`, and is also printed to the job log when the step fails.
|
|
`;
|
|
|
|
export const FLOW_CHAT_SPECIAL_MODULES = `## Special Modules
|
|
|
|
- Use \`set_preprocessor_module\` to add, replace, or remove the top-level \`value.preprocessor_module\`
|
|
- Use \`set_failure_module\` to add, replace, or remove the top-level \`value.failure_module\`
|
|
- Use \`set_flow_json\` only when you are replacing the whole flow, including normal modules and optional special modules
|
|
|
|
**Example - Update only the special modules:**
|
|
\`\`\`javascript
|
|
set_preprocessor_module({
|
|
module: JSON.stringify({
|
|
id: "preprocessor",
|
|
value: {
|
|
type: "rawscript",
|
|
language: "bun",
|
|
content: "export async function preprocessor(payload: string) { const trimmed = payload.trim(); if (!trimmed) { throw new Error('payload must not be empty'); } return { payload: trimmed }; }",
|
|
input_transforms: {
|
|
payload: { type: "javascript", expr: "flow_input.payload" }
|
|
}
|
|
}
|
|
})
|
|
})
|
|
|
|
set_failure_module({
|
|
module: JSON.stringify({
|
|
id: "failure",
|
|
value: {
|
|
type: "rawscript",
|
|
language: "bun",
|
|
content: "export async function main(message: string, name: string, step_id: string) { return { message, name, step_id }; }",
|
|
input_transforms: {
|
|
message: { type: "javascript", expr: "error.message" },
|
|
name: { type: "javascript", expr: "error.name" },
|
|
step_id: { type: "javascript", expr: "error.step_id" }
|
|
}
|
|
}
|
|
})
|
|
})
|
|
\`\`\`
|
|
`;
|
|
|
|
export const SDK_TYPESCRIPT = `# TypeScript SDK (windmill-client)
|
|
|
|
Import: import * as wmill from 'windmill-client'
|
|
|
|
The client configures itself from the job's environment — base URL, token and credentials mode
|
|
are all set before your code runs, so there is nothing to initialize and no reason to read
|
|
WM_TOKEN or BASE_INTERNAL_URL and build an API URL yourself. Reconstructing that by hand only
|
|
reintroduces details the client already handles. Call the SDK for anything Windmill, and use raw
|
|
HTTP for third-party APIs.
|
|
|
|
The helpers below are the surface to prefer. For an endpoint none of them covers, import the
|
|
generated service classes (JobService, ScriptService, ...) from 'windmill-client' — they are not
|
|
listed here but they do exist. What does not exist is a helper name you guessed at: if it is
|
|
neither listed below nor a service method, do not call it.
|
|
|
|
To know who is running the script, read the contextual variables rather than calling the API:
|
|
\`process.env.WM_END_USER_EMAIL || process.env.WM_EMAIL\`. WM_END_USER_EMAIL is the app viewer when
|
|
the run was triggered from an app and empty otherwise (both variables are always defined), WM_EMAIL
|
|
is the user the job is permissioned as. WM_USERNAME is the matching username.
|
|
|
|
workerHasInternalServer(): boolean
|
|
|
|
/**
|
|
* Initialize the Windmill client with authentication token and base URL
|
|
* @param token - Authentication token (defaults to WM_TOKEN env variable)
|
|
* @param baseUrl - API base URL (defaults to BASE_INTERNAL_URL or BASE_URL env variable)
|
|
*/
|
|
setClient(token?: string, baseUrl?: string): void
|
|
|
|
/**
|
|
* Create a client configuration from env variables
|
|
* @returns client configuration
|
|
*/
|
|
getWorkspace(): string
|
|
|
|
/**
|
|
* Get a resource value by path
|
|
* @param path path of the resource, default to internal state path
|
|
* @param undefinedIfEmpty if the resource does not exist, return undefined instead of throwing an error
|
|
* @returns resource value
|
|
*/
|
|
async getResource(path?: string, undefinedIfEmpty?: boolean): Promise<any>
|
|
|
|
/**
|
|
* Get the true root job id
|
|
* @param jobId job id to get the root job id from (default to current job)
|
|
* @returns root job id
|
|
*/
|
|
async getRootJobId(jobId?: string): Promise<string>
|
|
|
|
/**
|
|
* Run a script synchronously by its path and wait for the result
|
|
* @param path - Script path in Windmill
|
|
* @param args - Arguments to pass to the script
|
|
* @param verbose - Enable verbose logging
|
|
* @param tag - Override the worker tag the job runs on
|
|
* @returns Script execution result
|
|
*/
|
|
async runScriptByPath(path: string, args: Record<string, any> | null = null, verbose: boolean = false, tag: string | null = null): Promise<any>
|
|
|
|
/**
|
|
* Run a script synchronously by its hash and wait for the result
|
|
* @param hash_ - Script hash in Windmill
|
|
* @param args - Arguments to pass to the script
|
|
* @param verbose - Enable verbose logging
|
|
* @param tag - Override the worker tag the job runs on
|
|
* @returns Script execution result
|
|
*/
|
|
async runScriptByHash(hash_: string, args: Record<string, any> | null = null, verbose: boolean = false, tag: string | null = null): Promise<any>
|
|
|
|
/**
|
|
* Append a text to the result stream
|
|
* @param text text to append to the result stream
|
|
*/
|
|
appendToResultStream(text: string): void
|
|
|
|
/**
|
|
* Stream to the result stream
|
|
* @param stream stream to stream to the result stream
|
|
*/
|
|
async streamResult(stream: AsyncIterable<string>): Promise<void>
|
|
|
|
/**
|
|
* Run a flow synchronously by its path and wait for the result
|
|
* @param path - Flow path in Windmill
|
|
* @param args - Arguments to pass to the flow
|
|
* @param verbose - Enable verbose logging
|
|
* @param tag - Override the worker tag the job runs on
|
|
* @returns Flow execution result
|
|
*/
|
|
async runFlow(path: string | null = null, args: Record<string, any> | null = null, verbose: boolean = false, tag: string | null = null): Promise<any>
|
|
|
|
/**
|
|
* Wait for a job to complete and return its result
|
|
* @param jobId - ID of the job to wait for
|
|
* @param verbose - Enable verbose logging
|
|
* @returns Job result when completed
|
|
*/
|
|
async waitJob(jobId: string, verbose: boolean = false): Promise<any>
|
|
|
|
/**
|
|
* Get the result of a completed job
|
|
* @param jobId - ID of the completed job
|
|
* @returns Job result
|
|
*/
|
|
async getResult(jobId: string): Promise<any>
|
|
|
|
/**
|
|
* Get the result of a job if completed, or its current status
|
|
* @param jobId - ID of the job
|
|
* @returns Object with started, completed, success, and result properties
|
|
*/
|
|
async getResultMaybe(jobId: string): Promise<any>
|
|
|
|
/**
|
|
* Cancel a queued or running job by ID.
|
|
* @param jobId - UUID of the job to cancel
|
|
* @param reason - Optional reason for cancellation
|
|
* @returns Response message from the cancel endpoint
|
|
*/
|
|
async cancelJob(jobId: string, reason: string | undefined = undefined): Promise<string>
|
|
|
|
/**
|
|
* Run a script asynchronously by its path
|
|
* @param path - Script path in Windmill
|
|
* @param args - Arguments to pass to the script
|
|
* @param scheduledInSeconds - Schedule execution for a future time (in seconds)
|
|
* @param tag - Override the worker tag the job runs on
|
|
* @returns Job ID of the created job
|
|
*/
|
|
async runScriptByPathAsync(path: string, args: Record<string, any> | null = null, scheduledInSeconds: number | null = null, tag: string | null = null): Promise<string>
|
|
|
|
/**
|
|
* Run a script asynchronously by its hash
|
|
* @param hash_ - Script hash in Windmill
|
|
* @param args - Arguments to pass to the script
|
|
* @param scheduledInSeconds - Schedule execution for a future time (in seconds)
|
|
* @param tag - Override the worker tag the job runs on
|
|
* @returns Job ID of the created job
|
|
*/
|
|
async runScriptByHashAsync(hash_: string, args: Record<string, any> | null = null, scheduledInSeconds: number | null = null, tag: string | null = null): Promise<string>
|
|
|
|
/**
|
|
* Run a flow asynchronously by its path
|
|
* @param path - Flow path in Windmill
|
|
* @param args - Arguments to pass to the flow
|
|
* @param scheduledInSeconds - Schedule execution for a future time (in seconds)
|
|
* @param doNotTrackInParent - If false, tracks state in parent job (only use when fully awaiting the job)
|
|
* @param tag - Override the worker tag the job runs on
|
|
* @returns Job ID of the created job
|
|
*/
|
|
async runFlowAsync(path: string | null, args: Record<string, any> | null, scheduledInSeconds: number | null = null, // can only be set to false if this the job will be fully await and not concurrent with any other job // as otherwise the child flow and its own child will store their state in the parent job which will // lead to incorrectness and failures doNotTrackInParent: boolean = true, tag: string | null = null): Promise<string>
|
|
|
|
/**
|
|
* Resolve a resource value in case the default value was picked because the input payload was undefined
|
|
* @param obj resource value or path of the resource under the format \`$res:path\`
|
|
* @returns resource value
|
|
*/
|
|
async resolveDefaultResource(obj: any): Promise<any>
|
|
|
|
/**
|
|
* Get the state file path from environment variables
|
|
* @returns State path string
|
|
*/
|
|
getStatePath(): string
|
|
|
|
/**
|
|
* Set a resource value by path
|
|
* @param path path of the resource to set, default to state path
|
|
* @param value new value of the resource to set
|
|
* @param initializeToTypeIfNotExist if the resource does not exist, initialize it with this type
|
|
*/
|
|
async setResource(value: any, path?: string, initializeToTypeIfNotExist?: string): Promise<void>
|
|
|
|
/**
|
|
* Set the state
|
|
* @param state state to set
|
|
* @param path Optional state resource path override. Defaults to \`getStatePath()\`.
|
|
*/
|
|
async setState(state: any, path?: string): Promise<void>
|
|
|
|
/**
|
|
* Set the progress
|
|
* Progress cannot go back and limited to 0% to 99% range
|
|
* @param percent Progress to set in %
|
|
* @param jobId? Job to set progress for
|
|
*/
|
|
async setProgress(percent: number, jobId?: any): Promise<void>
|
|
|
|
/**
|
|
* Get the progress
|
|
* @param jobId? Job to get progress from
|
|
* @returns Optional clamped between 0 and 100 progress value
|
|
*/
|
|
async getProgress(jobId?: any): Promise<number | null>
|
|
|
|
/**
|
|
* Set a flow user state
|
|
* @param key key of the state
|
|
* @param value value of the state
|
|
*/
|
|
async setFlowUserState(key: string, value: any, errorIfNotPossible?: boolean): Promise<void>
|
|
|
|
/**
|
|
* Get a flow user state
|
|
* @param path path of the variable
|
|
*/
|
|
async getFlowUserState(key: string, errorIfNotPossible?: boolean): Promise<any>
|
|
|
|
/**
|
|
* Get the state shared across executions
|
|
* @param path Optional state resource path override. Defaults to \`getStatePath()\`.
|
|
*/
|
|
async getState(path?: string): Promise<any>
|
|
|
|
/**
|
|
* Get a variable by path
|
|
* @param path path of the variable
|
|
* @returns variable value
|
|
*/
|
|
async getVariable(path: string): Promise<string>
|
|
|
|
/**
|
|
* Set a variable by path, create if not exist
|
|
* @param path path of the variable
|
|
* @param value value of the variable
|
|
* @param isSecretIfNotExist if the variable does not exist, create it as secret or not (default: false)
|
|
* @param descriptionIfNotExist if the variable does not exist, create it with this description (default: "")
|
|
*/
|
|
async setVariable(path: string, value: string, isSecretIfNotExist?: boolean, descriptionIfNotExist?: string): Promise<void>
|
|
|
|
/**
|
|
* Build a PostgreSQL connection URL from a database resource
|
|
* @param path - Path to the database resource
|
|
* @returns PostgreSQL connection URL string
|
|
*/
|
|
async databaseUrlFromResource(path: string): Promise<string>
|
|
|
|
async polarsConnectionSettings(s3_resource_path: string | undefined): Promise<any>
|
|
|
|
async duckdbConnectionSettings(s3_resource_path: string | undefined): Promise<any>
|
|
|
|
/**
|
|
* Get S3 client settings from a resource or workspace default
|
|
* @param s3_resource_path - Path to S3 resource (uses workspace default if undefined)
|
|
* @param workspace - Workspace to read from (defaults to the \`WM_WORKSPACE\` env var)
|
|
* @returns S3 client configuration settings
|
|
*/
|
|
async denoS3LightClientSettings(s3_resource_path: string | undefined, workspace: string | undefined = undefined): Promise<DenoS3LightClientSettings>
|
|
|
|
/**
|
|
* Load the content of a file stored in S3. If the s3ResourcePath is undefined, it will default to the workspace S3 resource.
|
|
*
|
|
* \`\`\`typescript
|
|
* let fileContent = await wmill.loadS3FileContent(inputFile)
|
|
* // if the file is a raw text file, it can be decoded and printed directly:
|
|
* const text = new TextDecoder().decode(fileContentStream)
|
|
* console.log(text);
|
|
* \`\`\`
|
|
*
|
|
* @param workspace - Workspace to read from (defaults to the \`WM_WORKSPACE\` env var)
|
|
*/
|
|
async loadS3File(s3object: S3Object, s3ResourcePath: string | undefined = undefined, workspace: string | undefined = undefined): Promise<Uint8Array | undefined>
|
|
|
|
/**
|
|
* Load the content of a file stored in S3 as a stream. If the s3ResourcePath is undefined, it will default to the workspace S3 resource.
|
|
*
|
|
* \`\`\`typescript
|
|
* let fileContentBlob = await wmill.loadS3FileStream(inputFile)
|
|
* // if the content is plain text, the blob can be read directly:
|
|
* console.log(await fileContentBlob.text());
|
|
* \`\`\`
|
|
*
|
|
* @param workspace - Workspace to read from (defaults to the \`WM_WORKSPACE\` env var)
|
|
*/
|
|
async loadS3FileStream(s3object: S3Object, s3ResourcePath: string | undefined = undefined, workspace: string | undefined = undefined): Promise<Blob | undefined>
|
|
|
|
/**
|
|
* Persist a file to the S3 bucket. If the s3ResourcePath is undefined, it will default to the workspace S3 resource.
|
|
*
|
|
* \`\`\`typescript
|
|
* const s3object = await writeS3File(s3Object, "Hello Windmill!")
|
|
* const fileContentAsUtf8Str = (await s3object.toArray()).toString('utf-8')
|
|
* console.log(fileContentAsUtf8Str)
|
|
* \`\`\`
|
|
*
|
|
* @param workspace - Workspace to write to (defaults to the \`WM_WORKSPACE\` env var)
|
|
*/
|
|
async writeS3File(s3object: S3Object | undefined, fileContent: string | Blob, s3ResourcePath: string | undefined = undefined, contentType: string | undefined = undefined, contentDisposition: string | undefined = undefined, workspace: string | undefined = undefined): Promise<S3Object>
|
|
|
|
/**
|
|
* Permanently delete a file from S3 by key.
|
|
*
|
|
* \`\`\`typescript
|
|
* await wmill.deleteS3File({ s3: "path/to/file.txt" })
|
|
* \`\`\`
|
|
*
|
|
* @param s3object - S3 object identifying the file to delete (must have \`s3\` set)
|
|
* @param workspace - Workspace to delete from (defaults to the \`WM_WORKSPACE\` env var)
|
|
*/
|
|
async deleteS3File(s3object: S3Object, workspace: string | undefined = undefined): Promise<void>
|
|
|
|
/**
|
|
* Sign S3 objects to be used by anonymous users in public apps
|
|
* @param s3objects s3 objects to sign
|
|
* @param expirySecs how long the signature stays valid, in seconds (default 43200 = 12h, clamped to [60, 604800])
|
|
* @returns signed s3 objects
|
|
*/
|
|
async signS3Objects(s3objects: S3Object[], { expirySecs }: { expirySecs?: number } = {}): Promise<S3Object[]>
|
|
|
|
/**
|
|
* Sign S3 object to be used by anonymous users in public apps
|
|
* @param s3object s3 object to sign
|
|
* @param expirySecs how long the signature stays valid, in seconds (default 43200 = 12h, clamped to [60, 604800])
|
|
* @returns signed s3 object
|
|
*/
|
|
async signS3Object(s3object: S3Object, { expirySecs }: { expirySecs?: number } = {}): Promise<S3Object>
|
|
|
|
/**
|
|
* Generate a presigned public URL for an array of S3 objects.
|
|
* If an S3 object is not signed yet, it will be signed first.
|
|
* @param s3Objects s3 objects to sign
|
|
* @param expirySecs how long the signature stays valid, in seconds (default 43200 = 12h, clamped to [60, 604800])
|
|
* @returns list of signed public URLs
|
|
*/
|
|
async getPresignedS3PublicUrls(s3Objects: S3Object[], { baseUrl, expirySecs }: { baseUrl?: string; expirySecs?: number } = {}): Promise<string[]>
|
|
|
|
/**
|
|
* Generate a presigned public URL for an S3 object. If the S3 object is not signed yet, it will be signed first.
|
|
* @param s3Object s3 object to sign
|
|
* @param expirySecs how long the signature stays valid, in seconds (default 43200 = 12h, clamped to [60, 604800])
|
|
* @returns signed public URL
|
|
*/
|
|
async getPresignedS3PublicUrl(s3Objects: S3Object, { baseUrl, expirySecs }: { baseUrl?: string; expirySecs?: number } = {}): Promise<string>
|
|
|
|
/**
|
|
* Get URLs needed for resuming a flow after this step
|
|
* @param approver approver name
|
|
* @param flowLevel if true, generate resume URLs for the parent flow instead of the specific step.
|
|
* This allows pre-approvals that can be consumed by any later suspend step in the same flow.
|
|
* @returns approval page UI URL, resume and cancel API URLs for resuming the flow
|
|
*/
|
|
async getResumeUrls(approver?: string, flowLevel?: boolean): Promise<{
|
|
approvalPage: string;
|
|
resume: string;
|
|
cancel: string;
|
|
}>
|
|
|
|
/**
|
|
* Get an OIDC jwt token for auth to external services (e.g: Vault, AWS) (ee only)
|
|
* @param audience audience of the token
|
|
* @param expiresIn Optional number of seconds until the token expires
|
|
* @returns jwt token
|
|
*/
|
|
async getIdToken(audience: string, expiresIn?: number): Promise<string>
|
|
|
|
/**
|
|
* Convert a base64-encoded string to Uint8Array
|
|
* @param data - Base64-encoded string
|
|
* @returns Decoded Uint8Array
|
|
*/
|
|
base64ToUint8Array(data: string): Uint8Array
|
|
|
|
/**
|
|
* Convert a Uint8Array to base64-encoded string
|
|
* @param arrayBuffer - Uint8Array to encode
|
|
* @returns Base64-encoded string
|
|
*/
|
|
uint8ArrayToBase64(arrayBuffer: Uint8Array): string
|
|
|
|
/**
|
|
* Sends an interactive approval request via Slack, allowing optional customization of the message, approver, and form fields.
|
|
*
|
|
* **[Enterprise Edition Only]** To include form fields in the Slack approval request, go to **Advanced -> Suspend -> Form**
|
|
* and define a form. Learn more at [Windmill Documentation](https://www.windmill.dev/docs/flows/flow_approval#form).
|
|
*
|
|
* @param {Object} options - The configuration options for the Slack approval request.
|
|
* @param {string} options.slackResourcePath - The path to the Slack resource in Windmill.
|
|
* @param {string} options.channelId - The Slack channel ID where the approval request will be sent.
|
|
* @param {string} [options.message] - Optional custom message to include in the Slack approval request.
|
|
* @param {string} [options.approver] - Optional user ID or name of the approver for the request.
|
|
* @param {DefaultArgs} [options.defaultArgsJson] - Optional object defining or overriding the default arguments to a form field.
|
|
* @param {Enums} [options.dynamicEnumsJson] - Optional object overriding the enum default values of an enum form field.
|
|
* @param {string} [options.resumeButtonText] - Optional text for the resume button.
|
|
* @param {string} [options.cancelButtonText] - Optional text for the cancel button.
|
|
*
|
|
* @returns {Promise<void>} Resolves when the Slack approval request is successfully sent.
|
|
*
|
|
* @throws {Error} If the function is not called within a flow or flow preview.
|
|
* @throws {Error} If the \`JobService.getSlackApprovalPayload\` call fails.
|
|
*
|
|
* **Usage Example:**
|
|
* \`\`\`typescript
|
|
* await requestInteractiveSlackApproval({
|
|
* slackResourcePath: "/u/alex/my_slack_resource",
|
|
* channelId: "admins-slack-channel",
|
|
* message: "Please approve this request",
|
|
* approver: "approver123",
|
|
* defaultArgsJson: { key1: "value1", key2: 42 },
|
|
* dynamicEnumsJson: { foo: ["choice1", "choice2"], bar: ["optionA", "optionB"] },
|
|
* resumeButtonText: "Resume",
|
|
* cancelButtonText: "Cancel",
|
|
* });
|
|
* \`\`\`
|
|
*
|
|
* **Note:** This function requires execution within a Windmill flow or flow preview.
|
|
*/
|
|
async requestInteractiveSlackApproval({ slackResourcePath, channelId, message, approver, defaultArgsJson, dynamicEnumsJson, resumeButtonText, cancelButtonText, }: SlackApprovalOptions): Promise<void>
|
|
|
|
/**
|
|
* Sends an interactive approval request via Teams, allowing optional customization of the message, approver, and form fields.
|
|
*
|
|
* **[Enterprise Edition Only]** To include form fields in the Teams approval request, go to **Advanced -> Suspend -> Form**
|
|
* and define a form. Learn more at [Windmill Documentation](https://www.windmill.dev/docs/flows/flow_approval#form).
|
|
*
|
|
* @param {Object} options - The configuration options for the Teams approval request.
|
|
* @param {string} options.teamName - The Teams team name where the approval request will be sent.
|
|
* @param {string} options.channelName - The Teams channel name where the approval request will be sent.
|
|
* @param {string} [options.message] - Optional custom message to include in the Teams approval request.
|
|
* @param {string} [options.approver] - Optional user ID or name of the approver for the request.
|
|
* @param {DefaultArgs} [options.defaultArgsJson] - Optional object defining or overriding the default arguments to a form field.
|
|
* @param {Enums} [options.dynamicEnumsJson] - Optional object overriding the enum default values of an enum form field.
|
|
*
|
|
* @returns {Promise<void>} Resolves when the Teams approval request is successfully sent.
|
|
*
|
|
* @throws {Error} If the function is not called within a flow or flow preview.
|
|
* @throws {Error} If the \`JobService.getTeamsApprovalPayload\` call fails.
|
|
*
|
|
* **Usage Example:**
|
|
* \`\`\`typescript
|
|
* await requestInteractiveTeamsApproval({
|
|
* teamName: "admins-teams",
|
|
* channelName: "admins-teams-channel",
|
|
* message: "Please approve this request",
|
|
* approver: "approver123",
|
|
* defaultArgsJson: { key1: "value1", key2: 42 },
|
|
* dynamicEnumsJson: { foo: ["choice1", "choice2"], bar: ["optionA", "optionB"] },
|
|
* });
|
|
* \`\`\`
|
|
*
|
|
* **Note:** This function requires execution within a Windmill flow or flow preview.
|
|
*/
|
|
async requestInteractiveTeamsApproval({ teamName, channelName, message, approver, defaultArgsJson, dynamicEnumsJson, }: TeamsApprovalOptions): Promise<void>
|
|
|
|
setWorkflowCtx(ctx: WorkflowCtx | null): void
|
|
|
|
async sleep(seconds: number): Promise<void>
|
|
|
|
/**
|
|
* Execute \`fn\` inline and checkpoint the result. On replay the cached value is
|
|
* returned without re-executing \`fn\`.
|
|
*
|
|
* \`fn\`'s result is encoded as JSON and decoded back before it is returned, so
|
|
* the round that runs the body sees the same types every replay sees: a \`Date\`
|
|
* comes back as a string, a \`Map\` as \`{}\`. {@link Jsonified} is that shape.
|
|
*/
|
|
async step<T>(name: string, fn: () => T | Promise<T>,): Promise<Jsonified<Awaited<T>>>
|
|
|
|
/**
|
|
* Create a task that dispatches to a separate Windmill script.
|
|
*
|
|
* @example
|
|
* const extract = taskScript("f/data/extract");
|
|
* // inside workflow: await extract({ url: "https://..." })
|
|
*/
|
|
taskScript(path: string, options?: TaskOptions): (...args: any[]) => PromiseLike<any>
|
|
|
|
/**
|
|
* Create a task that dispatches to a separate Windmill flow.
|
|
*
|
|
* @example
|
|
* const pipeline = taskFlow("f/etl/pipeline");
|
|
* // inside workflow: await pipeline({ input: data })
|
|
*/
|
|
taskFlow(path: string, options?: TaskOptions): (...args: any[]) => PromiseLike<any>
|
|
|
|
/**
|
|
* Mark an async function as a workflow-as-code entry point.
|
|
*
|
|
* The function must be **deterministic**: given the same inputs it must call
|
|
* tasks in the same order on every replay. Branching on task results is fine
|
|
* (results are replayed from checkpoint), but branching on external state
|
|
* (current time, random values, external API calls) must use \`step()\` to
|
|
* checkpoint the value so replays see the same result.
|
|
*/
|
|
workflow<T>(fn: (...args: any[]) => Promise<T>): void
|
|
|
|
/**
|
|
* Suspend the workflow and wait for an external approval.
|
|
*
|
|
* Pass \`key\` to name the step, then \`getApprovalUrls(key)\` yields the URLs that
|
|
* resume exactly this approval — route them through your own channel. Without a
|
|
* key the steps are named \`approval\`, \`approval_2\`, ...
|
|
*
|
|
* \`skin: "minimal"\` shows approvers only the request (form and approve/reject)
|
|
* instead of the detailed page with the workflow's details. \`description\` is
|
|
* shown above the form: a string, or a rich value such as \`{ markdown: "..." }\`.
|
|
*
|
|
* @example
|
|
* const urls = await step("urls", () => getApprovalUrls("manager"));
|
|
* await step("notify", () => sendEmail(urls.resume, urls.cancel));
|
|
* const { value, approver } = await waitForApproval({ key: "manager", timeout: 3600 });
|
|
*/
|
|
waitForApproval(options?: { timeout?: number; form?: object; selfApproval?: boolean; key?: string; skin?: "detailed" | "minimal"; description?: string | object; }): PromiseLike<{ value: any; approver: string; approved: boolean }>
|
|
|
|
/**
|
|
* Resume/cancel/approval-page URLs bound to one \`waitForApproval\` step.
|
|
*
|
|
* Unlike \`getResumeUrls()\`, which signs a random nonce, these address the very
|
|
* \`resume_job\` record the step's built-in approval buttons use, so they are
|
|
* stable across replays and safe to embed in a custom notification.
|
|
*
|
|
* \`stepKey\` must match the \`key\` given to \`waitForApproval\`. Keys must be unique
|
|
* within a workflow; reusing one throws rather than silently renaming it. The URL
|
|
* only resumes while that step is awaiting approval; used at any other moment it is
|
|
* rejected rather than banking a row a different approval would consume. Send it
|
|
* ahead of time — approvers just cannot act before the workflow reaches the step.
|
|
*
|
|
* \`resume\` and \`cancel\` are step-bound; \`approvalPage\` is not — it opens the job's
|
|
* approval page, which acts on whichever approval is pending when it is used.
|
|
*
|
|
* @example
|
|
* const urls = await step("urls", () => getApprovalUrls("manager"));
|
|
* await step("notify", () => sendEmail(urls.resume, urls.cancel));
|
|
* await waitForApproval({ key: "manager" });
|
|
*/
|
|
async getApprovalUrls(stepKey: string = "approval", approver?: string): Promise<{
|
|
approvalPage: string;
|
|
resume: string;
|
|
cancel: string;
|
|
}>
|
|
|
|
/**
|
|
* Process items in parallel with optional concurrency control.
|
|
*
|
|
* Each item is processed by calling \`fn(item)\`, which should be a task().
|
|
* Items are dispatched in batches of \`concurrency\` (default: all at once).
|
|
*
|
|
* @example
|
|
* const process = task(async (item: string) => { ... });
|
|
* const results = await parallel(items, process, { concurrency: 5 });
|
|
*/
|
|
async parallel<T, R>(items: T[], fn: (item: T) => PromiseLike<R> | R, options?: { concurrency?: number },): Promise<R[]>
|
|
|
|
/**
|
|
* Commit Kafka offsets for a trigger with auto_commit disabled.
|
|
* @param triggerPath - Path to the Kafka trigger (from event.wm_trigger.trigger_path)
|
|
* @param topic - Kafka topic name (from event.topic)
|
|
* @param partition - Partition number (from event.partition)
|
|
* @param offset - Message offset to commit (from event.offset)
|
|
*/
|
|
async commitKafkaOffsets(triggerPath: string, topic: string, partition: number, offset: number,): Promise<void>
|
|
|
|
/**
|
|
* Parse an S3 object from URI string or record format
|
|
* @param s3Object - S3 object as URI string (\`s3://storage/key\`, \`s3:///key\`
|
|
* for the default storage) or record. Any other string throws rather than
|
|
* falling back to an auto-generated key: an auto key is requested by
|
|
* omitting the object, and a fallback would silently misplace the upload
|
|
* on any typo.
|
|
* @returns S3 object record with storage and s3 key
|
|
*/
|
|
parseS3Object(s3Object: S3Object): S3ObjectRecord
|
|
|
|
/**
|
|
* Create a SQL template function for PostgreSQL/datatable queries
|
|
* @param name - Database/datatable name (default: "main")
|
|
* @param opts.role - Connect as this data table role instead of the data table's default one.
|
|
* Only meaningful on a data table under roles, and only for a role you are a tenant of.
|
|
* @returns SQL template function for building parameterized queries
|
|
* @example
|
|
* let sql = wmill.datatable()
|
|
* let name = 'Robin'
|
|
* let age = 21
|
|
* await sql\`
|
|
* SELECT * FROM friends
|
|
* WHERE name = \${name} AND age = \${age}::int
|
|
* \`.fetch()
|
|
* @example
|
|
* // Read through a restricted role
|
|
* let sql = wmill.datatable("main", { role: "analytics" })
|
|
*/
|
|
datatable(name: string = "main", opts?: DatatableOptions): DatatableSqlTemplateFunction
|
|
|
|
/**
|
|
* Create a SQL template function for DuckDB/ducklake queries
|
|
* @param name - DuckDB database name, optionally with a schema as \`name:schema\` (default: "main")
|
|
* @returns SQL template function for building parameterized queries
|
|
* @example
|
|
* let sql = wmill.ducklake()
|
|
* let name = 'Robin'
|
|
* let age = 21
|
|
* await sql\`
|
|
* SELECT * FROM friends
|
|
* WHERE name = \${name} AND age = \${age}
|
|
* \`.fetch()
|
|
* @example
|
|
* // Target a specific schema within the ducklake
|
|
* let sql = wmill.ducklake("my_lake:analytics")
|
|
*/
|
|
ducklake(name: string = "main"): SqlTemplateFunction
|
|
|
|
/**
|
|
* Idempotently materialize \`selectSql\` into a ducklake table for one
|
|
* partition (or the whole table when \`partition\` is omitted) — the client-side
|
|
* equivalent of the \`// materialize\` engine.
|
|
* With \`uniqueKey\` it upserts the slice (delete-by-key + insert); otherwise it
|
|
* replaces it (whole table → \`CREATE OR REPLACE\`; partition → delete + insert).
|
|
* Safe to re-run for the same partition (backfill / failure-recovery).
|
|
*
|
|
* Returns a lazy statement — call \`.execute()\` to run it:
|
|
* \`await wmill.upsertPartition({ table, selectSql, partition }).execute()\`.
|
|
*/
|
|
upsertPartition(opts: DucklakeMaterializeOptions): SqlStatement<any>
|
|
|
|
/**
|
|
* INSERT-only materialization (no dedup/replace) for append-only tables.
|
|
* Re-running the same partition duplicates rows — use only for immutable
|
|
* event-log sources.
|
|
*
|
|
* Returns a lazy statement — call \`.execute()\` to run it:
|
|
* \`await wmill.appendPartition({ table, selectSql, partition }).execute()\`.
|
|
*/
|
|
appendPartition(opts: Omit<DucklakeMaterializeOptions, "uniqueKey">,): SqlStatement<any>
|
|
`;
|
|
|
|
export const SDK_PYTHON = `# Python SDK (wmill)
|
|
|
|
Import: import wmill
|
|
|
|
The client configures itself from the job's environment — base URL, token and credentials mode
|
|
are all set before your code runs, so there is nothing to initialize and no reason to read
|
|
WM_TOKEN or BASE_INTERNAL_URL and build an API URL yourself. Reconstructing that by hand only
|
|
reintroduces details the client already handles. Call the SDK for anything Windmill, and use raw
|
|
HTTP for third-party APIs.
|
|
|
|
The functions below are the surface to prefer. For an endpoint none of them covers,
|
|
wmill.Windmill().get(endpoint) and .post(endpoint) issue an authenticated request against this
|
|
instance. What does not exist is a function name you guessed at: if it is not listed below, do
|
|
not call it.
|
|
|
|
To know who is running the script, read the contextual variables rather than calling the API:
|
|
\`os.environ.get("WM_END_USER_EMAIL") or os.environ.get("WM_EMAIL")\`. WM_END_USER_EMAIL is the app
|
|
viewer when the run was triggered from an app and empty otherwise (both variables are always
|
|
defined), WM_EMAIL is the user the job is permissioned as. WM_USERNAME is the matching username.
|
|
|
|
def worker_has_internal_server() -> bool
|
|
|
|
def get_mocked_api() -> Optional[dict]
|
|
|
|
# Get the HTTP client instance.
|
|
#
|
|
# Returns:
|
|
# Configured httpx.Client for API requests
|
|
def get_client() -> httpx.Client
|
|
|
|
# Make an HTTP GET request to the Windmill API.
|
|
#
|
|
# Args:
|
|
# endpoint: API endpoint path
|
|
# raise_for_status: Whether to raise an exception on HTTP errors
|
|
# **kwargs: Additional arguments passed to httpx.get
|
|
#
|
|
# Returns:
|
|
# HTTP response object
|
|
def get(endpoint, raise_for_status = True, **kwargs) -> httpx.Response
|
|
|
|
# Make an HTTP POST request to the Windmill API.
|
|
#
|
|
# Args:
|
|
# endpoint: API endpoint path
|
|
# raise_for_status: Whether to raise an exception on HTTP errors
|
|
# **kwargs: Additional arguments passed to httpx.post
|
|
#
|
|
# Returns:
|
|
# HTTP response object
|
|
def post(endpoint, raise_for_status = True, **kwargs) -> httpx.Response
|
|
|
|
# Create a new authentication token.
|
|
#
|
|
# Args:
|
|
# duration: Token validity duration (default: 1 day)
|
|
#
|
|
# Returns:
|
|
# New authentication token string
|
|
def create_token(duration = dt.timedelta(days=1)) -> str
|
|
|
|
# Create a script job by path and return its job id.
|
|
def run_script_by_path_async(path: str, args: dict = None, scheduled_in_secs: int = None, tag: str = None) -> str
|
|
|
|
# Create a script job by hash and return its job id.
|
|
def run_script_by_hash_async(hash_: str, args: dict = None, scheduled_in_secs: int = None, tag: str = None) -> str
|
|
|
|
# Create a flow job and return its job id.
|
|
def run_flow_async(path: str, args: dict = None, scheduled_in_secs: int = None, do_not_track_in_parent: bool = True, tag: str = None) -> str
|
|
|
|
# Run script by path synchronously and return its result.
|
|
def run_script_by_path(path: str, args: dict = None, timeout: dt.timedelta | int | float | None = None, verbose: bool = False, cleanup: bool = True, assert_result_is_not_none: bool = False, tag: str = None) -> Any
|
|
|
|
# Run script by hash synchronously and return its result.
|
|
def run_script_by_hash(hash_: str, args: dict = None, timeout: dt.timedelta | int | float | None = None, verbose: bool = False, cleanup: bool = True, assert_result_is_not_none: bool = False, tag: str = None) -> Any
|
|
|
|
# Run a script on the current worker without creating a job.
|
|
#
|
|
# On agent workers (no internal server), falls back to running a normal
|
|
# preview job and waiting for the result.
|
|
def run_inline_script_preview(content: str, language: str, args: dict = None) -> Any
|
|
|
|
# Wait for a job to complete and return its result.
|
|
#
|
|
# Args:
|
|
# job_id: ID of the job to wait for
|
|
# timeout: Maximum time to wait (seconds or timedelta)
|
|
# verbose: Enable verbose logging
|
|
# cleanup: Register cleanup handler to cancel job on exit
|
|
# assert_result_is_not_none: Raise exception if result is None
|
|
#
|
|
# Returns:
|
|
# Job result when completed
|
|
#
|
|
# Raises:
|
|
# TimeoutError: If timeout is reached
|
|
# Exception: If job fails
|
|
def wait_job(job_id, timeout: dt.timedelta | int | float | None = None, verbose: bool = False, cleanup: bool = True, assert_result_is_not_none: bool = False)
|
|
|
|
# Cancel a specific job by ID.
|
|
#
|
|
# Args:
|
|
# job_id: UUID of the job to cancel
|
|
# reason: Optional reason for cancellation
|
|
#
|
|
# Returns:
|
|
# Response message from the cancel endpoint
|
|
def cancel_job(job_id: str, reason: str = None) -> str
|
|
|
|
# Cancel currently running executions of the same script.
|
|
def cancel_running() -> dict
|
|
|
|
# Get job details by ID.
|
|
#
|
|
# Args:
|
|
# job_id: UUID of the job
|
|
#
|
|
# Returns:
|
|
# Job details dictionary
|
|
def get_job(job_id: str) -> dict
|
|
|
|
# Get the root job ID for a flow hierarchy.
|
|
#
|
|
# Args:
|
|
# job_id: Job ID (defaults to current WM_JOB_ID)
|
|
#
|
|
# Returns:
|
|
# Root job ID
|
|
def get_root_job_id(job_id: str | None = None) -> dict
|
|
|
|
# Get an OIDC JWT token for authentication to external services.
|
|
#
|
|
# Args:
|
|
# audience: Token audience (e.g., "vault", "aws")
|
|
# expires_in: Optional expiration time in seconds
|
|
#
|
|
# Returns:
|
|
# JWT token string
|
|
def get_id_token(audience: str, expires_in: int | None = None) -> str
|
|
|
|
# Get the status of a job.
|
|
#
|
|
# Args:
|
|
# job_id: UUID of the job
|
|
#
|
|
# Returns:
|
|
# Job status: "RUNNING", "WAITING", or "COMPLETED"
|
|
def get_job_status(job_id: str) -> JobStatus
|
|
|
|
# Get the result of a completed job.
|
|
#
|
|
# Args:
|
|
# job_id: UUID of the completed job
|
|
# assert_result_is_not_none: Raise exception if result is None
|
|
#
|
|
# Returns:
|
|
# Job result
|
|
def get_result(job_id: str, assert_result_is_not_none: bool = True) -> Any
|
|
|
|
# Get a variable value by path.
|
|
#
|
|
# Args:
|
|
# path: Variable path in Windmill
|
|
#
|
|
# Returns:
|
|
# Variable value as string
|
|
def get_variable(path: str) -> str
|
|
|
|
# Set a variable value by path, creating it if it doesn't exist.
|
|
#
|
|
# Args:
|
|
# path: Variable path in Windmill
|
|
# value: Variable value to set
|
|
# is_secret: Whether the variable should be secret (default: False)
|
|
def set_variable(path: str, value: str, is_secret: bool = False) -> None
|
|
|
|
# Get a resource value by path.
|
|
#
|
|
# Args:
|
|
# path: Resource path in Windmill
|
|
# none_if_undefined: Return None instead of raising if not found
|
|
# interpolated: if variables and resources are fully unrolled
|
|
#
|
|
# Returns:
|
|
# Resource value dictionary or None
|
|
def get_resource(path: str, none_if_undefined: bool = False, interpolated: bool = True) -> dict | None
|
|
|
|
# Set a resource value by path, creating it if it doesn't exist.
|
|
#
|
|
# Args:
|
|
# value: Resource value to set
|
|
# path: Resource path in Windmill
|
|
# resource_type: Resource type for creation
|
|
def set_resource(value: Any, path: str, resource_type: str)
|
|
|
|
# List resources from Windmill workspace.
|
|
#
|
|
# Args:
|
|
# resource_type: Optional resource type to filter by (e.g., "postgresql", "mysql", "s3")
|
|
# page: Optional page number for pagination
|
|
# per_page: Optional number of results per page
|
|
#
|
|
# Returns:
|
|
# List of resource dictionaries
|
|
def list_resources(resource_type: str = None, page: int = None, per_page: int = None) -> list[dict]
|
|
|
|
# Set the workflow state.
|
|
#
|
|
# Args:
|
|
# value: State value to set
|
|
# path: Optional state resource path override.
|
|
def set_state(value: Any, path: str | None = None) -> None
|
|
|
|
# Get the workflow state.
|
|
#
|
|
# Args:
|
|
# path: Optional state resource path override.
|
|
#
|
|
# Returns:
|
|
# State value or None if not set
|
|
def get_state(path: str | None = None) -> Any
|
|
|
|
# Set job progress percentage (0-99).
|
|
#
|
|
# Args:
|
|
# value: Progress percentage
|
|
# job_id: Job ID (defaults to current WM_JOB_ID)
|
|
def set_progress(value: int, job_id: Optional[str] = None)
|
|
|
|
# Get job progress percentage.
|
|
#
|
|
# Args:
|
|
# job_id: Job ID (defaults to current WM_JOB_ID)
|
|
#
|
|
# Returns:
|
|
# Progress value (0-100) or None if not set
|
|
def get_progress(job_id: Optional[str] = None) -> Any
|
|
|
|
# Set the user state of a flow at a given key
|
|
def set_flow_user_state(key: str, value: Any) -> None
|
|
|
|
# Get the user state of a flow at a given key
|
|
def get_flow_user_state(key: str) -> Any
|
|
|
|
# Get the Windmill server version.
|
|
#
|
|
# Returns:
|
|
# Version string
|
|
def version()
|
|
|
|
# Convenient helpers that takes an S3 resource as input and returns the settings necessary to
|
|
# initiate an S3 connection from DuckDB
|
|
def get_duckdb_connection_settings(s3_resource_path: str = '') -> DuckDbConnectionSettings | None
|
|
|
|
# Convenient helpers that takes an S3 resource as input and returns the settings necessary to
|
|
# initiate an S3 connection from Polars
|
|
def get_polars_connection_settings(s3_resource_path: str = '') -> PolarsConnectionSettings
|
|
|
|
# Convenient helpers that takes an S3 resource as input and returns the settings necessary to
|
|
# initiate an S3 connection using boto3
|
|
def get_boto3_connection_settings(s3_resource_path: str = '') -> Boto3ConnectionSettings
|
|
|
|
# Load a file from the workspace s3 bucket and returns its content as bytes.
|
|
#
|
|
# '''python
|
|
# from wmill import S3Object
|
|
#
|
|
# s3_obj = S3Object(s3="/path/to/my_file.txt")
|
|
# my_obj_content = client.load_s3_file(s3_obj)
|
|
# file_content = my_obj_content.decode("utf-8")
|
|
# '''
|
|
def load_s3_file(s3object: S3Object | str, s3_resource_path: str | None) -> bytes
|
|
|
|
# Load a file from the workspace s3 bucket and returns the bytes stream.
|
|
#
|
|
# '''python
|
|
# from wmill import S3Object
|
|
#
|
|
# s3_obj = S3Object(s3="/path/to/my_file.txt")
|
|
# with wmill.load_s3_file_reader(s3object, s3_resource_path) as file_reader:
|
|
# print(file_reader.read())
|
|
# '''
|
|
def load_s3_file_reader(s3object: S3Object | str, s3_resource_path: str | None) -> BufferedReader
|
|
|
|
# Write a file to the workspace S3 bucket
|
|
#
|
|
# '''python
|
|
# from wmill import S3Object
|
|
#
|
|
# s3_obj = S3Object(s3="/path/to/my_file.txt")
|
|
#
|
|
# # for an in memory bytes array:
|
|
# file_content = b'Hello Windmill!'
|
|
# client.write_s3_file(s3_obj, file_content)
|
|
#
|
|
# # for a file:
|
|
# with open("my_file.txt", "rb") as my_file:
|
|
# client.write_s3_file(s3_obj, my_file)
|
|
# '''
|
|
def write_s3_file(s3object: S3Object | str | None, file_content: BufferedReader | bytes, s3_resource_path: str | None, content_type: str | None = None, content_disposition: str | None = None) -> S3Object
|
|
|
|
# Permanently delete a file from the workspace S3 bucket.
|
|
#
|
|
# '''python
|
|
# from wmill import S3Object
|
|
#
|
|
# s3_obj = S3Object(s3="/path/to/my_file.txt")
|
|
# client.delete_s3_object(s3_obj)
|
|
# '''
|
|
def delete_s3_object(s3object: S3Object | str, s3_resource_path: str | None = None) -> None
|
|
|
|
# Sign S3 objects for use by anonymous users in public apps.
|
|
#
|
|
# Args:
|
|
# s3_objects: List of S3 objects to sign
|
|
# expiry_secs: How long the signature stays valid, in seconds
|
|
# (defaults to 43200 = 12h, clamped to [60, 604800])
|
|
#
|
|
# Returns:
|
|
# List of signed S3 objects
|
|
def sign_s3_objects(s3_objects: list[S3Object | str], expiry_secs: int | None = None) -> list[S3Object]
|
|
|
|
# Sign a single S3 object for use by anonymous users in public apps.
|
|
#
|
|
# Args:
|
|
# s3_object: S3 object to sign
|
|
# expiry_secs: How long the signature stays valid, in seconds
|
|
# (defaults to 43200 = 12h, clamped to [60, 604800])
|
|
#
|
|
# Returns:
|
|
# Signed S3 object
|
|
def sign_s3_object(s3_object: S3Object | str, expiry_secs: int | None = None) -> S3Object
|
|
|
|
# Generate presigned public URLs for an array of S3 objects.
|
|
# If an S3 object is not signed yet, it will be signed first.
|
|
#
|
|
# Args:
|
|
# s3_objects: List of S3 objects to sign
|
|
# base_url: Optional base URL for the presigned URLs (defaults to WM_BASE_URL)
|
|
# expiry_secs: How long the signatures stay valid, in seconds
|
|
# (defaults to 43200 = 12h, clamped to [60, 604800])
|
|
#
|
|
# Returns:
|
|
# List of signed public URLs
|
|
#
|
|
# Example:
|
|
# >>> s3_objs = [S3Object(s3="/path/to/file1.txt"), S3Object(s3="/path/to/file2.txt")]
|
|
# >>> urls = client.get_presigned_s3_public_urls(s3_objs)
|
|
def get_presigned_s3_public_urls(s3_objects: list[S3Object | str], base_url: str | None = None, expiry_secs: int | None = None) -> list[str]
|
|
|
|
# Generate a presigned public URL for an S3 object.
|
|
# If the S3 object is not signed yet, it will be signed first.
|
|
#
|
|
# Args:
|
|
# s3_object: S3 object to sign
|
|
# base_url: Optional base URL for the presigned URL (defaults to WM_BASE_URL)
|
|
# expiry_secs: How long the signature stays valid, in seconds
|
|
# (defaults to 43200 = 12h, clamped to [60, 604800])
|
|
#
|
|
# Returns:
|
|
# Signed public URL
|
|
#
|
|
# Example:
|
|
# >>> s3_obj = S3Object(s3="/path/to/file.txt")
|
|
# >>> url = client.get_presigned_s3_public_url(s3_obj)
|
|
def get_presigned_s3_public_url(s3_object: S3Object | str, base_url: str | None = None, expiry_secs: int | None = None) -> str
|
|
|
|
# Get the current user information.
|
|
#
|
|
# Returns:
|
|
# User details dictionary
|
|
def whoami() -> dict
|
|
|
|
# Get the current user information (alias for whoami).
|
|
#
|
|
# Returns:
|
|
# User details dictionary
|
|
def user() -> dict
|
|
|
|
# Get the state resource path from environment.
|
|
#
|
|
# Returns:
|
|
# State path string
|
|
def state_path() -> str
|
|
|
|
# Get the workflow state.
|
|
#
|
|
# Returns:
|
|
# State value or None if not set
|
|
def state() -> Any
|
|
|
|
# Set the state in the shared folder using pickle
|
|
def set_shared_state_pickle(value: Any, path: str = 'state.pickle') -> None
|
|
|
|
# Get the state in the shared folder using pickle
|
|
def get_shared_state_pickle(path: str = 'state.pickle') -> Any
|
|
|
|
# Set the state in the shared folder using pickle
|
|
def set_shared_state(value: Any, path: str = 'state.json') -> None
|
|
|
|
# Get the state in the shared folder using pickle
|
|
def get_shared_state(path: str = 'state.json') -> None
|
|
|
|
# Get URLs needed for resuming a flow after suspension.
|
|
#
|
|
# Args:
|
|
# approver: Optional approver name
|
|
# flow_level: If True, generate resume URLs for the parent flow instead of the
|
|
# specific step. This allows pre-approvals that can be consumed by any later
|
|
# suspend step in the same flow.
|
|
#
|
|
# Returns:
|
|
# Dictionary with approvalPage, resume, and cancel URLs
|
|
def get_resume_urls(approver: str = None, flow_level: bool = None) -> dict
|
|
|
|
# Get the resume URLs bound to one \`\`wait_for_approval\`\` step of this workflow.
|
|
#
|
|
# Args:
|
|
# step_key: Checkpoint key of the approval step, as passed to
|
|
# \`\`wait_for_approval(key=...)\`\`
|
|
# approver: Optional approver name
|
|
#
|
|
# Returns:
|
|
# Dictionary with approvalPage, resume, and cancel URLs
|
|
def get_approval_urls(step_key: str = 'approval', approver: str = None) -> dict
|
|
|
|
# Sends an interactive approval request via Slack, allowing optional customization of the message, approver, and form fields.
|
|
#
|
|
# **[Enterprise Edition Only]** To include form fields in the Slack approval request, use the "Advanced -> Suspend -> Form" functionality.
|
|
# Learn more at: https://www.windmill.dev/docs/flows/flow_approval#form
|
|
#
|
|
# :param slack_resource_path: The path to the Slack resource in Windmill.
|
|
# :type slack_resource_path: str
|
|
# :param channel_id: The Slack channel ID where the approval request will be sent.
|
|
# :type channel_id: str
|
|
# :param message: Optional custom message to include in the Slack approval request.
|
|
# :type message: str, optional
|
|
# :param approver: Optional user ID or name of the approver for the request.
|
|
# :type approver: str, optional
|
|
# :param default_args_json: Optional dictionary defining or overriding the default arguments for form fields.
|
|
# :type default_args_json: dict, optional
|
|
# :param dynamic_enums_json: Optional dictionary overriding the enum default values of enum form fields.
|
|
# :type dynamic_enums_json: dict, optional
|
|
#
|
|
# :raises Exception: If the function is not called within a flow or flow preview.
|
|
# :raises Exception: If the required flow job or flow step environment variables are not set.
|
|
#
|
|
# :return: None
|
|
#
|
|
# **Usage Example:**
|
|
# >>> client.request_interactive_slack_approval(
|
|
# ... slack_resource_path="/u/alex/my_slack_resource",
|
|
# ... channel_id="admins-slack-channel",
|
|
# ... message="Please approve this request",
|
|
# ... approver="approver123",
|
|
# ... default_args_json={"key1": "value1", "key2": 42},
|
|
# ... dynamic_enums_json={"foo": ["choice1", "choice2"], "bar": ["optionA", "optionB"]},
|
|
# ... )
|
|
#
|
|
# **Notes:**
|
|
# - This function must be executed within a Windmill flow or flow preview.
|
|
# - The function checks for required environment variables (\`WM_FLOW_JOB_ID\`, \`WM_FLOW_STEP_ID\`) to ensure it is run in the appropriate context.
|
|
def request_interactive_slack_approval(slack_resource_path: str, channel_id: str, message: str = None, approver: str = None, default_args_json: dict = None, dynamic_enums_json: dict = None) -> None
|
|
|
|
# Send a message to a Microsoft Teams conversation with conversation_id, where success is used to style the message
|
|
def send_teams_message(conversation_id: str, text: str, success: bool = True, card_block: dict = None)
|
|
|
|
# Get a DataTable client for SQL queries.
|
|
#
|
|
# Args:
|
|
# name: Database name (default: "main")
|
|
# role: Connect as this data table role instead of the data table's default one.
|
|
# Only meaningful on a data table under roles, and only for a role you are a
|
|
# tenant of.
|
|
#
|
|
# Returns:
|
|
# DataTableClient instance
|
|
def datatable(name: str = 'main', *, role: Optional[str] = None)
|
|
|
|
# Get a DuckLake client for DuckDB queries.
|
|
#
|
|
# Args:
|
|
# name: Database name (default: "main")
|
|
#
|
|
# Returns:
|
|
# DucklakeClient instance
|
|
def ducklake(name: str = 'main')
|
|
|
|
def init_global_client(f)
|
|
|
|
def deprecate(in_favor_of: str)
|
|
|
|
# Get the current workspace ID.
|
|
#
|
|
# Returns:
|
|
# Workspace ID string
|
|
def get_workspace() -> str
|
|
|
|
def get_version() -> str
|
|
|
|
# Create a script job and return its job ID.
|
|
#
|
|
# Args:
|
|
# hash_or_path: Script hash or path (determined by presence of '/')
|
|
# args: Script arguments
|
|
# scheduled_in_secs: Delay before execution in seconds
|
|
# tag: Override the worker tag the job runs on
|
|
#
|
|
# Returns:
|
|
# Job ID string
|
|
def run_script_async(hash_or_path: str, args: Dict[str, Any] = None, scheduled_in_secs: int = None, tag: str = None) -> str
|
|
|
|
# Run a script synchronously by hash and return its result.
|
|
#
|
|
# Args:
|
|
# hash: Script hash
|
|
# args: Script arguments
|
|
# verbose: Enable verbose logging
|
|
# assert_result_is_not_none: Raise exception if result is None
|
|
# cleanup: Register cleanup handler to cancel job on exit
|
|
# timeout: Maximum time to wait
|
|
# tag: Override the worker tag the job runs on
|
|
#
|
|
# Returns:
|
|
# Script result
|
|
def run_script_sync(hash: str, args: Dict[str, Any] = None, verbose: bool = False, assert_result_is_not_none: bool = True, cleanup: bool = True, timeout: dt.timedelta = None, tag: str = None) -> Any
|
|
|
|
# Run a script synchronously by path and return its result.
|
|
#
|
|
# Args:
|
|
# path: Script path
|
|
# args: Script arguments
|
|
# verbose: Enable verbose logging
|
|
# assert_result_is_not_none: Raise exception if result is None
|
|
# cleanup: Register cleanup handler to cancel job on exit
|
|
# timeout: Maximum time to wait
|
|
# tag: Override the worker tag the job runs on
|
|
#
|
|
# Returns:
|
|
# Script result
|
|
def run_script_by_path_sync(path: str, args: Dict[str, Any] = None, verbose: bool = False, assert_result_is_not_none: bool = True, cleanup: bool = True, timeout: dt.timedelta = None, tag: str = None) -> Any
|
|
|
|
# Convenient helpers that takes an S3 resource as input and returns the settings necessary to
|
|
# initiate an S3 connection from DuckDB
|
|
def duckdb_connection_settings(s3_resource_path: str = '') -> DuckDbConnectionSettings
|
|
|
|
# Convenient helpers that takes an S3 resource as input and returns the settings necessary to
|
|
# initiate an S3 connection from Polars
|
|
def polars_connection_settings(s3_resource_path: str = '') -> PolarsConnectionSettings
|
|
|
|
# Convenient helpers that takes an S3 resource as input and returns the settings necessary to
|
|
# initiate an S3 connection using boto3
|
|
def boto3_connection_settings(s3_resource_path: str = '') -> Boto3ConnectionSettings
|
|
|
|
# Get the state resource path from environment.
|
|
#
|
|
# Returns:
|
|
# State path string
|
|
def get_state_path() -> str
|
|
|
|
# Parse resource syntax from string.
|
|
def parse_resource_syntax(s: str) -> Optional[str]
|
|
|
|
# Parse S3 object from a \`s3://<storage>/<key>\` URI string (\`s3:///<key>\`
|
|
# for the default storage) or S3Object format. Any other string raises
|
|
# rather than falling back to an auto-generated key: an auto key is
|
|
# requested by omitting the object, and a fallback would silently misplace
|
|
# the upload on any typo.
|
|
def parse_s3_object(s3_object: S3Object | str) -> S3Object
|
|
|
|
# Parse variable syntax from string.
|
|
def parse_variable_syntax(s: str) -> Optional[str]
|
|
|
|
# Append a text to the result stream.
|
|
#
|
|
# Args:
|
|
# text: text to append to the result stream
|
|
def append_to_result_stream(text: str) -> None
|
|
|
|
# Stream to the result stream.
|
|
#
|
|
# Args:
|
|
# stream: stream to stream to the result stream
|
|
def stream_result(stream) -> None
|
|
|
|
# Execute a SQL query against the DataTable.
|
|
#
|
|
# Args:
|
|
# sql: SQL query string with $1, $2, etc. placeholders
|
|
# *args: Positional arguments to bind to query placeholders
|
|
#
|
|
# Returns:
|
|
# SqlQuery instance for fetching results
|
|
def query(sql: str, *args) -> SqlQuery
|
|
|
|
# Idempotently materialize the rows of \`select_sql\` into ducklake
|
|
# \`table\` for one \`partition\` (or the whole table when \`partition\` is
|
|
# None). Client-side equivalent of the \`// materialize\` engine: with
|
|
# \`unique_key\` it upserts within the slice (delete-by-key + insert);
|
|
# without it, it replaces (whole table → CREATE OR REPLACE; partition →
|
|
# delete the partition + insert). Re-running the same slice is safe — the
|
|
# backfill / failure-recovery contract.
|
|
#
|
|
# The partition value is bound as a DuckDB arg (never string-interpolated)
|
|
# so it cannot inject SQL. \`select_sql\` is trusted (your own query).
|
|
def upsert_partition(table: str, select_sql: str, partition: str = None, unique_key: str = None, partition_col: str = '_wm_partition', schema: str = None)
|
|
|
|
# INSERT-only materialization (no dedup / no replace) for an immutable
|
|
# event-log table — for one \`partition\`, or the whole table when
|
|
# \`partition\` is None. NOTE: unlike \`upsert_partition\`, re-running the same
|
|
# slice duplicates rows — use only for append-only sources.
|
|
def append_partition(table: str, select_sql: str, partition: str = None, partition_col: str = '_wm_partition', schema: str = None)
|
|
|
|
# Read a materialized ducklake table, optionally a single partition.
|
|
def read(table: str, partition: str = None, partition_col: str = '_wm_partition', schema: str = None)
|
|
|
|
# Execute query and fetch results.
|
|
#
|
|
# Args:
|
|
# result_collection: Optional result collection mode
|
|
#
|
|
# Returns:
|
|
# Query results
|
|
def fetch(result_collection: str | None = None)
|
|
|
|
# Execute query and fetch first row of results.
|
|
#
|
|
# Returns:
|
|
# First row of query results
|
|
def fetch_one()
|
|
|
|
# Execute query and fetch first row of results. Return result as a scalar value.
|
|
#
|
|
# Returns:
|
|
# First row of query result as a scalar value
|
|
def fetch_one_scalar()
|
|
|
|
# Execute query and don't return any results.
|
|
#
|
|
def execute()
|
|
|
|
# DuckDB executor requires explicit argument types at declaration
|
|
# These types exist in both DuckDB and Postgres
|
|
# Check that the types exist if you plan to extend this function for other SQL engines.
|
|
def infer_sql_type(value) -> str
|
|
|
|
def parse_sql_client_name(name: str) -> tuple[str, Optional[str]]
|
|
|
|
# Decorator that marks a function as a workflow task.
|
|
#
|
|
# Works in both WAC v1 (sync, HTTP-based dispatch) and WAC v2
|
|
# (async, checkpoint/replay) modes:
|
|
#
|
|
# - **v2 (inside @workflow)**: dispatches as a checkpoint step.
|
|
# - **v1 (WM_JOB_ID set, no @workflow)**: dispatches via HTTP API.
|
|
# - **Standalone**: executes the function body directly.
|
|
#
|
|
# A task runs as its own job, so its result is always encoded as JSON and
|
|
# decoded back before the caller sees it: a \`\`datetime\`\` comes back as a
|
|
# string, a tuple as a list.
|
|
#
|
|
# \`\`retry\`\` re-dispatches the task after a failure, inside \`\`@workflow\`\` only.
|
|
# Every attempt is a step of its own (\`\`call_api\`\`, \`\`call_api#2\`\`, ...) and
|
|
# the wait between two of them is a durable sleep, so a retrying task holds no
|
|
# worker while it backs off. Keys: \`\`attempts\`\` (retries after the first
|
|
# failure, a whole number from 0 to 100), \`\`delay\`\` (seconds before the first
|
|
# retry, sub-second delays dropped), \`\`multiplier\`\` (applied to the delay
|
|
# after each attempt, 1 keeps it constant), \`\`max_delay\`\` (ceiling in
|
|
# seconds). \`\`attempts\`\` is required, and an out-of-range or unknown key is
|
|
# rejected where the policy is written.
|
|
#
|
|
# A workflow sleeps once per round, so tasks backing off in the same fan-out
|
|
# wait one after another rather than together: the delay before a fan-out
|
|
# retries is the sum of every backoff pending in it, not the longest one, and
|
|
# it grows with both the width of the fan-out and \`\`attempts\`\`. Retries with
|
|
# no \`\`delay\`\` all go out in a single round.
|
|
#
|
|
# \`\`cache_ttl\`\` serves a previous result of the task for that many seconds
|
|
# instead of running it again. A task is keyed on its step key (its name and
|
|
# call order) and the workflow's input, not on the arguments it is called
|
|
# with, so cache one only when whether it runs, and what it receives, follow
|
|
# from the workflow's input alone. A \`\`task_script\`\` target is keyed on the
|
|
# arguments it is called with. It has no effect on a \`\`task_flow\`\` target,
|
|
# which keeps its flow's own cache policy.
|
|
#
|
|
# Usage::
|
|
#
|
|
# @task
|
|
# async def extract_data(url: str): ...
|
|
#
|
|
# @task(path="f/external_script", timeout=600, tag="gpu")
|
|
# async def run_external(x: int): ...
|
|
#
|
|
# @task(retry={"attempts": 3, "delay": 30, "multiplier": 2})
|
|
# async def call_api(payload: dict): ...
|
|
def task(_func = None, *, path: Optional[str] = None, tag: Optional[str] = None, timeout: Optional[int] = None, cache_ttl: Optional[int] = None, priority: Optional[int] = None, concurrency_limit: Optional[int] = None, concurrency_key: Optional[str] = None, concurrency_time_window_s: Optional[int] = None, retry: Optional[dict] = None)
|
|
|
|
# Create a task that dispatches to a separate Windmill script.
|
|
#
|
|
# \`\`retry\`\` takes the same policy as :func:\`task\`.
|
|
#
|
|
# Usage::
|
|
#
|
|
# extract = task_script("f/data/extract", timeout=600)
|
|
#
|
|
# @workflow
|
|
# async def main():
|
|
# data = await extract(url="https://...")
|
|
def task_script(path: str, *, timeout: Optional[int] = None, tag: Optional[str] = None, cache_ttl: Optional[int] = None, priority: Optional[int] = None, concurrency_limit: Optional[int] = None, concurrency_key: Optional[str] = None, concurrency_time_window_s: Optional[int] = None, retry: Optional[dict] = None)
|
|
|
|
# Create a task that dispatches to a separate Windmill flow.
|
|
#
|
|
# \`\`retry\`\` takes the same policy as :func:\`task\`.
|
|
#
|
|
# Usage::
|
|
#
|
|
# pipeline = task_flow("f/etl/pipeline", priority=10)
|
|
#
|
|
# @workflow
|
|
# async def main():
|
|
# result = await pipeline(input=data)
|
|
def task_flow(path: str, *, timeout: Optional[int] = None, tag: Optional[str] = None, cache_ttl: Optional[int] = None, priority: Optional[int] = None, concurrency_limit: Optional[int] = None, concurrency_key: Optional[str] = None, concurrency_time_window_s: Optional[int] = None, retry: Optional[dict] = None)
|
|
|
|
# Decorator marking an async function as a workflow-as-code entry point.
|
|
#
|
|
# The function must be **deterministic**: given the same inputs it must call
|
|
# tasks in the same order on every replay. Branching on task results is fine
|
|
# (results are replayed from checkpoint), but branching on external state
|
|
# (current time, random values, external API calls) must use \`\`step()\`\` to
|
|
# checkpoint the value so replays see the same result.
|
|
def workflow(func)
|
|
|
|
# Execute \`\`fn\`\` inline and checkpoint the result.
|
|
#
|
|
# On replay the cached value is returned without re-executing \`\`fn\`\`.
|
|
# Use for lightweight deterministic operations (timestamps, random IDs,
|
|
# config reads) that should not incur the overhead of a child job.
|
|
#
|
|
# \`\`fn\`\`'s result is encoded as JSON and decoded back before it is returned,
|
|
# so the round that runs the body sees the same types every replay sees:
|
|
# a \`\`datetime\`\` comes back as a string, a tuple as a list.
|
|
async def step(name: str, fn)
|
|
|
|
# Server-side sleep — suspend the workflow for the given duration without holding a worker.
|
|
#
|
|
# Inside a @workflow, the parent job suspends and auto-resumes after \`\`seconds\`\`.
|
|
# Outside a workflow, falls back to \`\`asyncio.sleep\`\`.
|
|
async def sleep(seconds: int)
|
|
|
|
# Suspend the workflow and wait for an external approval.
|
|
#
|
|
# Pass \`\`key\`\` to name the step, then \`\`get_approval_urls(key)\`\` yields the URLs
|
|
# that resume exactly this approval — route them through your own channel.
|
|
# Without a key the steps are named \`\`approval\`\`, \`\`approval_2\`\`, ...
|
|
#
|
|
# Returns a dict with \`\`value\`\` (form data), \`\`approver\`\`, and \`\`approved\`\`.
|
|
#
|
|
# Args:
|
|
# timeout: Approval timeout in seconds (default 1800).
|
|
# form: Optional form schema for the approval page.
|
|
# self_approval: Whether the user who triggered the flow can approve it (default True).
|
|
# key: Optional checkpoint key naming this approval step.
|
|
# skin: \`\`"minimal"\`\` shows approvers only the request (form and approve/reject)
|
|
# instead of the detailed page with the workflow's details.
|
|
# description: Shown to approvers above the form: a string, or a rich value such as
|
|
# \`\`{"markdown": "..."}\`\`.
|
|
#
|
|
# Example::
|
|
#
|
|
# urls = await step("urls", lambda: get_approval_urls("manager"))
|
|
# await step("notify", lambda: send_email(urls["resume"], urls["cancel"]))
|
|
# result = await wait_for_approval(key="manager", timeout=3600)
|
|
async def wait_for_approval(timeout: int = 1800, form: dict | None = None, self_approval: bool = True, key: str | None = None, skin: Literal['detailed', 'minimal'] | None = None, description: str | dict | None = None) -> dict
|
|
|
|
# Process items in parallel with optional concurrency control.
|
|
#
|
|
# Each item is processed by calling \`\`fn(item)\`\`, which should be a @task.
|
|
# Items are dispatched in batches of \`\`concurrency\`\` (default: all at once).
|
|
#
|
|
# Example::
|
|
#
|
|
# @task
|
|
# async def process(item: str):
|
|
# ...
|
|
#
|
|
# results = await parallel(items, process, concurrency=5)
|
|
async def parallel(items, fn, *, concurrency: Optional[int] = None)
|
|
|
|
# Commit Kafka offsets for a trigger with auto_commit disabled.
|
|
#
|
|
# Args:
|
|
# trigger_path: Path to the Kafka trigger (from event['wm_trigger']['trigger_path'])
|
|
# topic: Kafka topic name (from event['topic'])
|
|
# partition: Partition number (from event['partition'])
|
|
# offset: Message offset to commit (from event['offset'])
|
|
def commit_kafka_offsets(trigger_path: str, topic: str, partition: int, offset: int) -> None
|
|
|
|
`;
|
|
|
|
export const WAC_SDK_TYPESCRIPT = `## TypeScript Workflow-as-Code API (windmill-client)
|
|
|
|
Import: \`import { workflow, task, taskScript, taskFlow, step, sleep, waitForApproval, getApprovalUrls, getResumeUrls, parallel } from "windmill-client"\`
|
|
|
|
\`\`\`typescript
|
|
/**
|
|
* Re-dispatch policy for a failed task.
|
|
*
|
|
* Every attempt is a step of its own (\`fetch\`, \`fetch#2\`, \`fetch#3\`), and the
|
|
* wait between two of them is a durable sleep, so a retrying task holds no
|
|
* worker while it backs off.
|
|
*
|
|
* A workflow sleeps once per round, so tasks backing off in the same fan-out
|
|
* wait one after another rather than together: the delay before a fan-out
|
|
* retries is the sum of every backoff pending in it, not the longest one, and
|
|
* it grows with both the width of the fan-out and \`attempts\`. Retries with no
|
|
* \`delay\` all go out in a single round.
|
|
*/
|
|
export interface TaskRetry {
|
|
/** Attempts after the first failure: \`2\` runs the task at most 3 times.
|
|
* A whole number from 0 to 100; anything else is rejected where the policy
|
|
* is written. */
|
|
attempts: number;
|
|
/** Seconds to wait before the first retry. Default 0, retry immediately.
|
|
* Sub-second delays are dropped — a durable sleep resolves to the second. */
|
|
delay?: number;
|
|
/** Applied to the delay after each attempt: 1 (the default) keeps it
|
|
* constant, 2 doubles it. */
|
|
multiplier?: number;
|
|
/** Ceiling for the delay in seconds, for a \`multiplier\` above 1. */
|
|
max_delay?: number;
|
|
}
|
|
|
|
export interface TaskOptions {
|
|
timeout?: number;
|
|
tag?: string;
|
|
/** Seconds during which a previous result of this task is served instead of
|
|
* running it again. A task written inline in the workflow is keyed on its
|
|
* step key (its name and call order) and the workflow's input, not on the
|
|
* arguments it is called with, so cache one only when whether it runs, and
|
|
* what it receives, follow from the workflow's input alone. A \`taskScript\`
|
|
* target is keyed on the arguments it is called with. It has no effect on a
|
|
* \`taskFlow\` target, which keeps its flow's own cache policy. */
|
|
cache_ttl?: number;
|
|
priority?: number;
|
|
concurrency_limit?: number;
|
|
concurrency_key?: string;
|
|
concurrency_time_window_s?: number;
|
|
retry?: TaskRetry;
|
|
}
|
|
|
|
/**
|
|
* Get URLs needed for resuming a flow after this step
|
|
* @param approver approver name
|
|
* @param flowLevel if true, generate resume URLs for the parent flow instead of the specific step.
|
|
* This allows pre-approvals that can be consumed by any later suspend step in the same flow.
|
|
* @returns approval page UI URL, resume and cancel API URLs for resuming the flow
|
|
*/
|
|
export async function getResumeUrls(approver?: string, flowLevel?: boolean): Promise<{ approvalPage: string; resume: string; cancel: string; }>
|
|
|
|
/**
|
|
* Wrap an async function as a workflow task.
|
|
*
|
|
* @example
|
|
* const extract_data = task(async (url: string) => { ... });
|
|
* const run_external = task("f/external_script", async (x: number) => { ... });
|
|
* const call_api = task(fetchOrders, { retry: { attempts: 3, delay: 30, multiplier: 2 } });
|
|
*
|
|
* Inside a \`workflow()\`, calling a task dispatches it as a step.
|
|
* Outside a workflow, the function body executes directly and
|
|
* {@link TaskOptions} — retry included — does not apply.
|
|
*
|
|
* A task runs as its own job, so its result is always encoded as JSON and
|
|
* decoded back before the caller sees it: a \`Date\` comes back as a string, a
|
|
* \`Map\` as \`{}\`. {@link JsonifiedFn} is that shape.
|
|
*/
|
|
export function task<T extends (...args: any[]) => Promise<any>>(fnOrPath: T | string, maybeFnOrOptions?: T | TaskOptions, maybeOptions?: TaskOptions,): JsonifiedFn<T>
|
|
|
|
/**
|
|
* Create a task that dispatches to a separate Windmill script.
|
|
*
|
|
* @example
|
|
* const extract = taskScript("f/data/extract");
|
|
* // inside workflow: await extract({ url: "https://..." })
|
|
*/
|
|
export function taskScript(path: string, options?: TaskOptions): (...args: any[]) => PromiseLike<any>
|
|
|
|
/**
|
|
* Create a task that dispatches to a separate Windmill flow.
|
|
*
|
|
* @example
|
|
* const pipeline = taskFlow("f/etl/pipeline");
|
|
* // inside workflow: await pipeline({ input: data })
|
|
*/
|
|
export function taskFlow(path: string, options?: TaskOptions): (...args: any[]) => PromiseLike<any>
|
|
|
|
/**
|
|
* Mark an async function as a workflow-as-code entry point.
|
|
*
|
|
* The function must be **deterministic**: given the same inputs it must call
|
|
* tasks in the same order on every replay. Branching on task results is fine
|
|
* (results are replayed from checkpoint), but branching on external state
|
|
* (current time, random values, external API calls) must use \`step()\` to
|
|
* checkpoint the value so replays see the same result.
|
|
*/
|
|
export function workflow<T>(fn: (...args: any[]) => Promise<T>)
|
|
|
|
/**
|
|
* Execute \`fn\` inline and checkpoint the result. On replay the cached value is
|
|
* returned without re-executing \`fn\`.
|
|
*
|
|
* \`fn\`'s result is encoded as JSON and decoded back before it is returned, so
|
|
* the round that runs the body sees the same types every replay sees: a \`Date\`
|
|
* comes back as a string, a \`Map\` as \`{}\`. {@link Jsonified} is that shape.
|
|
*/
|
|
export async function step<T>(name: string, fn: () => T | Promise<T>,): Promise<Jsonified<Awaited<T>>>
|
|
|
|
export async function sleep(seconds: number): Promise<void>
|
|
|
|
/**
|
|
* Suspend the workflow and wait for an external approval.
|
|
*
|
|
* Pass \`key\` to name the step, then \`getApprovalUrls(key)\` yields the URLs that
|
|
* resume exactly this approval — route them through your own channel. Without a
|
|
* key the steps are named \`approval\`, \`approval_2\`, ...
|
|
*
|
|
* \`skin: "minimal"\` shows approvers only the request (form and approve/reject)
|
|
* instead of the detailed page with the workflow's details. \`description\` is
|
|
* shown above the form: a string, or a rich value such as \`{ markdown: "..." }\`.
|
|
*
|
|
* @example
|
|
* const urls = await step("urls", () => getApprovalUrls("manager"));
|
|
* await step("notify", () => sendEmail(urls.resume, urls.cancel));
|
|
* const { value, approver } = await waitForApproval({ key: "manager", timeout: 3600 });
|
|
*/
|
|
export function waitForApproval(options?: { timeout?: number; form?: object; selfApproval?: boolean; key?: string; skin?: "detailed" | "minimal"; description?: string | object; }): PromiseLike<{ value: any; approver: string; approved: boolean }>
|
|
|
|
/**
|
|
* Resume/cancel/approval-page URLs bound to one \`waitForApproval\` step.
|
|
*
|
|
* Unlike \`getResumeUrls()\`, which signs a random nonce, these address the very
|
|
* \`resume_job\` record the step's built-in approval buttons use, so they are
|
|
* stable across replays and safe to embed in a custom notification.
|
|
*
|
|
* \`stepKey\` must match the \`key\` given to \`waitForApproval\`. Keys must be unique
|
|
* within a workflow; reusing one throws rather than silently renaming it. The URL
|
|
* only resumes while that step is awaiting approval; used at any other moment it is
|
|
* rejected rather than banking a row a different approval would consume. Send it
|
|
* ahead of time — approvers just cannot act before the workflow reaches the step.
|
|
*
|
|
* \`resume\` and \`cancel\` are step-bound; \`approvalPage\` is not — it opens the job's
|
|
* approval page, which acts on whichever approval is pending when it is used.
|
|
*
|
|
* @example
|
|
* const urls = await step("urls", () => getApprovalUrls("manager"));
|
|
* await step("notify", () => sendEmail(urls.resume, urls.cancel));
|
|
* await waitForApproval({ key: "manager" });
|
|
*/
|
|
export async function getApprovalUrls(stepKey: string = "approval", approver?: string): Promise<{ approvalPage: string; resume: string; cancel: string; }>
|
|
|
|
/**
|
|
* Process items in parallel with optional concurrency control.
|
|
*
|
|
* Each item is processed by calling \`fn(item)\`, which should be a task().
|
|
* Items are dispatched in batches of \`concurrency\` (default: all at once).
|
|
*
|
|
* @example
|
|
* const process = task(async (item: string) => { ... });
|
|
* const results = await parallel(items, process, { concurrency: 5 });
|
|
*/
|
|
export async function parallel<T, R>(items: T[], fn: (item: T) => PromiseLike<R> | R, options?: { concurrency?: number },): Promise<R[]>
|
|
\`\`\`
|
|
`;
|
|
|
|
export const WAC_SDK_PYTHON = `## Python Workflow-as-Code API (wmill)
|
|
|
|
Import: \`from wmill import workflow, task, task_script, task_flow, step, sleep, wait_for_approval, get_approval_urls, get_resume_urls, parallel, TaskError\`
|
|
|
|
\`\`\`python
|
|
# Raised when a WAC \`\`task\`\` or \`\`step\`\` failed.
|
|
#
|
|
# Attributes:
|
|
# step_key: The checkpoint key of the failed step.
|
|
# child_job_id: The UUID of the failed child job, or \`\`None\`\` for a
|
|
# \`\`step()\`\`, which runs in the workflow job and has no child job.
|
|
# result: \`\`{"error": {"name", "message", "stack"?, "extra"?}}\`\` — the
|
|
# same shape whether a task or a step failed. \`\`name\`\` and \`\`message\`\`
|
|
# are always present; \`\`stack\`\` only when the failure had a traceback,
|
|
# and \`\`extra\`\` only when it carried custom fields of its own, dropped
|
|
# with \`\`extra_omitted: True\`\` beside it when too large to checkpoint.
|
|
class TaskError(Exception):
|
|
def __init__(self, message: str, *, step_key: str = '', child_job_id: Optional[str] = None, result = None)
|
|
|
|
# Get URLs needed for resuming a flow after suspension.
|
|
#
|
|
# Args:
|
|
# approver: Optional approver name
|
|
# flow_level: If True, generate resume URLs for the parent flow instead of the
|
|
# specific step. This allows pre-approvals that can be consumed by any later
|
|
# suspend step in the same flow.
|
|
#
|
|
# Returns:
|
|
# Dictionary with approvalPage, resume, and cancel URLs
|
|
def get_resume_urls(approver: str = None, flow_level: bool = None) -> dict
|
|
|
|
# Decorator that marks a function as a workflow task.
|
|
#
|
|
# Works in both WAC v1 (sync, HTTP-based dispatch) and WAC v2
|
|
# (async, checkpoint/replay) modes:
|
|
#
|
|
# - **v2 (inside @workflow)**: dispatches as a checkpoint step.
|
|
# - **v1 (WM_JOB_ID set, no @workflow)**: dispatches via HTTP API.
|
|
# - **Standalone**: executes the function body directly.
|
|
#
|
|
# A task runs as its own job, so its result is always encoded as JSON and
|
|
# decoded back before the caller sees it: a \`\`datetime\`\` comes back as a
|
|
# string, a tuple as a list.
|
|
#
|
|
# \`\`retry\`\` re-dispatches the task after a failure, inside \`\`@workflow\`\` only.
|
|
# Every attempt is a step of its own (\`\`call_api\`\`, \`\`call_api#2\`\`, ...) and
|
|
# the wait between two of them is a durable sleep, so a retrying task holds no
|
|
# worker while it backs off. Keys: \`\`attempts\`\` (retries after the first
|
|
# failure, a whole number from 0 to 100), \`\`delay\`\` (seconds before the first
|
|
# retry, sub-second delays dropped), \`\`multiplier\`\` (applied to the delay
|
|
# after each attempt, 1 keeps it constant), \`\`max_delay\`\` (ceiling in
|
|
# seconds). \`\`attempts\`\` is required, and an out-of-range or unknown key is
|
|
# rejected where the policy is written.
|
|
#
|
|
# A workflow sleeps once per round, so tasks backing off in the same fan-out
|
|
# wait one after another rather than together: the delay before a fan-out
|
|
# retries is the sum of every backoff pending in it, not the longest one, and
|
|
# it grows with both the width of the fan-out and \`\`attempts\`\`. Retries with
|
|
# no \`\`delay\`\` all go out in a single round.
|
|
#
|
|
# \`\`cache_ttl\`\` serves a previous result of the task for that many seconds
|
|
# instead of running it again. A task is keyed on its step key (its name and
|
|
# call order) and the workflow's input, not on the arguments it is called
|
|
# with, so cache one only when whether it runs, and what it receives, follow
|
|
# from the workflow's input alone. A \`\`task_script\`\` target is keyed on the
|
|
# arguments it is called with. It has no effect on a \`\`task_flow\`\` target,
|
|
# which keeps its flow's own cache policy.
|
|
#
|
|
# Usage::
|
|
#
|
|
# @task
|
|
# async def extract_data(url: str): ...
|
|
#
|
|
# @task(path="f/external_script", timeout=600, tag="gpu")
|
|
# async def run_external(x: int): ...
|
|
#
|
|
# @task(retry={"attempts": 3, "delay": 30, "multiplier": 2})
|
|
# async def call_api(payload: dict): ...
|
|
def task(_func = None, *, path: Optional[str] = None, tag: Optional[str] = None, timeout: Optional[int] = None, cache_ttl: Optional[int] = None, priority: Optional[int] = None, concurrency_limit: Optional[int] = None, concurrency_key: Optional[str] = None, concurrency_time_window_s: Optional[int] = None, retry: Optional[dict] = None)
|
|
|
|
# Create a task that dispatches to a separate Windmill script.
|
|
#
|
|
# \`\`retry\`\` takes the same policy as :func:\`task\`.
|
|
#
|
|
# Usage::
|
|
#
|
|
# extract = task_script("f/data/extract", timeout=600)
|
|
#
|
|
# @workflow
|
|
# async def main():
|
|
# data = await extract(url="https://...")
|
|
def task_script(path: str, *, timeout: Optional[int] = None, tag: Optional[str] = None, cache_ttl: Optional[int] = None, priority: Optional[int] = None, concurrency_limit: Optional[int] = None, concurrency_key: Optional[str] = None, concurrency_time_window_s: Optional[int] = None, retry: Optional[dict] = None)
|
|
|
|
# Create a task that dispatches to a separate Windmill flow.
|
|
#
|
|
# \`\`retry\`\` takes the same policy as :func:\`task\`.
|
|
#
|
|
# Usage::
|
|
#
|
|
# pipeline = task_flow("f/etl/pipeline", priority=10)
|
|
#
|
|
# @workflow
|
|
# async def main():
|
|
# result = await pipeline(input=data)
|
|
def task_flow(path: str, *, timeout: Optional[int] = None, tag: Optional[str] = None, cache_ttl: Optional[int] = None, priority: Optional[int] = None, concurrency_limit: Optional[int] = None, concurrency_key: Optional[str] = None, concurrency_time_window_s: Optional[int] = None, retry: Optional[dict] = None)
|
|
|
|
# Decorator marking an async function as a workflow-as-code entry point.
|
|
#
|
|
# The function must be **deterministic**: given the same inputs it must call
|
|
# tasks in the same order on every replay. Branching on task results is fine
|
|
# (results are replayed from checkpoint), but branching on external state
|
|
# (current time, random values, external API calls) must use \`\`step()\`\` to
|
|
# checkpoint the value so replays see the same result.
|
|
def workflow(func)
|
|
|
|
# Execute \`\`fn\`\` inline and checkpoint the result.
|
|
#
|
|
# On replay the cached value is returned without re-executing \`\`fn\`\`.
|
|
# Use for lightweight deterministic operations (timestamps, random IDs,
|
|
# config reads) that should not incur the overhead of a child job.
|
|
#
|
|
# \`\`fn\`\`'s result is encoded as JSON and decoded back before it is returned,
|
|
# so the round that runs the body sees the same types every replay sees:
|
|
# a \`\`datetime\`\` comes back as a string, a tuple as a list.
|
|
async def step(name: str, fn)
|
|
|
|
# Server-side sleep — suspend the workflow for the given duration without holding a worker.
|
|
#
|
|
# Inside a @workflow, the parent job suspends and auto-resumes after \`\`seconds\`\`.
|
|
# Outside a workflow, falls back to \`\`asyncio.sleep\`\`.
|
|
async def sleep(seconds: int)
|
|
|
|
# Suspend the workflow and wait for an external approval.
|
|
#
|
|
# Pass \`\`key\`\` to name the step, then \`\`get_approval_urls(key)\`\` yields the URLs
|
|
# that resume exactly this approval — route them through your own channel.
|
|
# Without a key the steps are named \`\`approval\`\`, \`\`approval_2\`\`, ...
|
|
#
|
|
# Returns a dict with \`\`value\`\` (form data), \`\`approver\`\`, and \`\`approved\`\`.
|
|
#
|
|
# Args:
|
|
# timeout: Approval timeout in seconds (default 1800).
|
|
# form: Optional form schema for the approval page.
|
|
# self_approval: Whether the user who triggered the flow can approve it (default True).
|
|
# key: Optional checkpoint key naming this approval step.
|
|
# skin: \`\`"minimal"\`\` shows approvers only the request (form and approve/reject)
|
|
# instead of the detailed page with the workflow's details.
|
|
# description: Shown to approvers above the form: a string, or a rich value such as
|
|
# \`\`{"markdown": "..."}\`\`.
|
|
#
|
|
# Example::
|
|
#
|
|
# urls = await step("urls", lambda: get_approval_urls("manager"))
|
|
# await step("notify", lambda: send_email(urls["resume"], urls["cancel"]))
|
|
# result = await wait_for_approval(key="manager", timeout=3600)
|
|
async def wait_for_approval(timeout: int = 1800, form: dict | None = None, self_approval: bool = True, key: str | None = None, skin: Literal['detailed', 'minimal'] | None = None, description: str | dict | None = None) -> dict
|
|
|
|
# Get the resume/cancel/approval-page URLs bound to one \`\`wait_for_approval\`\` step.
|
|
#
|
|
# Unlike :func:\`get_resume_urls\`, which signs a random nonce, these address the
|
|
# very \`\`resume_job\`\` record the step's built-in approval buttons use, so they
|
|
# are stable across replays and safe to embed in a custom notification.
|
|
#
|
|
# Args:
|
|
# step_key: Checkpoint key of the approval step, as passed to
|
|
# \`\`wait_for_approval(key=...)\`\`. Keys must be unique within a workflow;
|
|
# reusing one raises rather than silently renaming it. The URL only
|
|
# resumes while that step is awaiting approval; used at any other moment
|
|
# it is rejected rather than banking a row a different approval would
|
|
# consume. Send it ahead of time — approvers just cannot act before the
|
|
# workflow reaches the step.
|
|
# \`\`resume\`\` and \`\`cancel\`\` are step-bound; \`\`approvalPage\`\` is not — it
|
|
# opens the job's approval page, which acts on whichever approval is
|
|
# pending when it is used.
|
|
# approver: Optional approver name
|
|
#
|
|
# Returns:
|
|
# Dictionary with approvalPage, resume, and cancel URLs
|
|
def get_approval_urls(step_key: str = 'approval', approver: str = None) -> dict
|
|
|
|
# Process items in parallel with optional concurrency control.
|
|
#
|
|
# Each item is processed by calling \`\`fn(item)\`\`, which should be a @task.
|
|
# Items are dispatched in batches of \`\`concurrency\`\` (default: all at once).
|
|
#
|
|
# Example::
|
|
#
|
|
# @task
|
|
# async def process(item: str):
|
|
# ...
|
|
#
|
|
# results = await parallel(items, process, concurrency=5)
|
|
async def parallel(items, fn, *, concurrency: Optional[int] = None)
|
|
\`\`\`
|
|
`;
|
|
|
|
export const DATATABLE_SDK_TYPESCRIPT = `## TypeScript Datatable API (windmill-client)
|
|
|
|
Import: \`import * as wmill from 'windmill-client'\`
|
|
|
|
SQL statement object with query content, arguments, and execution methods
|
|
\`\`\`typescript
|
|
type SqlStatement<T> = {
|
|
/** Raw SQL content with formatted arguments */
|
|
content: string;
|
|
|
|
/** Argument values keyed by parameter name */
|
|
args: Record<string, any>;
|
|
|
|
/**
|
|
* Execute the SQL query and return results
|
|
* @param params - Optional parameters including result collection mode
|
|
* @returns Query results based on the result collection mode
|
|
*/
|
|
fetch<ResultCollectionT extends ResultCollection = "last_statement_all_rows">(
|
|
params?: FetchParams<ResultCollectionT | ResultCollection> // The union is for auto-completion
|
|
): Promise<SqlResult<T, ResultCollectionT>>;
|
|
|
|
/**
|
|
* Execute the SQL query and return only the first row
|
|
* @param params - Optional parameters
|
|
* @returns First row of the query result
|
|
*/
|
|
fetchOne(
|
|
params?: Omit<FetchParams<"last_statement_first_row">, "resultCollection">
|
|
): Promise<SqlResult<T, "last_statement_first_row">>;
|
|
|
|
/**
|
|
* Execute the SQL query and return only the first row as a scalar value
|
|
* @param params - Optional parameters
|
|
* @returns First row of the query result
|
|
*/
|
|
fetchOneScalar(
|
|
params?: Omit<
|
|
FetchParams<"last_statement_first_row_scalar">,
|
|
"resultCollection"
|
|
>
|
|
): Promise<SqlResult<T, "last_statement_first_row_scalar">>;
|
|
|
|
/**
|
|
* Execute the SQL query without fetching rows
|
|
* @param params - Optional parameters
|
|
*/
|
|
execute(
|
|
params?: Omit<FetchParams<"last_statement_first_row">, "resultCollection">
|
|
): Promise<void>;
|
|
};
|
|
\`\`\`
|
|
|
|
\`\`\`typescript
|
|
// Template tag function: sql\`SELECT * FROM table WHERE id = \${id}\`.fetch()
|
|
interface DatatableSqlTemplateFunction {
|
|
// Tagged template usage:
|
|
<T = any>(strings: TemplateStringsArray, ...values: any[]): SqlStatement<T>;
|
|
query<T = any>(sql: string, ...params: any[]): SqlStatement<T>;
|
|
};
|
|
\`\`\`
|
|
|
|
Create a SQL template function for PostgreSQL/datatable queries
|
|
@param name - Database/datatable name (default: "main")
|
|
@param opts.role - Connect as this data table role instead of the data table's default one.
|
|
Only meaningful on a data table under roles, and only for a role you are a tenant of.
|
|
@returns SQL template function for building parameterized queries
|
|
@example
|
|
let sql = wmill.datatable()
|
|
let name = 'Robin'
|
|
let age = 21
|
|
await sql\`
|
|
SELECT * FROM friends
|
|
WHERE name = \${name} AND age = \${age}::int
|
|
\`.fetch()
|
|
@example
|
|
// Read through a restricted role
|
|
let sql = wmill.datatable("main", { role: "analytics" })
|
|
\`\`\`typescript
|
|
function datatable(name: string = "main",
|
|
opts?: DatatableOptions): DatatableSqlTemplateFunction
|
|
\`\`\`
|
|
`;
|
|
|
|
export const DATATABLE_SDK_PYTHON = `## Python Datatable API (wmill)
|
|
|
|
Import: \`import wmill\`
|
|
|
|
# Get a DataTable client for SQL queries.
|
|
#
|
|
# Args:
|
|
# name: Database name (default: "main")
|
|
# role: Connect as this data table role instead of the data table's default one.
|
|
#
|
|
# Returns:
|
|
# DataTableClient instance
|
|
def datatable(name: str = 'main', *, role: Optional[str] = None) -> DataTableClient
|
|
|
|
# Client for executing SQL queries against Windmill DataTables.
|
|
class DataTableClient:
|
|
# Initialize DataTableClient.
|
|
#
|
|
# Args:
|
|
# client: Windmill client instance
|
|
# name: DataTable name
|
|
# role: Data table role to connect as, or None for the data table's default
|
|
def __init__(client: Windmill, name: str, role: Optional[str] = None)
|
|
|
|
# Execute a SQL query against the DataTable.
|
|
#
|
|
# Args:
|
|
# sql: SQL query string with $1, $2, etc. placeholders
|
|
# *args: Positional arguments to bind to query placeholders
|
|
#
|
|
# Returns:
|
|
# SqlQuery instance for fetching results
|
|
def query(sql: str, *args) -> SqlQuery
|
|
|
|
|
|
# Query result handler for DataTable and DuckLake queries.
|
|
class SqlQuery:
|
|
# Initialize SqlQuery.
|
|
#
|
|
# Args:
|
|
# sql: SQL query string
|
|
# fetch_fn: Function to execute the query
|
|
def __init__(sql: str, fetch_fn)
|
|
|
|
# Execute query and fetch results.
|
|
#
|
|
# Args:
|
|
# result_collection: Optional result collection mode
|
|
#
|
|
# Returns:
|
|
# Query results
|
|
def fetch(result_collection: str | None = None)
|
|
|
|
# Execute query and fetch first row of results.
|
|
#
|
|
# Returns:
|
|
# First row of query results
|
|
def fetch_one()
|
|
|
|
# Execute query and fetch first row of results. Return result as a scalar value.
|
|
#
|
|
# Returns:
|
|
# First row of query result as a scalar value
|
|
def fetch_one_scalar()
|
|
|
|
# Execute query and don't return any results.
|
|
#
|
|
def execute()
|
|
|
|
|
|
`;
|
|
|
|
export const OPENFLOW_SCHEMA = `## OpenFlow Schema
|
|
|
|
{"OpenFlow":{"type":"object","description":"Top-level flow definition containing metadata, configuration, and the flow structure","properties":{"summary":{"type":"string","description":"Short description of what this flow does"},"description":{"type":"string","description":"Detailed documentation for this flow"},"value":{"$ref":"#/components/schemas/FlowValue"},"schema":{"type":"object","description":"JSON Schema for flow inputs. Use this to define input parameters, their types, defaults, and validation. For resource inputs, set type to 'object' and format to 'resource-<type>' (e.g., 'resource-stripe')"},"on_behalf_of_email":{"type":"string","description":"Address of the account the flow runs on behalf of. Derived from on_behalf_of on read; accepted on write, where it is resolved to the account it names."},"on_behalf_of":{"type":"string","description":"The flow runs with the permissions of this identity: u/{username}, g/{group}, or a bare email when the username is itself email-shaped. The only stored half of the identity; on_behalf_of_email is derived from it. Omit it when writing and it is resolved from that address instead."}},"required":["summary","value"]},"FlowValue":{"type":"object","description":"The flow structure containing modules and optional preprocessor/failure handlers","properties":{"modules":{"type":"array","description":"Array of steps that execute in sequence. Each step can be a script, subflow, loop, or branch","items":{"$ref":"#/components/schemas/FlowModule"}},"failure_module":{"description":"Special module that executes when the flow fails. Receives error object with message, name, stack, and step_id. Must have id 'failure'. Only supports script/rawscript types","$ref":"#/components/schemas/FlowModule"},"preprocessor_module":{"description":"Special module that runs before the first step on external triggers. Must have id 'preprocessor'. Only supports script/rawscript types. Cannot reference other step results","$ref":"#/components/schemas/FlowModule"},"same_worker":{"type":"boolean","description":"If true, all steps run on the same worker for better performance"},"preserve_step_tags":{"type":"boolean","description":"If true and the flow runs on a custom worker tag, steps that declare their own non-empty tag run on it instead of inheriting the flow tag. Steps without their own tag still inherit the flow tag."},"concurrent_limit":{"type":"number","description":"Maximum number of concurrent executions of this flow"},"concurrency_key":{"type":"string","description":"Expression to group concurrent executions (e.g., by user ID)"},"concurrency_time_window_s":{"type":"number","description":"Time window in seconds for concurrent_limit"},"debounce_delay_s":{"type":"integer","description":"Delay in seconds to debounce flow executions"},"debounce_key":{"type":"string","description":"Expression to group debounced executions"},"debounce_args_to_accumulate":{"type":"array","description":"Arguments to accumulate across debounced executions","items":{"type":"string"}},"max_total_debouncing_time":{"type":"integer","description":"Maximum total time in seconds that a job can be debounced"},"max_total_debounces_amount":{"type":"integer","description":"Maximum number of times a job can be debounced"},"skip_expr":{"type":"string","description":"JavaScript expression to conditionally skip the entire flow"},"cache_ttl":{"type":"number","description":"Cache duration in seconds for flow results"},"cache_ignore_s3_path":{"type":"boolean"},"delete_after_secs":{"type":"integer","description":"If set, delete the flow job's args, result and logs after this many seconds following job completion"},"flow_env":{"type":"object","description":"Environment variables available to all steps. Values can be strings, JSON values, or special references: '$var:path' (workspace variable) or '$res:path' (resource).","additionalProperties":{}},"priority":{"type":"number","description":"Execution priority (higher numbers run first)"},"early_return":{"type":"string","description":"JavaScript expression to return early from the flow"},"chat_input_enabled":{"type":"boolean","description":"Whether this flow accepts chat-style input"},"notes":{"type":"array","description":"Sticky notes attached to the flow","items":{"$ref":"#/components/schemas/FlowNote"}},"groups":{"type":"array","description":"Semantic groups of modules for organizational purposes","items":{"$ref":"#/components/schemas/FlowGroup"}}},"required":["modules"]},"Retry":{"type":"object","description":"Retry configuration for failed module executions","properties":{"constant":{"type":"object","description":"Retry with constant delay between attempts","properties":{"attempts":{"type":"integer","description":"Number of retry attempts"},"seconds":{"type":"integer","description":"Seconds to wait between retries"}}},"exponential":{"type":"object","description":"Retry with exponential backoff (delay doubles each time)","properties":{"attempts":{"type":"integer","description":"Number of retry attempts"},"multiplier":{"type":"integer","description":"Multiplier for exponential backoff"},"seconds":{"type":"integer","minimum":1,"description":"Initial delay in seconds"},"random_factor":{"type":"integer","minimum":0,"maximum":100,"description":"Random jitter percentage (0-100) to avoid thundering herd"}}},"retry_if":{"$ref":"#/components/schemas/RetryIf"}}},"FlowNote":{"type":"object","description":"A sticky note attached to a flow for documentation and annotation","properties":{"id":{"type":"string","description":"Unique identifier for the note"},"text":{"type":"string","description":"Content of the note"},"position":{"type":"object","description":"Position of the note in the flow editor","properties":{"x":{"type":"number","description":"X coordinate"},"y":{"type":"number","description":"Y coordinate"}},"required":["x","y"]},"size":{"type":"object","description":"Size of the note in the flow editor","properties":{"width":{"type":"number","description":"Width in pixels"},"height":{"type":"number","description":"Height in pixels"}},"required":["width","height"]},"color":{"type":"string","description":"Color of the note (e.g., \\"yellow\\", \\"#ffff00\\")"},"type":{"type":"string","enum":["free","group"],"description":"Type of note - 'free' for standalone notes, 'group' for notes that group other nodes"},"locked":{"type":"boolean","default":false,"description":"Whether the note is locked and cannot be edited or moved"},"contained_node_ids":{"type":"array","items":{"type":"string"},"description":"For group notes, the IDs of nodes contained within this group"}},"required":["id","text","color","type"]},"FlowGroup":{"type":"object","description":"A semantic group of flow modules for organizational purposes. Does not affect execution \\u2014 modules remain in their original position in the flow. Groups provide naming and collapsibility in the editor. Members are computed dynamically from all nodes on paths between start_id and end_id.","properties":{"summary":{"type":"string","description":"Display name for this group"},"note":{"type":"string","description":"Markdown note shown below the group header"},"autocollapse":{"type":"boolean","default":false,"description":"If true, this group is collapsed by default in the flow editor. UI hint only."},"start_id":{"type":"string","description":"ID of the first flow module in this group (topological entry point)"},"end_id":{"type":"string","description":"ID of the last flow module in this group (topological exit point)"},"color":{"type":"string","description":"Color for the group in the flow editor"}},"required":["start_id","end_id"]},"RetryIf":{"type":"object","description":"Conditional retry based on error or result","properties":{"expr":{"type":"string","description":"JavaScript expression that returns true to retry. Has access to 'result' and 'error' variables"}},"required":["expr"]},"StopAfterIf":{"type":"object","description":"Early termination condition for a module","properties":{"skip_if_stopped":{"type":"boolean","description":"If true, following steps are skipped when this condition triggers"},"expr":{"type":"string","description":"JavaScript expression evaluated after the module runs. Can use 'result' (step's result) or 'flow_input'. Return true to stop"},"error_message":{"type":"string","nullable":true,"description":"Custom error message when stopping with an error. Mutually exclusive with skip_if_stopped. If set to a non-empty string, the flow stops with this error. If empty string, a default error message is used. If null or omitted, no error is raised."},"error_include_result":{"type":"boolean","description":"When stopping with an error (error_message set), embed the stopping step's own result inside the raised error object (as error.result) instead of discarding it. The top-level result stays { error }. Defaults to false."}},"required":["expr"]},"FlowModule":{"type":"object","description":"A single step in a flow. Can be a script, subflow, loop, or branch","properties":{"id":{"type":"string","description":"Unique identifier for this step. Used to reference results via 'results.step_id'. Must be a valid identifier (alphanumeric, underscore, hyphen)"},"value":{"$ref":"#/components/schemas/FlowModuleValue"},"stop_after_if":{"description":"Early termination condition evaluated after this step completes","$ref":"#/components/schemas/StopAfterIf"},"stop_after_all_iters_if":{"description":"For loops only - early termination condition evaluated after all iterations complete","$ref":"#/components/schemas/StopAfterIf"},"skip_if":{"type":"object","description":"Conditionally skip this step based on previous results or flow inputs","properties":{"expr":{"type":"string","description":"JavaScript expression that returns true to skip. Can use 'flow_input' or 'results.<step_id>'"}},"required":["expr"]},"sleep":{"description":"Delay before executing this step (in seconds or as expression)","$ref":"#/components/schemas/InputTransform"},"cache_ttl":{"type":"number","description":"Cache duration in seconds for this step's results"},"cache_ignore_s3_path":{"type":"boolean"},"timeout":{"description":"Maximum execution time in seconds (static value or expression)","$ref":"#/components/schemas/InputTransform"},"delete_after_secs":{"type":"integer","description":"If set, delete the step's args, result and logs after this many seconds following job completion"},"summary":{"type":"string","description":"Short description of what this step does"},"mock":{"type":"object","description":"Mock configuration for testing without executing the actual step","properties":{"enabled":{"type":"boolean","description":"If true, return mock value instead of executing"},"return_value":{"description":"Value to return when mocked"}}},"suspend":{"type":"object","description":"Configuration for approval/resume steps that wait for user input","properties":{"required_events":{"type":"integer","description":"Number of approvals required before continuing"},"timeout":{"type":"integer","description":"Timeout in seconds before auto-continuing or canceling"},"resume_form":{"type":"object","description":"Form schema for collecting input when resuming","properties":{"schema":{"type":"object","description":"JSON Schema for the resume form"}}},"user_auth_required":{"type":"boolean","description":"If true, only authenticated users can approve"},"user_groups_required":{"description":"Expression or list of groups that can approve","$ref":"#/components/schemas/InputTransform"},"self_approval_disabled":{"type":"boolean","description":"If true, the user who started the flow cannot approve"},"hide_cancel":{"type":"boolean","description":"If true, hide the cancel button on the approval form"},"continue_on_disapprove_timeout":{"type":"boolean","description":"If true, continue flow on timeout instead of canceling"},"skin":{"type":"string","enum":["detailed","minimal"],"description":"How the approval request is presented, on the approval page and in Slack/Teams approval messages. 'detailed' (used when unset) shows the flow details (arguments, graph, approvers); 'minimal' shows only the request: the step description, form and approve/reject actions"}}},"priority":{"type":"number","description":"Execution priority for this step (higher numbers run first)"},"continue_on_error":{"type":"boolean","description":"If true, flow continues even if this step fails"},"retry":{"description":"Retry configuration if this step fails","$ref":"#/components/schemas/Retry"},"debouncing":{"description":"Debounce configuration for this step (EE only)","type":"object","properties":{"debounce_delay_s":{"type":"integer","description":"Delay in seconds to debounce this step's executions across flow runs"},"debounce_key":{"type":"string","description":"Expression to group debounced executions. Supports $workspace and $args[name]. Default: $workspace/flow/<flow_path>-<step_id>"},"debounce_args_to_accumulate":{"type":"array","description":"Array-type arguments to accumulate across debounced executions","items":{"type":"string"}},"max_total_debouncing_time":{"type":"integer","description":"Maximum total time in seconds before forced execution"},"max_total_debounces_amount":{"type":"integer","description":"Maximum number of debounces before forced execution"}}}},"required":["value","id"]},"InputTransform":{"description":"Maps input parameters for a step. Can be a static value or a JavaScript expression that references previous results or flow inputs","oneOf":[{"$ref":"#/components/schemas/StaticTransform"},{"$ref":"#/components/schemas/JavascriptTransform"},{"$ref":"#/components/schemas/AiTransform"}],"discriminator":{"propertyName":"type","mapping":{"static":"#/components/schemas/StaticTransform","javascript":"#/components/schemas/JavascriptTransform","ai":"#/components/schemas/AiTransform"}}},"StaticTransform":{"type":"object","description":"Static value passed directly to the step. Use for hardcoded values or resource references like '$res:path/to/resource'","properties":{"value":{"description":"The static value. For resources, use format '$res:path/to/resource'"},"type":{"type":"string","enum":["static"]}},"required":["type"]},"JavascriptTransform":{"type":"object","description":"JavaScript expression evaluated at runtime. Can reference previous step results via 'results.step_id' or flow inputs via 'flow_input.property'. Inside for loops, use 'flow_input.iter.value' for the current iteration value (in while loops it equals 'flow_input.iter.index')","properties":{"expr":{"type":"string","description":"JavaScript expression returning the value. Available variables - results (object with all previous step results), flow_input (flow inputs), flow_input.iter (in loops)"},"type":{"type":"string","enum":["javascript"]}},"required":["expr","type"]},"AiTransform":{"type":"object","description":"Value resolved by the AI runtime for this input. The AI engine decides how to satisfy the parameter.","properties":{"type":{"type":"string","enum":["ai"]}},"required":["type"]},"AIProviderKind":{"type":"string","description":"Supported AI provider types","enum":["openai","azure_openai","azure_foundry","anthropic","mistral","deepseek","googleai","groq","openrouter","togetherai","customai","aws_bedrock"]},"ProviderConfig":{"type":"object","description":"Complete AI provider configuration with resource reference and model selection","properties":{"kind":{"$ref":"#/components/schemas/AIProviderKind"},"resource":{"type":"string","description":"Resource reference in format '$res:{resource_path}' pointing to provider credentials"},"model":{"type":"string","description":"Model identifier (e.g., 'gpt-4', 'claude-3-opus-20240229', 'gemini-pro')"},"reasoning_effort":{"type":"string","description":"Provider-native reasoning effort token (e.g. 'low', 'high', 'none') for models that support extended thinking. Optional; unset leaves the provider default."}},"required":["kind","resource","model"]},"StaticProviderTransform":{"type":"object","description":"Static provider configuration passed directly to the AI agent","properties":{"value":{"$ref":"#/components/schemas/ProviderConfig"},"type":{"type":"string","enum":["static"]}},"required":["type","value"]},"ProviderTransform":{"description":"Provider configuration - can be static (ProviderConfig), JavaScript expression, or AI-determined","oneOf":[{"$ref":"#/components/schemas/StaticProviderTransform"},{"$ref":"#/components/schemas/JavascriptTransform"},{"$ref":"#/components/schemas/AiTransform"}],"discriminator":{"propertyName":"type","mapping":{"static":"#/components/schemas/StaticProviderTransform","javascript":"#/components/schemas/JavascriptTransform","ai":"#/components/schemas/AiTransform"}}},"MemoryOff":{"type":"object","description":"No conversation memory/context","properties":{"kind":{"type":"string","enum":["off"]}},"required":["kind"]},"MemoryWindow":{"type":"object","description":"Keeps the most recent messages of the memory named by the run's memory id (or the step's\\n\`memory_id\`). Without a memory id the agent runs without memory.\\n","properties":{"kind":{"type":"string","enum":["window"]},"context_length":{"type":"integer","description":"Number of most recent messages to load and store. 0 turns memory off."}},"required":["kind","context_length"]},"MemoryAuto":{"type":"object","deprecated":true,"description":"Deprecated, still read as it was written: the run's memory id, else the \`memory_id\` here.\\nThe step's own \`memory_id\` is not read while this kind is set; switch the kind to \`window\`\\nto use it. Without a \`context_length\`, or with 0, it is \`off\` and reads \`previous_messages\`.\\n","properties":{"kind":{"type":"string","enum":["auto"]},"context_length":{"type":"integer","description":"Maximum number of messages to retain in context"},"memory_id":{"type":"string","description":"Identifier for persistent memory across agent invocations"}},"required":["kind"]},"MemoryMessage":{"type":"object","description":"A single message in conversation history","properties":{"role":{"type":"string","enum":["user","assistant","system"]},"content":{"type":"string"}},"required":["role","content"]},"MemoryManual":{"type":"object","deprecated":true,"description":"Deprecated, still read as it was written. Move the step to \`off\` with \`previous_messages\` instead.","properties":{"kind":{"type":"string","enum":["manual"]},"messages":{"type":"array","items":{"$ref":"#/components/schemas/MemoryMessage"}}},"required":["kind","messages"]},"MemoryConfig":{"description":"Managed memory, stored by Windmill and replayed with each request. The memory is named by a memory id, see \`memory_id\`. While it is off, a step can supply its history in \`previous_messages\`.","oneOf":[{"$ref":"#/components/schemas/MemoryOff"},{"$ref":"#/components/schemas/MemoryWindow"},{"$ref":"#/components/schemas/MemoryAuto"},{"$ref":"#/components/schemas/MemoryManual"}],"discriminator":{"propertyName":"kind","mapping":{"off":"#/components/schemas/MemoryOff","window":"#/components/schemas/MemoryWindow","auto":"#/components/schemas/MemoryAuto","manual":"#/components/schemas/MemoryManual"}}},"StaticMemoryTransform":{"type":"object","description":"Static memory configuration passed directly to the AI agent","properties":{"value":{"$ref":"#/components/schemas/MemoryConfig"},"type":{"type":"string","enum":["static"]}},"required":["type","value"]},"MemoryTransform":{"description":"Memory configuration - can be static (MemoryConfig), JavaScript expression, or AI-determined","oneOf":[{"$ref":"#/components/schemas/StaticMemoryTransform"},{"$ref":"#/components/schemas/JavascriptTransform"},{"$ref":"#/components/schemas/AiTransform"}],"discriminator":{"propertyName":"type","mapping":{"static":"#/components/schemas/StaticMemoryTransform","javascript":"#/components/schemas/JavascriptTransform","ai":"#/components/schemas/AiTransform"}}},"FlowModuleValue":{"description":"The actual implementation of a flow step. Can be a script (inline or referenced), subflow, loop, branch, or special module type","oneOf":[{"$ref":"#/components/schemas/RawScript"},{"$ref":"#/components/schemas/PathScript"},{"$ref":"#/components/schemas/PathFlow"},{"$ref":"#/components/schemas/ForloopFlow"},{"$ref":"#/components/schemas/WhileloopFlow"},{"$ref":"#/components/schemas/BranchOne"},{"$ref":"#/components/schemas/BranchAll"},{"$ref":"#/components/schemas/Identity"},{"$ref":"#/components/schemas/AiAgent"}],"discriminator":{"propertyName":"type","mapping":{"rawscript":"#/components/schemas/RawScript","script":"#/components/schemas/PathScript","flow":"#/components/schemas/PathFlow","forloopflow":"#/components/schemas/ForloopFlow","whileloopflow":"#/components/schemas/WhileloopFlow","branchone":"#/components/schemas/BranchOne","branchall":"#/components/schemas/BranchAll","identity":"#/components/schemas/Identity","aiagent":"#/components/schemas/AiAgent"}}},"RawScript":{"type":"object","description":"Inline script with code defined directly in the flow. Use 'bun' as default language if unspecified. The script receives arguments from input_transforms","properties":{"input_transforms":{"type":"object","description":"Map of parameter names to their values (static or JavaScript expressions). These become the script's input arguments","additionalProperties":{"$ref":"#/components/schemas/InputTransform"}},"content":{"type":"string","description":"The script source code. Should export a 'main' function"},"language":{"type":"string","description":"Programming language for this script","enum":["deno","bun","bunnative","python3","go","bash","powershell","postgresql","mysql","bigquery","snowflake","mssql","oracledb","graphql","nativets","php","rust","ansible","csharp","nu","java","ruby","rlang","duckdb"]},"path":{"type":"string","description":"Optional path for saving this script"},"lock":{"type":"string","description":"Lock file content for dependencies"},"type":{"type":"string","enum":["rawscript"]},"tag":{"type":"string","description":"Worker group tag for execution routing"},"concurrent_limit":{"type":"number","description":"Maximum concurrent executions of this script"},"concurrency_time_window_s":{"type":"number","description":"Time window for concurrent_limit"},"custom_concurrency_key":{"type":"string","description":"Custom key for grouping concurrent executions"},"is_trigger":{"type":"boolean","description":"If true, this script is a trigger that can start the flow"},"assets":{"type":"array","description":"External resources this script accesses (S3 objects, resources, etc.)","items":{"type":"object","required":["path","kind"],"properties":{"path":{"type":"string","description":"Path to the asset"},"kind":{"type":"string","description":"Type of asset","enum":["s3object","resource","ducklake","datatable","volume","dbt"]},"access_type":{"type":"string","nullable":true,"description":"Access level for this asset","enum":["r","w","rw"]},"alt_access_type":{"type":"string","nullable":true,"description":"Alternative access level","enum":["r","w","rw"]}}}}},"required":["type","content","language","input_transforms"]},"PathScript":{"type":"object","description":"Reference to an existing script by path. Use this when calling a previously saved script instead of writing inline code","properties":{"input_transforms":{"type":"object","description":"Map of parameter names to their values (static or JavaScript expressions). These become the script's input arguments","additionalProperties":{"$ref":"#/components/schemas/InputTransform"}},"path":{"type":"string","description":"Path to the script in the workspace (e.g., 'f/scripts/send_email')"},"hash":{"type":"string","description":"Optional specific version hash of the script to use"},"type":{"type":"string","enum":["script"]},"tag_override":{"type":"string","description":"Override the script's default worker group tag"},"is_trigger":{"type":"boolean","description":"If true, this script is a trigger that can start the flow"}},"required":["type","path","input_transforms"]},"PathFlow":{"type":"object","description":"Reference to an existing flow by path. Use this to call another flow as a subflow","properties":{"input_transforms":{"type":"object","description":"Map of parameter names to their values (static or JavaScript expressions). These become the subflow's input arguments","additionalProperties":{"$ref":"#/components/schemas/InputTransform"}},"path":{"type":"string","description":"Path to the flow in the workspace (e.g., 'f/flows/process_user')"},"type":{"type":"string","enum":["flow"]}},"required":["type","path","input_transforms"]},"ForloopFlow":{"type":"object","description":"Executes nested modules in a loop over an iterator. Inside the loop, use 'flow_input.iter.value' to access the current iteration value, and 'flow_input.iter.index' for the index. Supports parallel execution for better performance on I/O-bound operations","properties":{"modules":{"type":"array","description":"Steps to execute for each iteration. These can reference the iteration value via 'flow_input.iter.value'","items":{"$ref":"#/components/schemas/FlowModule"}},"iterator":{"description":"JavaScript expression that returns an array to iterate over. Can reference 'results.step_id' or 'flow_input'","$ref":"#/components/schemas/InputTransform"},"skip_failures":{"type":"boolean","description":"If true, iteration failures don't stop the loop. Failed iterations return null"},"type":{"type":"string","enum":["forloopflow"]},"parallel":{"type":"boolean","description":"If true, iterations run concurrently (faster for I/O-bound operations). Use with parallelism to control concurrency"},"parallelism":{"description":"Maximum number of concurrent iterations when parallel=true. Limits resource usage. Can be static number or expression","$ref":"#/components/schemas/InputTransform"},"squash":{"type":"boolean"}},"required":["modules","iterator","skip_failures","type"]},"WhileloopFlow":{"type":"object","description":"Executes nested modules repeatedly until stopped. The implicit iterator is the iteration counter, so 'flow_input.iter.value' equals 'flow_input.iter.index' (0, 1, 2, ...) and never carries state. To carry state across iterations, a step reads its own previous-iteration result via 'results.<its_own_id>' with a first-iteration fallback - the loop's stop_after_if must then be on that inner step (a plain single-step body with stop_after_if on the loop module does not resolve 'results' across iterations and never terminates); plain counters can instead be derived from 'flow_input.iter.index', which works in every configuration. stop_after_if is evaluated after each iteration - on the loop module 'result' is the last iteration's result","properties":{"modules":{"type":"array","description":"Steps to execute in each iteration","items":{"$ref":"#/components/schemas/FlowModule"}},"skip_failures":{"type":"boolean","description":"If true, iteration failures don't stop the loop. Failed iterations return null"},"type":{"type":"string","enum":["whileloopflow"]},"parallel":{"type":"boolean","description":"If true, iterations run concurrently (use with caution in while loops)"},"parallelism":{"description":"Maximum number of concurrent iterations when parallel=true","$ref":"#/components/schemas/InputTransform"},"squash":{"type":"boolean"}},"required":["modules","skip_failures","type"]},"BranchOne":{"type":"object","description":"Conditional branching where only the first matching branch executes. Branches are evaluated in order, and the first one with a true expression runs. If no branches match, the default branch executes","properties":{"branches":{"type":"array","description":"Array of branches to evaluate in order. The first branch with expr evaluating to true executes","items":{"type":"object","properties":{"summary":{"type":"string","description":"Short description of this branch condition"},"expr":{"type":"string","description":"JavaScript expression that returns boolean. Can use 'results.step_id' or 'flow_input'. First true expr wins"},"modules":{"type":"array","description":"Steps to execute if this branch's expr is true","items":{"$ref":"#/components/schemas/FlowModule"}}},"required":["modules","expr"]}},"default":{"type":"array","description":"Steps to execute if no branch expressions match","items":{"$ref":"#/components/schemas/FlowModule"}},"type":{"type":"string","enum":["branchone"]}},"required":["branches","default","type"]},"BranchAll":{"type":"object","description":"Parallel branching where all branches execute simultaneously. Unlike BranchOne, all branches run regardless of conditions. Useful for executing independent tasks concurrently","properties":{"branches":{"type":"array","description":"Array of branches that all execute (either in parallel or sequentially)","items":{"type":"object","properties":{"summary":{"type":"string","description":"Short description of this branch's purpose"},"skip_failure":{"type":"boolean","description":"If true, failure in this branch doesn't fail the entire flow"},"modules":{"type":"array","description":"Steps to execute in this branch","items":{"$ref":"#/components/schemas/FlowModule"}}},"required":["modules"]}},"type":{"type":"string","enum":["branchall"]},"parallel":{"type":"boolean","description":"If true, all branches execute concurrently. If false, they execute sequentially"}},"required":["branches","type"]},"AgentTool":{"type":"object","description":"A tool available to an AI agent. Can be a flow module or an external MCP (Model Context Protocol) tool","properties":{"id":{"type":"string","description":"Unique identifier for this tool. Cannot contain spaces - use underscores instead (e.g., 'get_user_data' not 'get user data')"},"summary":{"type":"string","description":"The name the AI agent calls this tool by, not a human label. On a flowmodule tool it must match ^[a-zA-Z0-9_]+$ - letters, numbers and underscores only (e.g. 'search_documentation', not 'Search documentation') - and always be set; on an mcp or websearch tool it is a plain label. Put the human-readable explanation in 'description'."},"description":{"type":"string","description":"Free-text description of the tool given to the AI to decide when and how to call it. Overrides the description auto-derived from the underlying script."},"value":{"$ref":"#/components/schemas/ToolValue"}},"required":["id","value"]},"ToolValue":{"description":"The implementation of a tool. Can be a flow module (script/flow) or an MCP tool reference","oneOf":[{"$ref":"#/components/schemas/FlowModuleTool"},{"$ref":"#/components/schemas/McpToolValue"},{"$ref":"#/components/schemas/WebsearchToolValue"}],"discriminator":{"propertyName":"tool_type","mapping":{"flowmodule":"#/components/schemas/FlowModuleTool","mcp":"#/components/schemas/McpToolValue","websearch":"#/components/schemas/WebsearchToolValue"}}},"FlowModuleTool":{"description":"A tool implemented as a flow module (script, flow, etc.). The AI can call this like any other flow module","allOf":[{"type":"object","properties":{"tool_type":{"type":"string","enum":["flowmodule"]}},"required":["tool_type"]},{"$ref":"#/components/schemas/FlowModuleValue"}]},"WebsearchToolValue":{"type":"object","description":"A tool implemented as a websearch tool. The AI can call this like any other websearch tool","properties":{"tool_type":{"type":"string","enum":["websearch"]}},"required":["tool_type"]},"McpToolValue":{"type":"object","description":"Reference to an external MCP (Model Context Protocol) tool. The AI can call tools from MCP servers","properties":{"tool_type":{"type":"string","enum":["mcp"]},"resource_path":{"type":"string","description":"Path to the MCP resource/server configuration"},"include_tools":{"type":"array","description":"Whitelist of specific tools to include from this MCP server","items":{"type":"string"}},"exclude_tools":{"type":"array","description":"Blacklist of tools to exclude from this MCP server","items":{"type":"string"}}},"required":["tool_type","resource_path"]},"AiAgent":{"type":"object","description":"AI agent step that can use tools to accomplish tasks. The agent receives inputs and can call any of its configured tools to complete the task","properties":{"input_transforms":{"type":"object","description":"Input parameters for the AI agent mapped to their values","properties":{"provider":{"$ref":"#/components/schemas/ProviderTransform"},"output_type":{"allOf":[{"$ref":"#/components/schemas/InputTransform"}],"description":"Output format type.\\nValid values: 'text' (default) - plain text response, 'image' - image generation\\n"},"user_message":{"allOf":[{"$ref":"#/components/schemas/InputTransform"}],"description":"The user's prompt/message to the AI agent. Supports variable interpolation with\\nflow.input syntax. Required unless memory is off and \`previous_messages\` supplies\\nthe prompt; image output always needs it.\\n"},"system_prompt":{"allOf":[{"$ref":"#/components/schemas/InputTransform"}],"description":"System instructions that guide the AI's behavior, persona, and response style. Optional."},"streaming":{"allOf":[{"$ref":"#/components/schemas/InputTransform"}],"description":"Boolean. If true, stream the AI response incrementally.\\nStreaming events include: token_delta, reasoning_token_delta, tool_call, tool_call_arguments, tool_execution, tool_result\\n"},"memory":{"$ref":"#/components/schemas/MemoryTransform"},"memory_id":{"allOf":[{"$ref":"#/components/schemas/InputTransform"}],"description":"String. Names the memory this step reads and writes, overriding the memory id the run\\nwas started with (the chat conversation, an app chat session or the \`memory_id\` run\\nparameter). Leave unset to use the run's memory id. A fixed value shares one memory\\nacross every run; an expression such as \`flow_input.customer_id\` keeps one memory per\\nkey. When it evaluates to an empty value the agent runs without memory. Read only\\nwhile \`memory\` is \`window\`: it is ignored when memory is off, and an older \`auto\` or\\n\`manual\` memory reads neither history input.\\n"},"previous_messages":{"allOf":[{"$ref":"#/components/schemas/InputTransform"}],"description":"Array of MemoryMessage. History supplied by the flow, sent between the system prompt\\nand the user message. Read only while \`memory\` is off or absent: managed memory\\nignores it, and an older \`auto\` or \`manual\` memory reads neither history input.\\n"},"output_schema":{"allOf":[{"$ref":"#/components/schemas/InputTransform"}],"description":"JSON Schema object defining structured output format. Used when you need the AI to return data in a specific shape.\\nSupports standard JSON Schema properties: type, properties, required, items, enum, pattern, minLength, maxLength, minimum, maximum, etc.\\nExample: { type: 'object', properties: { name: { type: 'string' }, age: { type: 'integer' } }, required: ['name'] }\\n"},"user_attachments":{"allOf":[{"$ref":"#/components/schemas/InputTransform"}],"description":"Array of file references (images or PDFs) for the AI agent.\\nFormat: Array<{ bucket: string, key: string }> - S3 object references\\nExample: [{ bucket: 'my-bucket', key: 'documents/report.pdf' }]\\n"},"enabled_tools":{"allOf":[{"$ref":"#/components/schemas/InputTransform"}],"description":"Array of strings naming which of the tools configured in \`tools\` the agent may call\\nthis run. Leaving it unset carries every one of them; an empty array carries none.\\nA tool is named as the model is shown it. An entry the model is shown nothing of is\\nnamed by what identifies it instead: an MCP server by its resource path, carrying\\nevery tool it exposes (which of them stays that entry's include_tools/exclude_tools),\\nand a websearch entry by the reserved name '__wm_web_search', whatever summary it carries\\n(no tool may take that name).\\nExample: ['get_user', 'u/admin/github_mcp', '__wm_web_search']\\n"},"max_completion_tokens":{"allOf":[{"$ref":"#/components/schemas/InputTransform"}],"description":"Integer. Maximum number of tokens the AI will generate in its response.\\nRange: 1 to 4,294,967,295. Typical values: 256-4096 for most use cases.\\n"},"temperature":{"allOf":[{"$ref":"#/components/schemas/InputTransform"}],"description":"Float. Controls randomness/creativity of responses.\\nRange: 0.0 to 2.0 (provider-dependent)\\n- 0.0 = deterministic, focused responses\\n- 0.7 = balanced (common default)\\n- 1.0+ = more creative/random\\n"},"max_iterations":{"allOf":[{"$ref":"#/components/schemas/InputTransform"}],"description":"Number. Limits how many times the agent can loop through reasoning and tool use.\\nRange: 1-1000.\\n"}}},"tools":{"type":"array","description":"Array of tools the agent can use. The agent decides which tools to call based on the task","items":{"$ref":"#/components/schemas/AgentTool"}},"type":{"type":"string","enum":["aiagent"]},"tag":{"type":"string","description":"Worker group tag for execution routing. If not set, the AI agent step runs on the flow's tag (default \`flow\`)"},"omit_output_from_conversation":{"type":"boolean","default":false,"description":"If true, this AI agent step does not persist its assistant or tool messages to the flow conversation when chat mode is enabled."},"agent":{"type":"string","description":"Path of a reusable \`ai_agent\` resource (hybrid linking). When set, the agent brain\\nconfig (provider/model/system prompt/etc.) and tool set are resolved at runtime from\\nthat resource; the module's input_transforms then only carry the flow-local inputs\\n(user_message, user_attachments, enabled_tools and the history inputs memory_id and previous_messages).\\n"},"tool_inputs":{"type":"object","description":"Host-local wiring for an agent's tool inputs, keyed by tool id then input key. Binds the\\nreferenced agent's tools to this flow's context (flow_input/results) without mutating the\\nshared resource; overlaid onto the tools' input_transforms at runtime \\u2014 including when\\n\`agent\` is unset, since a step forked for editing keeps these overrides until it is saved\\nback or unlinked.\\n","additionalProperties":{"type":"object","additionalProperties":{"$ref":"#/components/schemas/InputTransform"}}},"parallel":{"type":"boolean","description":"If true, the agent can execute multiple tool calls in parallel"}},"required":["type","input_transforms"]},"Identity":{"type":"object","description":"Pass-through module that returns its input unchanged. Useful for flow structure or as a placeholder","properties":{"type":{"type":"string","enum":["identity"]},"flow":{"type":"boolean","description":"If true, marks this as a flow identity (special handling)"}},"required":["type"]},"FlowStatus":{"type":"object","properties":{"step":{"type":"integer"},"modules":{"type":"array","items":{"$ref":"#/components/schemas/FlowStatusModule"}},"user_states":{"additionalProperties":true},"preprocessor_module":{"allOf":[{"$ref":"#/components/schemas/FlowStatusModule"}]},"failure_module":{"allOf":[{"$ref":"#/components/schemas/FlowStatusModule"},{"type":"object","properties":{"parent_module":{"type":"string"}}}]},"retry":{"type":"object","properties":{"fail_count":{"type":"integer"},"failed_jobs":{"type":"array","items":{"type":"string","format":"uuid"}}}}},"required":["step","modules","failure_module"]},"FlowStatusModule":{"type":"object","properties":{"type":{"type":"string","enum":["WaitingForPriorSteps","WaitingForEvents","WaitingForExecutor","InProgress","Success","Failure"]},"id":{"type":"string"},"job":{"type":"string","format":"uuid"},"count":{"type":"integer"},"progress":{"type":"integer"},"iterator":{"type":"object","properties":{"index":{"type":"integer"},"itered":{"type":"array","items":{}},"itered_len":{"type":"integer"},"args":{}}},"flow_jobs":{"type":"array","items":{"type":"string"}},"flow_jobs_success":{"type":"array","items":{"type":"boolean"}},"flow_jobs_duration":{"type":"object","properties":{"started_at":{"type":"array","items":{"type":"string"}},"duration_ms":{"type":"array","items":{"type":"integer"}}}},"branch_chosen":{"type":"object","properties":{"type":{"type":"string","enum":["branch","default"]},"branch":{"type":"integer"}},"required":["type"]},"branchall":{"type":"object","properties":{"branch":{"type":"integer"},"len":{"type":"integer"}},"required":["branch","len"]},"approvers":{"type":"array","items":{"type":"object","properties":{"resume_id":{"type":"integer"},"approver":{"type":"string"}},"required":["resume_id","approver"]}},"failed_retries":{"type":"array","items":{"type":"string","format":"uuid"}},"skipped":{"type":"boolean"},"agent_actions":{"type":"array","items":{"type":"object","oneOf":[{"type":"object","properties":{"job_id":{"type":"string","format":"uuid"},"function_name":{"type":"string"},"type":{"type":"string","enum":["tool_call"]},"module_id":{"type":"string"}},"required":["job_id","function_name","type","module_id"]},{"type":"object","properties":{"call_id":{"type":"string","format":"uuid"},"function_name":{"type":"string"},"resource_path":{"type":"string"},"type":{"type":"string","enum":["mcp_tool_call"]},"arguments":{"type":"object"}},"required":["call_id","function_name","resource_path","type"]},{"type":"object","properties":{"type":{"type":"string","enum":["web_search"]}},"required":["type"]},{"type":"object","properties":{"type":{"type":"string","enum":["message"]}},"required":["content","type"]}]}},"agent_actions_success":{"type":"array","items":{"type":"boolean"}}},"required":["type"]}}`;
|
|
|
|
export const CLI_COMMANDS = `# Windmill CLI Commands
|
|
|
|
The Windmill CLI (\`wmill\`) provides commands for managing scripts, flows, apps, and other resources.
|
|
|
|
## Global Options
|
|
|
|
- \`--workspace <workspace:string>\` - Specify the target workspace. This overrides the default workspace.
|
|
- \`--debug --verbose\` - Show debug/verbose logs
|
|
- \`--show-diffs\` - Show diff informations when syncing (may show sensitive informations)
|
|
- \`--token <token:string>\` - Specify an API token. This will override any stored token.
|
|
- \`--base-url <baseUrl:string>\` - Specify the base URL of the API. If used, --token and --workspace are required and no local remote/workspace already set will be used.
|
|
- \`--config-dir <configDir:string>\` - Specify a custom config directory. Overrides WMILL_CONFIG_DIR environment variable and default ~/.config location.
|
|
|
|
## Commands
|
|
|
|
### app
|
|
|
|
app related commands
|
|
|
|
**Options:**
|
|
- \`--json\` - Output as JSON (for piping to jq)
|
|
|
|
**Subcommands:**
|
|
|
|
- \`app list\` - list all apps
|
|
- \`--json\` - Output as JSON (for piping to jq)
|
|
- \`app get <path:string>\` - get an app's details
|
|
- \`--json\` - Output as JSON (for piping to jq)
|
|
- \`app push [file_path:string] [remote_path:string]\` - push a local app. With no args, infers the app from the current directory and the remote path from its location relative to wmill.yaml.
|
|
- \`app dev [app_folder:string]\` - Start a development server for building apps with live reload and hot module replacement
|
|
- \`--port <port:number>\` - Port to run the dev server on (will find next available port if occupied)
|
|
- \`--host <host:string>\` - Host to bind the dev server to
|
|
- \`--entry <entry:string>\` - Entry point file (default: index.ts for Svelte/Vue, index.tsx otherwise)
|
|
- \`--no-open\` - Don't automatically open the browser
|
|
- \`--recording\` - Frame the app in a shell with a Record button, to capture a replayable session recording of the app under development
|
|
- \`app lint [app_folder:string]\` - Lint a raw app folder to validate structure and buildability
|
|
- \`--fix\` - Attempt to fix common issues (not implemented yet)
|
|
- \`app bundle [app_folder:string]\` - Bundle a raw app folder to js/css without deploying it
|
|
- \`--out <dir:string>\` - Directory to write bundle.js and bundle.css into (default: <app_folder>/dist)
|
|
- \`--no-minify\` - Skip minification
|
|
- \`app new\` - create a new raw app from a template
|
|
- \`--summary <summary:string>\` - App summary (short description). Skips the prompt when provided. Triggers non-interactive mode.
|
|
- \`--path <path:string>\` - App path (e.g., f/folder/my_app or u/username/my_app). Skips the prompt when provided. Triggers non-interactive mode.
|
|
- \`--framework <framework:string>\` - Framework template: react19 | react18 | svelte5 | vue. Skips the prompt when provided. Triggers non-interactive mode.
|
|
- \`--datatable <datatable:string>\` - Datatable to wire up. Without this flag in non-interactive mode, no datatable is configured.
|
|
- \`--schema <schema:string>\` - Schema to use with --datatable. Created (CREATE SCHEMA IF NOT EXISTS) if it doesn't already exist.
|
|
- \`--overwrite\` - Overwrite the target directory if it already exists, without prompting.
|
|
- \`--no-open-in-desktop\` - Do not prompt to open the new app in Claude Desktop.
|
|
- \`app generate-agents [app_folder:string]\` - regenerate AGENTS.md and DATATABLES.md from remote workspace
|
|
- \`app set-permissioned-as <path:string> <email:string>\` - Set the on_behalf_of_email for an app (requires admin or wm_deployers group)
|
|
|
|
### audit
|
|
|
|
View audit logs (requires admin)
|
|
|
|
**Subcommands:**
|
|
|
|
- \`audit list\` - List audit log entries
|
|
- \`audit get <id:string>\` - Get a specific audit log entry
|
|
- \`--json\` - Output as JSON (for piping to jq)
|
|
|
|
### config
|
|
|
|
Show all available wmill.yaml configuration options
|
|
|
|
**Options:**
|
|
- \`--json\` - Output as JSON for programmatic consumption
|
|
|
|
**Subcommands:**
|
|
|
|
- \`config migrate\` - Migrate wmill.yaml from gitBranches/environments to workspaces format
|
|
|
|
### datatable
|
|
|
|
datatable related commands
|
|
|
|
**Subcommands:**
|
|
|
|
- \`datatable list\` - list all datatables in the workspace
|
|
- \`--json\` - Output as JSON (for piping to jq)
|
|
- \`datatable run <sql:string>\` - run a SQL query on a datatable
|
|
- \`-n --name <name:string>\` - Datatable name (default: main)
|
|
- \`-s --silent\` - Output only the final result as JSON. Useful for scripting.
|
|
- \`datatable migrate\` - manage datatable migrations
|
|
- \`datatable migrate new <name:string>\` - scaffold a new migration (.up.sql / .down.sql files)
|
|
- \`-d --datatable <datatable:string>\` - Target datatable (default: main)
|
|
- \`datatable migrate up\` - apply all pending migrations to the main datatable (or one via --datatable)
|
|
- \`-d --datatable <datatable:string>\` - Target datatable (default: main)
|
|
- \`datatable migrate down\` - roll back the most recent migration on the main datatable (or one via --datatable)
|
|
- \`-d --datatable <datatable:string>\` - Target datatable (default: main)
|
|
- \`datatable create [name:string]\` - register a datatable database in the workspace (default: instance-backed 'main') so scripts can use datatable://<name>
|
|
- \`--resource <resource:string>\` - Back the datatable with an existing postgresql resource path instead of the instance database
|
|
- \`--force\` - Allow adding to a workspace that already has datatables (fork metadata on existing ones is not preserved)
|
|
- \`datatable serve\` - Serve all datatables as a Postgres-wire endpoint (psql, DBeaver, pgAdmin); the client picks the datatable via the database name in its connection string
|
|
- \`--port <port:number>\` - Port to listen on (default: first free port in 5433-5500)
|
|
- \`--host <host:string>\` - Bind address (default: 127.0.0.1)
|
|
- \`--password <password:string>\` - Password for Postgres clients (default: generate a random password at startup)
|
|
- \`datatable psql\` - Start a serve listener and launch psql connected to it
|
|
- \`-n --name <name:string>\` - Datatable to connect psql to (default: main)
|
|
- \`--port <port:number>\` - Port the proxy listens on (default: first free port in 5433-5500)
|
|
- \`--host <host:string>\` - Bind address for the proxy (default: 127.0.0.1)
|
|
- \`--password <password:string>\` - Password for the temporary Postgres proxy (default: generate a random password at startup)
|
|
|
|
### dependencies
|
|
|
|
workspace dependencies related commands
|
|
|
|
**Alias:** \`deps\`
|
|
|
|
**Subcommands:**
|
|
|
|
- \`dependencies push <file_path:string>\` - Push workspace dependencies from a local file
|
|
|
|
### dev
|
|
|
|
Watch local file changes and live-reload the dev page for preview. Does NOT deploy to the remote workspace — use wmill sync push for that.
|
|
|
|
**Options:**
|
|
- \`--includes <pattern...:string>\` - Filter paths given a glob pattern or path
|
|
- \`--proxy-port <port:number>\` - Port for a localhost reverse proxy to the remote Windmill server
|
|
- \`--path <path:string>\` - Watch a specific windmill path (e.g., u/admin/my_script or f/my_flow)
|
|
- \`--no-open\` - Do not open the browser automatically
|
|
|
|
### docs
|
|
|
|
Search Windmill documentation.
|
|
|
|
**Arguments:** \`<query:string>\`
|
|
|
|
**Options:**
|
|
- \`--json\` - Output results as JSON.
|
|
|
|
### ducklake
|
|
|
|
ducklake related commands
|
|
|
|
**Subcommands:**
|
|
|
|
- \`ducklake list\` - list all ducklakes in the workspace
|
|
- \`--json\` - Output as JSON (for piping to jq)
|
|
- \`ducklake run <sql:string>\` - run a SQL query on a ducklake
|
|
- \`-n --name <name:string>\` - Ducklake name (default: main)
|
|
- \`-s --silent\` - Output only the final result as JSON. Useful for scripting.
|
|
|
|
### flow
|
|
|
|
flow related commands
|
|
|
|
**Options:**
|
|
- \`--show-archived\` - Enable archived flows in output
|
|
- \`--json\` - Output as JSON (for piping to jq)
|
|
|
|
**Subcommands:**
|
|
|
|
- \`flow list\` - list all flows
|
|
- \`--show-archived\` - Enable archived flows in output
|
|
- \`--json\` - Output as JSON (for piping to jq)
|
|
- \`flow get <path:string>\` - get a flow's details
|
|
- \`--json\` - Output as JSON (for piping to jq)
|
|
- \`flow push <file_path:string> <remote_path:string>\` - push a local flow spec. This overrides any remote versions.
|
|
- \`--message <message:string>\` - Deployment message
|
|
- \`flow run <path:string>\` - run a flow by path.
|
|
- \`-d --data <data:string>\` - Inputs specified as a JSON string or a file using @<filename> or stdin using @-. A resource argument is the bare string $res:<path> as its whole value, and a variable argument is the bare string $var:<path> — not an object wrapper keyed on $res/$var, and not a plain path.
|
|
- \`-s --silent\` - Do not ouput anything other then the final output. Useful for scripting.
|
|
- \`--tag <tag:string>\` - Override the worker tag the run is dispatched to (e.g. to route it to dev workers instead of the flow's default tag).
|
|
- \`flow preview <flow_path:string>\` - preview a local flow without deploying it. Runs the flow definition from local files and uses local PathScripts by default. Pass --step <id> to run only one module in isolation (resolves nested steps inside branchone/branchall/forloopflow/whileloopflow plus the special preprocessor/failure modules; supported step types: rawscript, script, flow).
|
|
- \`-d --data <data:string>\` - Inputs specified as a JSON string or a file using @<filename> or stdin using @-. A resource argument is the bare string $res:<path> as its whole value, and a variable argument is the bare string $var:<path> — not an object wrapper keyed on $res/$var, and not a plain path.
|
|
- \`-s --silent\` - Do not output anything other then the final output. Useful for scripting.
|
|
- \`--remote\` - Use deployed workspace scripts for PathScript steps instead of local files.
|
|
- \`--step <step_id:string>\` - Run only the named step instead of the whole flow. Honors --data as the step's args and --remote / local-PathScript resolution the same way the full-flow preview does.
|
|
- \`--tag <tag:string>\` - Override the worker tag the preview is dispatched to (e.g. to route it to dev workers instead of the flow's default tag).
|
|
- \`flow new <flow_path:string>\` - create a new empty flow
|
|
- \`--summary <summary:string>\` - flow summary
|
|
- \`--description <description:string>\` - flow description
|
|
- \`flow bootstrap <flow_path:string>\` - create a new empty flow (alias for new)
|
|
- \`--summary <summary:string>\` - flow summary
|
|
- \`--description <description:string>\` - flow description
|
|
- \`flow history <path:string>\` - Show version history for a flow
|
|
- \`--json\` - Output as JSON (for piping to jq)
|
|
- \`flow show-version <path:string> <version:string>\` - Show a specific version of a flow
|
|
- \`--json\` - Output as JSON (for piping to jq)
|
|
- \`flow set-permissioned-as <path:string> <email:string>\` - Set the on_behalf_of_email for a flow (requires admin or wm_deployers group)
|
|
|
|
### folder
|
|
|
|
folder related commands
|
|
|
|
**Options:**
|
|
- \`--json\` - Output as JSON (for piping to jq)
|
|
|
|
**Subcommands:**
|
|
|
|
- \`folder list\` - list all folders
|
|
- \`--json\` - Output as JSON (for piping to jq)
|
|
- \`folder get <name:string>\` - get a folder's details
|
|
- \`--json\` - Output as JSON (for piping to jq)
|
|
- \`folder new <name:string>\` - create a new folder locally
|
|
- \`--summary <summary:string>\` - folder summary
|
|
- \`folder push <name:string>\` - push a local folder to the remote by name. This overrides any remote versions.
|
|
- \`folder add-missing\` - create default folder.meta.yaml for all subdirectories of f/ that are missing one
|
|
- \`-y, --yes\` - skip confirmation prompt
|
|
- \`folder show-rules <name:string>\` - Show default_permissioned_as rules for a folder. Use --test-path to see which rule matches a given item path.
|
|
- \`--test-path <path:string>\` - Test which rule matches this item path (e.g. f/prod/jobs/my_script)
|
|
- \`--json\` - Output as JSON
|
|
|
|
### generate-metadata
|
|
|
|
Regenerate stale local locks and script schemas and refresh wmill-lock.yaml content hashes (scripts, flows, apps). Writes local files only, not a deploy. Run it after edits that add or remove imports or change a script's arguments, so the lock, the auto-generated UI schema, and wmill-lock.yaml stay in sync.
|
|
|
|
**Arguments:** \`[folder:string]\`
|
|
|
|
**Options:**
|
|
- \`--yes\` - Skip confirmation prompt
|
|
- \`--dry-run\` - Show what would be updated without making changes
|
|
- \`--lock-only\` - Re-generate only the lock files
|
|
- \`--schema-only\` - Re-generate only script schemas (skips flows and apps)
|
|
- \`--skip-scripts\` - Skip processing scripts
|
|
- \`--skip-flows\` - Skip processing flows
|
|
- \`--skip-apps\` - Skip processing apps
|
|
- \`--strict-folder-boundaries\` - Only update items inside the specified folder (requires folder argument)
|
|
- \`--parallel <n:number>\` - Number of items to process in parallel
|
|
- \`-i --includes <patterns:file[]>\` - Comma separated patterns to specify which files to include
|
|
- \`-e --excludes <patterns:file[]>\` - Comma separated patterns to specify which files to exclude
|
|
|
|
**Subcommands:**
|
|
|
|
- \`generate-metadata rehash [folder:string]\` - Refresh wmill-lock.yaml content hashes from the on-disk .lock and .script.yaml without re-resolving dependencies or hitting the backend. Use when those files are already correct and only the hashes need updating: bootstrapping missing entries or recovering from hash drift.
|
|
- \`--skip-scripts\` - Skip processing scripts
|
|
- \`--skip-flows\` - Skip processing flows
|
|
- \`--skip-apps\` - Skip processing apps
|
|
- \`--parallel <n:number>\` - Number of items to process in parallel
|
|
- \`-i --includes <patterns:file[]>\` - Comma separated patterns to specify which files to include
|
|
- \`-e --excludes <patterns:file[]>\` - Comma separated patterns to specify which files to exclude
|
|
|
|
### gitsync-settings
|
|
|
|
Manage git-sync settings between local wmill.yaml and Windmill backend
|
|
|
|
**Subcommands:**
|
|
|
|
- \`gitsync-settings pull\` - Pull git-sync settings from Windmill backend to local wmill.yaml
|
|
- \`--repository <repo:string>\` - Specify repository path (e.g., u/user/repo)
|
|
- \`--default\` - Write settings to top-level defaults instead of overrides
|
|
- \`--replace\` - Replace existing settings (non-interactive mode)
|
|
- \`--override\` - Add branch-specific override (non-interactive mode)
|
|
- \`--diff\` - Show differences without applying changes
|
|
- \`--json-output\` - Output in JSON format
|
|
- \`--with-backend-settings <json:string>\` - Use provided JSON settings instead of querying backend (for testing)
|
|
- \`--yes\` - Skip interactive prompts and use default behavior
|
|
- \`--promotion <branch:string>\` - Use promotionOverrides from the specified branch instead of regular overrides
|
|
- \`gitsync-settings push\` - Push git-sync settings from local wmill.yaml to Windmill backend
|
|
- \`--repository <repo:string>\` - Specify repository path (e.g., u/user/repo)
|
|
- \`--diff\` - Show what would be pushed without applying changes
|
|
- \`--json-output\` - Output in JSON format
|
|
- \`--with-backend-settings <json:string>\` - Use provided JSON settings instead of querying backend (for testing)
|
|
- \`--yes\` - Skip interactive prompts and use default behavior
|
|
- \`--promotion <branch:string>\` - Use promotionOverrides from the specified branch instead of regular overrides
|
|
- \`gitsync-settings status\` - Report how local changes deploy to the workspace (git push vs wmill sync push)
|
|
- \`--json-output\` - Output in JSON format
|
|
|
|
### group
|
|
|
|
Manage workspace groups
|
|
|
|
**Options:**
|
|
- \`--json\` - Output as JSON (for piping to jq)
|
|
|
|
**Subcommands:**
|
|
|
|
- \`group list\` - List all groups in the workspace
|
|
- \`--json\` - Output as JSON (for piping to jq)
|
|
- \`group get <name:string>\` - Get group details and members
|
|
- \`--json\` - Output as JSON (for piping to jq)
|
|
- \`group create <name:string>\` - Create a new group
|
|
- \`--summary <summary:string>\` - Group summary/description
|
|
- \`group delete <name:string>\` - Delete a group
|
|
- \`group add-user <name:string> <username:string>\` - Add a user to a group
|
|
- \`group remove-user <name:string> <username:string>\` - Remove a user from a group
|
|
|
|
### hub
|
|
|
|
Hub related commands. EXPERIMENTAL. INTERNAL USE ONLY.
|
|
|
|
**Subcommands:**
|
|
|
|
- \`hub pull\` - pull any supported definitions. EXPERIMENTAL.
|
|
|
|
### init
|
|
|
|
Bootstrap a windmill project with a wmill.yaml file
|
|
|
|
**Options:**
|
|
- \`--use-default\` - Use default settings without checking backend
|
|
- \`--use-backend\` - Use backend git-sync settings if available
|
|
- \`--repository <repo:string>\` - Specify repository path (e.g., u/user/repo) when using backend settings
|
|
- \`--bind-profile\` - Automatically bind active workspace profile to current Git branch
|
|
- \`--no-bind-profile\` - Skip workspace profile binding prompt
|
|
|
|
### instance
|
|
|
|
sync local with a remote instance or the opposite (push or pull)
|
|
|
|
**Subcommands:**
|
|
|
|
- \`instance add [instance_name:string] [remote:string] [token:string]\` - Add a new instance
|
|
- \`instance remove <instance:string:instance>\` - Remove an instance
|
|
- \`instance switch <instance:string:instance>\` - Switch the current instance
|
|
- \`instance pull\` - Pull instance settings, users, configs, instance groups and overwrite local
|
|
- \`--yes\` - Pull without needing confirmation
|
|
- \`--dry-run\` - Perform a dry run without making changes
|
|
- \`--skip-users\` - Skip pulling users
|
|
- \`--skip-settings\` - Skip pulling settings
|
|
- \`--skip-configs\` - Skip pulling configs (worker groups)
|
|
- \`--skip-groups\` - Skip pulling instance groups
|
|
- \`--include-workspaces\` - Also pull workspaces
|
|
- \`--folder-per-instance\` - Create a folder per instance
|
|
- \`--instance <instance:string>\` - Name of the instance to pull from, override the active instance
|
|
- \`--prefix <prefix:string>\` - Prefix of the local workspaces to pull, used to create the folders when using --include-workspaces
|
|
- \`--prefix-settings\` - Store instance yamls inside prefixed folders when using --prefix and --folder-per-instance
|
|
- \`instance push\` - Push instance settings, users, configs, group and overwrite remote
|
|
- \`--yes\` - Push without needing confirmation
|
|
- \`--dry-run\` - Perform a dry run without making changes
|
|
- \`--skip-users\` - Skip pushing users
|
|
- \`--skip-settings\` - Skip pushing settings
|
|
- \`--skip-configs\` - Skip pushing configs (worker groups)
|
|
- \`--skip-groups\` - Skip pushing instance groups
|
|
- \`--include-workspaces\` - Also push workspaces
|
|
- \`--folder-per-instance\` - Create a folder per instance
|
|
- \`--instance <instance:string>\` - Name of the instance to push to, override the active instance
|
|
- \`--prefix <prefix:string>\` - Prefix of the local workspaces folders to push
|
|
- \`--prefix-settings\` - Store instance yamls inside prefixed folders when using --prefix and --folder-per-instance
|
|
- \`instance whoami\` - Display information about the currently logged-in user
|
|
- \`instance get-config\` - Dump the current instance config (global settings + worker configs) as YAML
|
|
- \`-o, --output-file <file:string>\` - Write YAML to a file instead of stdout
|
|
- \`--show-secrets\` - Include sensitive fields (license key, JWT secret) without prompting
|
|
- \`--instance <instance:string>\` - Name of the instance, override the active instance
|
|
- \`instance connect-slack\` - Non-interactively connect Slack at the instance level using a pre-minted bot token (xoxb-...). Produces the same artifacts as the UI OAuth flow: global_settings 'slack' row + encrypted f/slack_bot/global_bot_token variable and resource in the admins workspace.
|
|
- \`--bot-token <bot_token:string>\` - Slack bot token (xoxb-...)
|
|
- \`--team-id <team_id:string>\` - Slack team id
|
|
- \`--team-name <team_name:string>\` - Slack team name
|
|
- \`--instance <instance:string>\` - Instance profile to connect against (defaults to the active instance)
|
|
|
|
### job
|
|
|
|
Manage jobs (list, inspect, cancel)
|
|
|
|
**Subcommands:**
|
|
|
|
- \`job list\` - List recent jobs
|
|
- \`job get <id:string>\` - Get job details. For flows: shows step tree with sub-job IDs
|
|
- \`--json\` - Output as JSON (for piping to jq)
|
|
- \`job result <id:string>\` - Get the result of a completed job (machine-friendly)
|
|
- \`job logs <id:string>\` - Get job logs. For flows: aggregates all step logs
|
|
- \`job cancel <id:string>\` - Cancel a running or queued job
|
|
- \`--reason <reason:string>\` - Reason for cancellation
|
|
- \`job rerun <id:string>\` - Re-run a completed job with the same args. Prints the new job UUID on stdout.
|
|
- \`job restart <id:string>\` - Restart a completed flow at a given top-level step. Prints the new flow job UUID on stdout.
|
|
- \`--step <stepId:string>\` - Top-level step id to restart the flow from
|
|
- \`--iteration <n:number>\` - For a top-level branchall or for-loop step, the iteration to restart at
|
|
|
|
### jobs
|
|
|
|
Manage jobs (import/export)
|
|
|
|
**Subcommands:**
|
|
|
|
- \`jobs pull [workspace:string]\` - Pull completed and queued jobs from workspace
|
|
- \`-c, --completed-output <file:string>\` - Completed jobs output file (default: completed_jobs.json)
|
|
- \`-q, --queued-output <file:string>\` - Queued jobs output file (default: queued_jobs.json)
|
|
- \`--skip-worker-check\` - Skip checking for active workers before export
|
|
- \`jobs push [workspace:string]\` - Push completed and queued jobs to workspace
|
|
- \`-c, --completed-file <file:string>\` - Completed jobs input file (default: completed_jobs.json)
|
|
- \`-q, --queued-file <file:string>\` - Queued jobs input file (default: queued_jobs.json)
|
|
- \`--skip-worker-check\` - Skip checking for active workers before import
|
|
|
|
### lint
|
|
|
|
Validate Windmill flow, schedule, and trigger YAML files in a directory, and report script metadata that has no deployable content file
|
|
|
|
**Arguments:** \`[directory:string]\`
|
|
|
|
**Options:**
|
|
- \`--json\` - Output results in JSON format
|
|
- \`--fail-on-warn\` - Exit with code 1 when warnings are emitted
|
|
- \`--locks-required\` - Fail if scripts or flow inline scripts that need locks have no locks
|
|
- \`-w, --watch\` - Watch for file changes and re-lint automatically
|
|
|
|
### object-storage
|
|
|
|
Object storage (S3) related commands. Operates on the workspace's default object storage; use --storage to target a configured secondary storage.
|
|
|
|
**Alias:** \`s3\`
|
|
|
|
**Subcommands:**
|
|
|
|
- \`object-storage list\` - List configured object storages for the workspace (default + secondary).
|
|
- \`--json\` - Output as JSON (for piping to jq)
|
|
- \`object-storage files [prefix:string]\` - List files in an object storage. Optionally filter by prefix.
|
|
- \`--json\` - Output as JSON (for piping to jq)
|
|
- \`--max-keys <maxKeys:number>\` - Page size (default 100)
|
|
- \`--marker <marker:string>\` - Pagination marker from a previous response
|
|
- \`--storage <storage:string>\` - Secondary storage name (omit for the workspace default)
|
|
- \`object-storage upload <local_path:string> <file_key:string>\` - Upload a local file to object storage at the given file key.
|
|
- \`--storage <storage:string>\` - Secondary storage name
|
|
- \`--content-type <contentType:string>\` - Content-Type header to set on the object
|
|
- \`--content-disposition <contentDisposition:string>\` - Content-Disposition header to set on the object
|
|
- \`object-storage download <file_key:string> [output_path:string]\` - Download an object to a local file (or stdout). Default output path is the basename of the file key in the current directory.
|
|
- \`--storage <storage:string>\` - Secondary storage name
|
|
- \`--stdout\` - Write file contents to stdout instead of a file
|
|
- \`object-storage delete <file_key:string>\` - Delete an object from object storage. Prompts for confirmation unless --yes is set.
|
|
- \`--storage <storage:string>\` - Secondary storage name
|
|
- \`--yes\` - Skip the confirmation prompt
|
|
- \`object-storage move <src_file_key:string> <dest_file_key:string>\` - Move an object within the same storage (rename or relocate by key).
|
|
- \`--storage <storage:string>\` - Secondary storage name
|
|
- \`object-storage info <file_key:string>\` - Show metadata (size, mime, last-modified) for an object.
|
|
- \`--json\` - Output as JSON (for piping to jq)
|
|
- \`--storage <storage:string>\` - Secondary storage name
|
|
- \`object-storage preview <file_key:string>\` - Preview the contents of an object (text/CSV). Use --bytes-from / --bytes-length to peek at a slice of binary files.
|
|
- \`--storage <storage:string>\` - Secondary storage name
|
|
- \`--mime <mime:string>\` - Override the detected mime type (e.g. text/csv)
|
|
- \`--bytes-from <bytesFrom:number>\` - Start offset in bytes
|
|
- \`--bytes-length <bytesLength:number>\` - Number of bytes to read
|
|
- \`--csv-separator <csvSeparator:string>\` - CSV column separator (default ,)
|
|
- \`--csv-header\` - Treat the first CSV row as a header
|
|
|
|
### pipeline
|
|
|
|
inspect asset-driven pipelines (scripts marked \`// pipeline\`, wired by \`// on <spec>\` annotations)
|
|
|
|
**Subcommands:**
|
|
|
|
- \`pipeline list\` - list pipeline folders in the workspace
|
|
- \`--json\` - Output as JSON (for piping to jq)
|
|
- \`pipeline show <folder:string>\` - render a pipeline folder's DAG (sources, lineage, subscriptions) in the terminal
|
|
- \`--json\` - Output the raw asset graph as JSON
|
|
- \`--local\` - Build the graph from local working-tree files (// pipeline scripts) instead of the deployed workspace — no deploy needed.
|
|
- \`pipeline run <folder:string>\` - run a cascade: from --from (a root OR any mid-DAG model), fan downstream up to the --to end node(s)
|
|
- \`--from <script:string>\` - Start script (short name or path). May be any node, including a mid-DAG model — that node plus its transitive downstream runs, upstream is NOT re-run (dbt \`--select model+\`). Defaults to the folder's sole schedule/manual root.
|
|
- \`--to <node:string>\` - End node(s) to stop at — script names/paths or asset URIs (e.g. datatable://main/staged). Repeatable or comma-separated. Omit to run the full downstream.
|
|
- \`--dry-run\` - Print the topological run plan without executing.
|
|
- \`--json\` - Output the plan as JSON (for piping to jq).
|
|
- \`--local\` - Run the local working-tree scripts via preview (no deploy) instead of the deployed versions; the graph is built from local files.
|
|
- \`--upload <binding:string>\` - Bind an object to a data_upload/webhook entry point so it runs in the cascade, as SCRIPT[:PARAM]=SOURCE (SOURCE is a local file or an s3://key). Local files are uploaded to the workspace store; the S3Object param is inferred when the script has exactly one. Repeatable.
|
|
- \`--arg <binding:string>\` - Pass a plain run arg to a script in the cascade, as SCRIPT:PARAM=VALUE (VALUE is parsed as JSON when possible, else taken as a string — e.g. daily_report:partition=2026-07-02). Repeatable.
|
|
- \`--partition <value:string>\` - Partition value for \`// partitioned\` scripts in the run (e.g. 2026-06-30) — use it to backfill a past slice. With --local, time kinds (daily/hourly/weekly/monthly) default to the current UTC period when omitted; \`dynamic\` always needs it. Deployed runs without it defer to backend run-start resolution.
|
|
- \`pipeline docs <folder:string>\` - generate PIPELINE.md (+ AGENTS.md pointer) describing a folder's pipeline graph and datatable schemas, for an editor / agentic loop
|
|
- \`--local\` - Build the graph from local working-tree files instead of the deployed workspace.
|
|
- \`pipeline dev [folder:string]\` - Live-preview a data pipeline from local files: watch an \`f/<folder>\` of \`// pipeline\` scripts, push the working-tree graph to the dev page, and run the cascade via preview (no deploy).
|
|
- \`--port <port:number>\` - Port for the dev WebSocket server.
|
|
- \`--no-open\` - Do not open the browser automatically.
|
|
- \`--frontend <origin:string>\` - Origin serving the /pipeline_dev page (e.g. http://localhost:3000 for a locally-run frontend). Defaults to the workspace remote; use it when the remote's deployed frontend predates the dev page.
|
|
|
|
### protection-rules
|
|
|
|
Sync workspace protection rules between protection-rules.yaml and Windmill. The file is keyed by workspace name; keys must match wmill.yaml 'workspaces'.
|
|
|
|
**Subcommands:**
|
|
|
|
- \`protection-rules pull [workspace:string]\` - Pull protection rules from Windmill into protection-rules.yaml for a workspace
|
|
- \`--all\` - Pull every workspace defined in wmill.yaml
|
|
- \`--dry-run\` - Show what would change without writing the file
|
|
- \`--json-output\` - Output in JSON format
|
|
- \`protection-rules push [workspace:string]\` - Push protection rules from protection-rules.yaml to Windmill for a workspace (full reconcile: creates, updates, and deletes)
|
|
- \`--all\` - Push every workspace defined in protection-rules.yaml
|
|
- \`--dry-run\` - Show what would change without applying
|
|
- \`--json-output\` - Output in JSON format
|
|
- \`--yes\` - Skip the confirmation prompt (including deletions)
|
|
|
|
### queues
|
|
|
|
List all queues with their metrics
|
|
|
|
**Arguments:** \`[workspace:string] the optional workspace to filter by (default to all workspaces)\`
|
|
|
|
**Options:**
|
|
- \`--instance [instance]\` - Name of the instance to push to, override the active instance
|
|
- \`--base-url [baseUrl]\` - If used with --token, will be used as the base url for the instance
|
|
|
|
### refresh
|
|
|
|
Refresh wmill-managed project files (AGENTS.wmill.md, skills, tsconfig.wmill.json)
|
|
|
|
**Subcommands:**
|
|
|
|
- \`refresh prompts\` - Refresh AGENTS.wmill.md and managed skills. User-owned AGENTS.md and CLAUDE.md are never overwritten unless you opt in.
|
|
- \`--yes\` - Non-interactive: append the @AGENTS.wmill.md include to an existing AGENTS.md / CLAUDE.md without prompting. Without it, a non-interactive run leaves an unlinked file untouched.
|
|
- \`refresh tsconfig\` - Refresh the wmill-managed tsconfig.wmill.json (and Deno import map for Deno projects)
|
|
- \`--yes\` - Non-interactive: wire an existing custom tsconfig.json/deno.json to the managed file without prompting (a previously-generated config is always migrated automatically).
|
|
|
|
### resource
|
|
|
|
resource related commands
|
|
|
|
**Options:**
|
|
- \`--json\` - Output as JSON (for piping to jq)
|
|
|
|
**Subcommands:**
|
|
|
|
- \`resource list\` - list all resources
|
|
- \`--json\` - Output as JSON (for piping to jq)
|
|
- \`resource get <path:string>\` - get a resource's details
|
|
- \`--json\` - Output as JSON (for piping to jq)
|
|
- \`resource new <path:string>\` - create a new resource locally
|
|
- \`resource push <file_path:string> <remote_path:string>\` - push a local resource spec. This overrides any remote versions.
|
|
|
|
### resource-type
|
|
|
|
resource type related commands
|
|
|
|
**Options:**
|
|
- \`--json\` - Output as JSON (for piping to jq)
|
|
|
|
**Subcommands:**
|
|
|
|
- \`resource-type list\` - list all resource types
|
|
- \`--schema\` - Show schema in the output
|
|
- \`--json\` - Output as JSON (for piping to jq)
|
|
- \`resource-type get <path:string>\` - get a resource type's details
|
|
- \`--json\` - Output as JSON (for piping to jq)
|
|
- \`resource-type new <name:string>\` - create a new resource type locally
|
|
- \`resource-type push <file_path:string> <name:string>\` - push a local resource spec. This overrides any remote versions.
|
|
- \`resource-type generate-namespace\` - Create a TypeScript definition file with the RT namespace generated from the resource types
|
|
|
|
### schedule
|
|
|
|
schedule related commands
|
|
|
|
**Options:**
|
|
- \`--json\` - Output as JSON (for piping to jq)
|
|
|
|
**Subcommands:**
|
|
|
|
- \`schedule list\` - list all schedules
|
|
- \`--json\` - Output as JSON (for piping to jq)
|
|
- \`schedule get <path:string>\` - get a schedule's details
|
|
- \`--json\` - Output as JSON (for piping to jq)
|
|
- \`schedule new <path:string>\` - create a new schedule locally
|
|
- \`schedule push <file_path:string> <remote_path:string>\` - push a local schedule spec. This overrides any remote versions.
|
|
- \`schedule enable <path:string>\` - Enable a schedule
|
|
- \`--force\` - Bypass the fork-conflict warning when the parent workspace has the same schedule (acknowledges that both crons will fire)
|
|
- \`schedule disable <path:string>\` - Disable a schedule
|
|
- \`schedule set-permissioned-as <path:string> <email:string>\` - Set the email (run-as user) for a schedule (requires admin or wm_deployers group)
|
|
|
|
### script
|
|
|
|
script related commands
|
|
|
|
**Options:**
|
|
- \`--show-archived\` - Show archived scripts instead of active ones
|
|
- \`--json\` - Output as JSON (for piping to jq)
|
|
|
|
**Subcommands:**
|
|
|
|
- \`script list\` - list all scripts
|
|
- \`--show-archived\` - Show archived scripts instead of active ones
|
|
- \`--json\` - Output as JSON (for piping to jq)
|
|
- \`script push <path:file>\` - push a local script spec. This overrides any remote versions. Use the script file (.ts, .js, .py, .sh)
|
|
- \`--message <message:string>\` - Deployment message
|
|
- \`script get <path:file>\` - get a script's details
|
|
- \`--json\` - Output as JSON (for piping to jq)
|
|
- \`script show <path:file>\` - show a script's content (alias for get)
|
|
- \`script run <path:file>\` - run a script by path
|
|
- \`-d --data <data:file>\` - Inputs specified as a JSON string or a file using @<filename> or stdin using @-. A resource argument is the bare string $res:<path> as its whole value, and a variable argument is the bare string $var:<path> — not an object wrapper keyed on $res/$var, and not a plain path.
|
|
- \`-s --silent\` - Do not output anything other then the final output. Useful for scripting.
|
|
- \`--tag <tag:string>\` - Override the worker tag the run is dispatched to (e.g. to route it to dev workers instead of the script's default tag).
|
|
- \`script preview <path:file>\` - preview a local script without deploying it. Supports both regular and codebase scripts.
|
|
- \`-d --data <data:file>\` - Inputs specified as a JSON string or a file using @<filename> or stdin using @-. A resource argument is the bare string $res:<path> as its whole value, and a variable argument is the bare string $var:<path> — not an object wrapper keyed on $res/$var, and not a plain path.
|
|
- \`-s --silent\` - Do not output anything other than the final output. Useful for scripting.
|
|
- \`--tag <tag:string>\` - Override the worker tag the preview is dispatched to (e.g. to route it to dev workers instead of the script's default tag).
|
|
- \`script new <path:file> <language:string>\` - create a new script
|
|
- \`--summary <summary:string>\` - script summary
|
|
- \`--description <description:string>\` - script description
|
|
- \`script bootstrap <path:file> <language:string>\` - create a new script (alias for new)
|
|
- \`--summary <summary:string>\` - script summary
|
|
- \`--description <description:string>\` - script description
|
|
- \`script set-permissioned-as <path:string> <email:string>\` - Set the on_behalf_of_email for a script (requires admin or wm_deployers group)
|
|
- \`script history <path:string>\` - show version history for a script
|
|
- \`--json\` - Output as JSON (for piping to jq)
|
|
|
|
### sync
|
|
|
|
sync local with a remote workspaces or the opposite (push or pull)
|
|
|
|
**Subcommands:**
|
|
|
|
- \`sync pull\` - Pull any remote changes and apply them locally.
|
|
- \`--yes\` - Pull without needing confirmation
|
|
- \`--dry-run\` - Show changes that would be pulled without actually pushing
|
|
- \`--plain-secrets\` - Pull secrets as plain text
|
|
- \`--json\` - Use JSON instead of YAML
|
|
- \`--skip-variables\` - Skip syncing variables (including secrets)
|
|
- \`--skip-secrets\` - Skip syncing only secrets variables
|
|
- \`--include-secrets\` - Include secrets in sync (overrides skipSecrets in wmill.yaml)
|
|
- \`--skip-resources\` - Skip syncing resources
|
|
- \`--skip-resource-types\` - Skip syncing resource types
|
|
- \`--skip-scripts\` - Skip syncing scripts
|
|
- \`--skip-flows\` - Skip syncing flows
|
|
- \`--skip-apps\` - Skip syncing apps
|
|
- \`--skip-folders\` - Skip syncing folders
|
|
- \`--skip-workspace-dependencies\` - Skip syncing workspace dependencies
|
|
- \`--skip-scripts-metadata\` - Skip syncing scripts metadata, focus solely on logic
|
|
- \`--include-schedules\` - Include syncing schedules
|
|
- \`--include-triggers\` - Include syncing triggers
|
|
- \`--include-users\` - Include syncing users
|
|
- \`--include-groups\` - Include syncing groups
|
|
- \`--include-settings\` - Include syncing workspace settings
|
|
- \`--include-key\` - Include workspace encryption key
|
|
- \`--keep-deleted\` - Do not delete local files for items that no longer exist on the remote workspace. Only adds and updates.
|
|
- \`--skip-branch-validation\` - Skip git branch validation and prompts
|
|
- \`--json-output\` - Output results in JSON format
|
|
- \`-i --includes <patterns:file[]>\` - Comma separated patterns to specify which file to take into account (among files that are compatible with windmill). Patterns can include * (any string until '/') and ** (any string). Overrides wmill.yaml includes
|
|
- \`-e --excludes <patterns:file[]>\` - Comma separated patterns to specify which file to NOT take into account. Overrides wmill.yaml excludes
|
|
- \`--extra-includes <patterns:file[]>\` - Comma separated patterns to specify which file to take into account (among files that are compatible with windmill). Patterns can include * (any string until '/') and ** (any string). Useful to still take wmill.yaml into account and act as a second pattern to satisfy
|
|
- \`--repository <repo:string>\` - Specify repository path (e.g., u/user/repo) when multiple repositories exist
|
|
- \`--promotion <branch:string>\` - Use promotionOverrides from the specified branch instead of regular overrides
|
|
- \`--branch, --env <branch:string>\` - [Deprecated: use --workspace] Override the current git branch/environment
|
|
- \`sync push\` - Push any local changes and apply them remotely.
|
|
- \`--yes\` - Push without needing confirmation
|
|
- \`--dry-run\` - Show changes that would be pushed without actually pushing
|
|
- \`--plain-secrets\` - Push secrets as plain text
|
|
- \`--json\` - Use JSON instead of YAML
|
|
- \`--skip-variables\` - Skip syncing variables (including secrets)
|
|
- \`--skip-secrets\` - Skip syncing only secrets variables
|
|
- \`--include-secrets\` - Include secrets in sync (overrides skipSecrets in wmill.yaml)
|
|
- \`--skip-resources\` - Skip syncing resources
|
|
- \`--skip-resource-types\` - Skip syncing resource types
|
|
- \`--skip-scripts\` - Skip syncing scripts
|
|
- \`--skip-flows\` - Skip syncing flows
|
|
- \`--skip-apps\` - Skip syncing apps
|
|
- \`--skip-folders\` - Skip syncing folders
|
|
- \`--skip-workspace-dependencies\` - Skip syncing workspace dependencies
|
|
- \`--skip-scripts-metadata\` - Skip syncing scripts metadata, focus solely on logic
|
|
- \`--include-schedules\` - Include syncing schedules
|
|
- \`--include-triggers\` - Include syncing triggers
|
|
- \`--include-users\` - Include syncing users
|
|
- \`--include-groups\` - Include syncing groups
|
|
- \`--include-settings\` - Include syncing workspace settings
|
|
- \`--include-key\` - Include workspace encryption key
|
|
- \`--skip-reencrypt-on-key-change\` - When the pushed encryption key differs from the remote, do NOT re-encrypt existing remote secrets. Only safe if they are already encrypted with the new key (e.g. workspace/instance migration). Default is to re-encrypt.
|
|
- \`--keep-deleted\` - Do not delete remote items that no longer exist locally. Only adds and updates.
|
|
- \`--skip-branch-validation\` - Skip git branch validation and prompts
|
|
- \`--json-output\` - Output results in JSON format
|
|
- \`-i --includes <patterns:file[]>\` - Comma separated patterns to specify which file to take into account (among files that are compatible with windmill). Patterns can include * (any string until '/') and ** (any string)
|
|
- \`-e --excludes <patterns:file[]>\` - Comma separated patterns to specify which file to NOT take into account.
|
|
- \`--extra-includes <patterns:file[]>\` - Comma separated patterns to specify which file to take into account (among files that are compatible with windmill). Patterns can include * (any string until '/') and ** (any string). Useful to still take wmill.yaml into account and act as a second pattern to satisfy
|
|
- \`--message <message:string>\` - Include a message that will be added to all scripts/flows/apps updated during this push
|
|
- \`--parallel <number>\` - Number of changes to process in parallel
|
|
- \`--repository <repo:string>\` - Specify repository path (e.g., u/user/repo) when multiple repositories exist
|
|
- \`--branch, --env <branch:string>\` - [Deprecated: use --workspace] Override the current git branch/environment
|
|
- \`--lint\` - Run lint validation before pushing
|
|
- \`--locks-required\` - Fail if scripts or flow inline scripts that need locks have no locks
|
|
- \`--auto-metadata\` - Automatically regenerate stale metadata (locks and schemas) before pushing
|
|
- \`--accept-overriding-permissioned-as-with-self\` - Accept that items with a different permissioned_as will be updated with your own user
|
|
|
|
### token
|
|
|
|
Manage API tokens
|
|
|
|
**Options:**
|
|
- \`--json\` - Output as JSON (for piping to jq)
|
|
|
|
**Subcommands:**
|
|
|
|
- \`token list\` - List API tokens
|
|
- \`--json\` - Output as JSON (for piping to jq)
|
|
- \`token create\` - Create a new API token
|
|
- \`--label <label:string>\` - Token label
|
|
- \`--expiration <expiration:string>\` - Token expiration (ISO 8601 timestamp)
|
|
- \`token delete <token_prefix:string>\` - Delete a token by its prefix
|
|
|
|
### trash
|
|
|
|
List, inspect and restore items deleted in the last three days (requires admin)
|
|
|
|
**Options:**
|
|
- \`--json\` - Output as JSON (for piping to jq)
|
|
- \`--kind <kind:string>\` - Only items of this kind: script, flow, app, schedule, variable, resource or a trigger kind such as http_trigger
|
|
- \`--limit <limit:integer>\` - Number of items to return (default 100, max 1000)
|
|
- \`--page <page:integer>\` - Page to return, starting at 1
|
|
|
|
**Subcommands:**
|
|
|
|
- \`trash list\` - List trashed items, most recently deleted first
|
|
- \`--json\` - Output as JSON (for piping to jq)
|
|
- \`--kind <kind:string>\` - Only items of this kind: script, flow, app, schedule, variable, resource or a trigger kind such as http_trigger
|
|
- \`--limit <limit:integer>\` - Number of items to return (default 100, max 1000)
|
|
- \`--page <page:integer>\` - Page to return, starting at 1
|
|
- \`trash get <id:integer>\` - Show a trashed item and the data it was deleted with
|
|
- \`--json\` - Output as JSON (for piping to jq)
|
|
- \`trash restore <ids...:integer>\` - Put trashed items back at their paths
|
|
|
|
### trigger
|
|
|
|
trigger related commands
|
|
|
|
**Options:**
|
|
- \`--json\` - Output as JSON (for piping to jq)
|
|
|
|
**Subcommands:**
|
|
|
|
- \`trigger list\` - list all triggers
|
|
- \`--json\` - Output as JSON (for piping to jq)
|
|
- \`trigger get <path:string>\` - get a trigger's details
|
|
- \`--json\` - Output as JSON (for piping to jq)
|
|
- \`--kind <kind:string>\` - Trigger kind (http, websocket, kafka, nats, postgres, mqtt, amqp, sqs, gcp, azure, email). Recommended for faster lookup
|
|
- \`trigger new <path:string>\` - create a new trigger locally
|
|
- \`--kind <kind:string>\` - Trigger kind (required: http, websocket, kafka, nats, postgres, mqtt, amqp, sqs, gcp, azure, email)
|
|
- \`trigger push <file_path:string> <remote_path:string>\` - push a local trigger spec. This overrides any remote versions.
|
|
- \`trigger set-permissioned-as <path:string> <email:string>\` - Set the email (run-as user) for a trigger (requires admin or wm_deployers group)
|
|
- \`--kind <kind:string>\` - Trigger kind (required: http, websocket, kafka, nats, postgres, mqtt, amqp, sqs, gcp, azure, email)
|
|
|
|
### user
|
|
|
|
user related commands
|
|
|
|
**Subcommands:**
|
|
|
|
- \`user add <email:string> [password:string]\` - Create a user
|
|
- \`--superadmin\` - Specify to make the new user superadmin.
|
|
- \`--company <company:string>\` - Specify to set the company of the new user.
|
|
- \`--name <name:string>\` - Specify to set the name of the new user.
|
|
- \`user remove <email:string>\` - Delete a user
|
|
- \`user create-token\` - Create a new API token for the authenticated user
|
|
- \`--email <email:string>\` - Specify credentials to use for authentication. This will not be stored. It will only be used to exchange for a token with the API server, which will not be stored either.
|
|
- \`--password <password:string>\` - Specify credentials to use for authentication. This will not be stored. It will only be used to exchange for a token with the API server, which will not be stored either.
|
|
|
|
### variable
|
|
|
|
variable related commands
|
|
|
|
**Options:**
|
|
- \`--json\` - Output as JSON (for piping to jq)
|
|
|
|
**Subcommands:**
|
|
|
|
- \`variable list\` - list all variables
|
|
- \`--json\` - Output as JSON (for piping to jq)
|
|
- \`variable get <path:string>\` - get a variable's details
|
|
- \`--json\` - Output as JSON (for piping to jq)
|
|
- \`variable new <path:string>\` - create a new variable locally
|
|
- \`variable push <file_path:string> <remote_path:string>\` - Push a local variable spec. This overrides any remote versions.
|
|
- \`--plain-secrets\` - Push secrets as plain text
|
|
- \`variable add <value:string> <remote_path:string>\` - Create a new variable on the remote. This will update the variable if it already exists.
|
|
- \`--yes\` - Skip confirmation prompt when updating an existing variable
|
|
- \`--secret\` - Mark the variable as secret (default when creating a new variable)
|
|
- \`--no-secret\` - Mark the variable as non-secret (when updating, the existing setting is preserved if neither --secret nor --no-secret is passed)
|
|
- \`--description <description:string>\` - Set the variable description (when updating, the existing description is preserved if not passed)
|
|
- \`--plain-secrets\` - Push secrets as plain text
|
|
- \`--public\` - Legacy option, use --no-secret instead
|
|
|
|
### version
|
|
|
|
Show version information
|
|
|
|
### worker-groups
|
|
|
|
display worker groups, pull and push worker groups configs
|
|
|
|
**Subcommands:**
|
|
|
|
- \`worker-groups pull\` - Pull worker groups (similar to \`wmill instance pull --skip-users --skip-settings --skip-groups\`)
|
|
- \`--instance\` - Name of the instance to push to, override the active instance
|
|
- \`--base-url\` - Base url to be passed to the instance settings instead of the local one
|
|
- \`--yes\` - Pull without needing confirmation
|
|
- \`worker-groups push\` - Push worker groups (similar to \`wmill instance push --skip-users --skip-settings --skip-groups\`)
|
|
- \`--instance [instance]\` - Name of the instance to push to, override the active instance
|
|
- \`--base-url [baseUrl]\` - If used with --token, will be used as the base url for the instance
|
|
- \`--yes\` - Push without needing confirmation
|
|
|
|
### workers
|
|
|
|
List all workers grouped by worker groups
|
|
|
|
**Options:**
|
|
- \`--instance [instance]\` - Name of the instance to push to, override the active instance
|
|
- \`--base-url [baseUrl]\` - If used with --token, will be used as the base url for the instance
|
|
|
|
### workspace
|
|
|
|
workspace related commands
|
|
|
|
**Alias:** \`profile\`
|
|
|
|
**Subcommands:**
|
|
|
|
- \`workspace switch <workspace_name:string:workspace>\` - Switch to another workspace
|
|
- \`workspace add [workspace_name:string] [workspace_id:string] [remote:string]\` - Add a workspace
|
|
- \`-c --create\` - Create the workspace if it does not exist
|
|
- \`--create-workspace-name <workspace_name:string>\` - Specify the workspace name. Ignored if --create is not specified or the workspace already exists. Will default to the workspace id.
|
|
- \`--create-username <username:string>\` - Specify your own username in the newly created workspace. Ignored if --create is not specified, the workspace already exists or automatic username creation is enabled on the instance.
|
|
- \`workspace remove <workspace_name:string>\` - Remove a workspace
|
|
- \`workspace whoami\` - Show the currently active user
|
|
- \`workspace list\` - List local workspace profiles
|
|
- \`workspace list-remote\` - List workspaces on the remote server that you have access to
|
|
- \`--as-superadmin\` - List ALL workspaces on the instance (requires the token to belong to a superadmin/devops user)
|
|
- \`workspace list-forks\` - List forked workspaces on the remote server
|
|
- \`workspace bind\` - Create or update a workspace entry in wmill.yaml from the active profile
|
|
- \`--workspace <name:string>\` - Workspace name (default: current branch or workspaceId)
|
|
- \`--branch <branch:string>\` - Git branch to associate (default: workspace name)
|
|
- \`workspace unbind\` - Remove baseUrl and workspaceId from a workspace entry
|
|
- \`--workspace <name:string>\` - Workspace to unbind
|
|
- \`workspace fork [workspace_name:string] [workspace_id:string]\` - Create a forked workspace from its parent workspace.
|
|
|
|
The parent is resolved from your current git branch, not from the active profile: run this from a git repo checked out on the branch mapped to the parent workspace in wmill.yaml's \`workspaces:\` section (a fork branch of it resolves to the same parent). \`wmill workspace switch\` does not change which workspace is forked.
|
|
|
|
Arguments (omit both to be prompted interactively):
|
|
[workspace_name] Friendly display name for the fork, shown in the UI. May contain spaces, so quote it in the shell (e.g. "My Fork"). Max 50 chars. Defaults to "<parent workspace name>'s fork".
|
|
[workspace_id] Id for the fork. Must be a slug (no spaces or special characters) and is automatically prefixed with \`wm-fork-\`, so pass just the bare slug (e.g. \`my-fork\` becomes \`wm-fork-my-fork\`). This id also determines the fork's git branch name. Defaults to a slug derived from the name — or, when you are converting an existing branch into the fork branch, from that branch.
|
|
- \`--create-workspace-name <workspace_name:string>\` - Specify the workspace name. Ignored if --create is not specified or the workspace already exists. Will default to the workspace id.
|
|
- \`--color <color:string>\` - Workspace color (hex code, e.g. #ff0000)
|
|
- \`--datatable-behavior <behavior:string>\` - How to handle datatables: skip, schema_only, or schema_and_data (default: interactive prompt)
|
|
- \`--from-branch <branch:string>\` - Non-interactive override for the 'turn my current working branch into the fork' workflow: base the fork on <branch> (its bound workspace is the parent) and rename the current branch onto wm-fork/<branch>/<id>. Usually unneeded — from a working branch \`wmill workspace fork\` offers this interactively; from a base branch it creates a fresh fork branch.
|
|
- \`-y --yes\` - Skip interactive prompts (defaults datatable behavior to 'skip'). On a non-base branch, requires --from-branch since the base branch can't be prompted for.
|
|
- \`workspace delete-fork <fork_name:string>\` - Delete a forked workspace
|
|
- \`-y --yes\` - Skip confirmation prompt
|
|
- \`workspace merge\` - Compare and deploy changes between a fork and its parent workspace
|
|
- \`--direction <direction:string>\` - Deploy direction: to-parent or to-fork
|
|
- \`--all\` - Deploy all changed items including conflicts
|
|
- \`--skip-conflicts\` - Skip items modified in both workspaces
|
|
- \`--include <items:string>\` - Comma-separated kind:path items to include (e.g. script:f/test/main,flow:f/my/flow)
|
|
- \`--exclude <items:string>\` - Comma-separated kind:path items to exclude
|
|
- \`--preserve-on-behalf-of\` - Preserve original on_behalf_of/permissioned_as values
|
|
- \`-y --yes\` - Non-interactive mode (deploy without prompts)
|
|
- \`workspace connect-slack\` - Non-interactively connect Slack to the active workspace using a pre-minted bot token (xoxb-...). Produces the same artifacts as the UI OAuth flow: workspace_settings fields, g/slack group, f/slack_bot folder, and the encrypted bot token variable + resource at f/slack_bot/bot_token.
|
|
- \`--bot-token <bot_token:string>\` - Slack bot token (xoxb-...)
|
|
- \`--team-id <team_id:string>\` - Slack team id
|
|
- \`--team-name <team_name:string>\` - Slack team name
|
|
- \`workspace disconnect-slack\` - Clear slack_team_id / slack_name on the active workspace (marks the workspace as disconnected). Does NOT remove the bot token variable/resource/folder/group — delete those from the local sync folder and run 'wmill sync push' to tear them down. Does NOT remove the workspace-level OAuth override — set slack_oauth_client_id/_secret to '' in settings.yaml and push.
|
|
|
|
|
|
|
|
# Object Storage CLI
|
|
|
|
\`wmill object-storage\` (alias \`wmill s3\`) exposes the workspace's object storage (S3-compatible: AWS S3, MinIO, GCS, R2, Azure Blob) over the per-workspace \`/job_helpers/*\` endpoints.
|
|
|
|
## Key concepts (not obvious from per-command --help)
|
|
|
|
- **\`file_key\` is the path inside the bucket** (e.g. \`reports/2026-05/orders.csv\`), not a Windmill path. Do NOT pass \`u/...\` or \`f/...\` here — those are Windmill paths to scripts/flows/resources, unrelated to objects in the bucket.
|
|
- **Scope is the active workspace.** Object storage is configured per-workspace (default storage + optional secondary storages). Switching workspaces switches which bucket the commands target.
|
|
- **\`--storage <name>\` targets a secondary storage** configured on the workspace. Omit it to use the workspace's default object storage. Use \`wmill object-storage list\` to discover configured storages.
|
|
- **\`preview\` vs \`download\`**: \`preview\` returns a peek (CSV first rows, text content, or a byte slice via \`--bytes-from\`/\`--bytes-length\`) without writing to disk. Use \`download\` when you want the full file on disk.
|
|
|
|
## Choosing a subcommand
|
|
|
|
- Look at what's there: \`wmill object-storage files [prefix]\` (alias \`ls\`) — paginated, use \`--marker\` to continue.
|
|
- Inspect one file: \`wmill object-storage info <file_key>\` for size/mime/last-modified, \`wmill object-storage preview <file_key>\` for content peek.
|
|
- Move data in: \`wmill object-storage upload <local_path> <file_key>\` — set \`--content-type\` if the receiver cares (e.g. \`text/csv\`).
|
|
- Move data out: \`wmill object-storage download <file_key> [output_path]\` — \`--stdout\` to pipe.
|
|
- Reorganize: \`wmill object-storage move <src> <dest>\` (same storage), \`wmill object-storage delete <file_key>\` (interactive confirm unless \`--yes\`).
|
|
`;
|
|
|
|
export const LANG_ANSIBLE = `# Ansible
|
|
|
|
Windmill runs Ansible playbooks with \`ansible-playbook\`. A script is a single YAML
|
|
document made of two parts separated by a \`---\` line: a Windmill **header** and one or
|
|
more standard Ansible **plays**.
|
|
|
|
## Structure
|
|
|
|
\`\`\`yaml
|
|
---
|
|
# Windmill header: configures inventories, file resources, arguments and dependencies
|
|
extra_vars:
|
|
world_qualifier:
|
|
type: string
|
|
dependencies:
|
|
galaxy:
|
|
collections:
|
|
- name: community.general
|
|
python:
|
|
- jmespath
|
|
---
|
|
# Standard Ansible plays
|
|
- name: Echo
|
|
hosts: 127.0.0.1
|
|
connection: local
|
|
tasks:
|
|
- name: Print debug message
|
|
debug:
|
|
msg: "Hello, {{ world_qualifier }} world!"
|
|
\`\`\`
|
|
|
|
## Header
|
|
|
|
The header is **not** standard Ansible — it is parsed by Windmill to build the script's
|
|
inputs and runtime environment. Supported keys:
|
|
|
|
- \`extra_vars\`: defines the script arguments. Each entry is passed to the playbook via
|
|
\`--extra-vars\` and becomes a Jinja variable usable as \`{{ name }}\` in the plays. Give
|
|
each argument a \`type\` (\`string\`, \`number\`, \`boolean\`, \`object\`, ...) so Windmill can
|
|
generate the input form.
|
|
- \`inventory\`: lists inventories. Use \`resource_type: ansible_inventory\` (optionally
|
|
pinned with \`resource: u/user/your_resource\`) or \`resource_type: dynamic_inventory\`.
|
|
- \`files\`: writes Windmill resources/variables to files before the run, e.g.
|
|
\`- resource: u/user/template\` with \`target: ./config.j2\`, or
|
|
\`- variable: u/user/ssh_key\` with \`target: ./ssh_key\` and \`mode: '0600'\`.
|
|
- \`dependencies\`: \`galaxy\` collections/roles (installed with \`ansible-galaxy\`) and
|
|
\`python\` pip packages available to the playbook.
|
|
- \`options\`: extra \`ansible-playbook\` flags such as \`- verbosity: vvv\`.
|
|
- \`vault_password\`: a Windmill variable path to use as the Ansible Vault password.
|
|
|
|
## Arguments
|
|
|
|
Reference header \`extra_vars\` directly as Jinja variables in the plays:
|
|
|
|
\`\`\`yaml
|
|
extra_vars:
|
|
name:
|
|
type: string
|
|
count:
|
|
type: number
|
|
---
|
|
- hosts: localhost
|
|
tasks:
|
|
- debug:
|
|
msg: "{{ name }} x {{ count }}"
|
|
\`\`\`
|
|
|
|
## Environment variables
|
|
|
|
Windmill contextual variables are available as environment variables and read with the
|
|
\`env\` lookup:
|
|
|
|
\`\`\`yaml
|
|
- debug:
|
|
msg: "Running in workspace {{ lookup('env', 'WM_WORKSPACE') }}"
|
|
\`\`\`
|
|
|
|
## Output
|
|
|
|
To return a result, write JSON to a \`result.json\` file in the job directory:
|
|
|
|
\`\`\`yaml
|
|
- hosts: localhost
|
|
tasks:
|
|
- name: Write result
|
|
copy:
|
|
content: "{{ { 'ok': true, 'value': 42 } | to_json }}"
|
|
dest: result.json
|
|
\`\`\`
|
|
`;
|
|
|
|
export const LANG_BASH = `# Bash
|
|
|
|
## Structure
|
|
|
|
Do not include \`#!/bin/bash\`. Arguments are obtained as positional parameters:
|
|
|
|
\`\`\`bash
|
|
# Get arguments
|
|
var1="$1"
|
|
var2="$2"
|
|
|
|
echo "Processing $var1 and $var2"
|
|
|
|
# Return JSON by echoing to stdout
|
|
echo "{\\"result\\": \\"$var1\\", \\"count\\": $var2}"
|
|
\`\`\`
|
|
|
|
**Important:**
|
|
- Do not include shebang (\`#!/bin/bash\`)
|
|
- Arguments are always strings
|
|
- Access with \`$1\`, \`$2\`, etc.
|
|
|
|
## Output
|
|
|
|
The script output is captured as the result. For structured data, output valid JSON:
|
|
|
|
\`\`\`bash
|
|
name="$1"
|
|
count="$2"
|
|
|
|
# Output JSON result
|
|
cat << EOF
|
|
{
|
|
"name": "$name",
|
|
"count": $count,
|
|
"timestamp": "$(date -Iseconds)"
|
|
}
|
|
EOF
|
|
\`\`\`
|
|
|
|
## Environment Variables
|
|
|
|
Environment variables set in Windmill are available:
|
|
|
|
\`\`\`bash
|
|
# Access environment variable
|
|
echo "Workspace: $WM_WORKSPACE"
|
|
echo "Job ID: $WM_JOB_ID"
|
|
\`\`\`
|
|
`;
|
|
|
|
export const LANG_BIGQUERY = `# BigQuery
|
|
|
|
Arguments use \`@name\` syntax.
|
|
|
|
Name the parameters by adding comments before the statement:
|
|
|
|
\`\`\`sql
|
|
-- @name1 (string)
|
|
-- @name2 (int64) = 0
|
|
SELECT * FROM users WHERE name = @name1 AND age > @name2;
|
|
\`\`\`
|
|
|
|
## Receiving an S3Object as a script parameter
|
|
|
|
Declare the arg with type \`(s3object)\`. Windmill renders an S3 file picker for
|
|
it, downloads the file, and binds it as a \`STRING\` JSON parameter — Parquet/CSV
|
|
files are decoded server-side into a JSON array of records, JSON/JSONL pass
|
|
through. Consume with \`JSON_EXTRACT_ARRAY\` / \`JSON_VALUE\`:
|
|
|
|
\`\`\`sql
|
|
-- @file (s3object)
|
|
SELECT
|
|
CAST(JSON_VALUE(row, '$.id') AS INT64) AS id,
|
|
JSON_VALUE(row, '$.name') AS name
|
|
FROM UNNEST(JSON_EXTRACT_ARRAY(@file)) AS row;
|
|
\`\`\`
|
|
|
|
## Streaming query results to S3
|
|
|
|
Add a \`-- s3\` directive at the top of the script to stream the result set to S3
|
|
instead of returning rows. Windmill writes the file and returns its \`S3Object\`
|
|
as the script result.
|
|
|
|
\`\`\`sql
|
|
-- s3 prefix=exports/users format=parquet
|
|
SELECT id, name FROM users;
|
|
\`\`\`
|
|
|
|
All keys are optional: \`prefix\` (object key prefix), \`storage\` (named storage —
|
|
omit to use the workspace default), \`format\` (\`json\` (default), \`parquet\`, or
|
|
\`csv\`). Use this for large result sets — rows stream directly to S3 instead of
|
|
being buffered, bypassing the 10000-row return cap.
|
|
`;
|
|
|
|
export const LANG_BUN = `# TypeScript (Bun)
|
|
|
|
Bun runtime with full npm ecosystem and fastest execution. **Bun is the default and preferred TypeScript runtime** — choose it for any TypeScript script unless there is a major reason to use Deno for that specific use-case.
|
|
|
|
## Structure
|
|
|
|
Export a single **async** function called \`main\`:
|
|
|
|
\`\`\`typescript
|
|
export async function main(param1: string, param2: number) {
|
|
// Your code here
|
|
return { result: param1, count: param2 };
|
|
}
|
|
\`\`\`
|
|
|
|
Do not call the main function. Libraries are installed automatically.
|
|
|
|
## Resource Types
|
|
|
|
On Windmill, credentials and configuration are stored in resources and passed as parameters to main.
|
|
|
|
Use the \`RT\` namespace for resource types:
|
|
|
|
\`\`\`typescript
|
|
export async function main(stripe: RT.Stripe) {
|
|
// stripe contains API key and config from the resource
|
|
}
|
|
\`\`\`
|
|
|
|
Only use resource types if you need them to satisfy the instructions. Always use the RT namespace.
|
|
|
|
Before using a resource type, check the \`rt.d.ts\` file in the project root to see all available resource types and their fields. This file is generated by \`wmill resource-type generate-namespace\`.
|
|
|
|
## Imports
|
|
|
|
\`\`\`typescript
|
|
import Stripe from "stripe";
|
|
import { someFunction } from "some-package";
|
|
\`\`\`
|
|
|
|
## Prefer \`//native\` when the runtime allows it
|
|
|
|
If a script only needs \`fetch\` and the JavaScript standard library — including when it uses \`windmill-client\` — prefer making it a **native** script: add \`//native\` as the first line and write it with the \`write-script-bunnative\` skill. Native scripts run on a lightweight V8 isolate, start faster, and parallelize heavily. \`windmill-client\` works on the native worker (its calls go over \`fetch\`), so needing the Windmill client is **not** a reason to avoid \`//native\`. Use the regular \`bun\` language only when the code (or a dependency) needs Node/Bun runtime APIs — \`node:*\` modules, the filesystem, child processes, or native addons.
|
|
|
|
## Windmill Client
|
|
|
|
Import the windmill client for platform interactions:
|
|
|
|
\`\`\`typescript
|
|
import * as wmill from "windmill-client";
|
|
\`\`\`
|
|
|
|
**Prefer \`windmill-client\` over raw \`fetch\` for anything that talks to Windmill** — reading resources/variables/states, running scripts and flows, S3 object operations, etc. It handles auth, the workspace, and the base URL for you, so you don't hand-roll URLs or tokens. Reserve \`fetch\` for calling *external* HTTP APIs that aren't Windmill.
|
|
|
|
The full \`windmill-client\` API reference (every exported function and its signature) is included in this skill below — consult it for the exact method to use instead of guessing or falling back to \`fetch\`.
|
|
|
|
## Preprocessor Scripts
|
|
|
|
For preprocessor scripts, the function should be named \`preprocessor\` and receives an \`event\` parameter:
|
|
|
|
\`\`\`typescript
|
|
type Event = {
|
|
kind:
|
|
| "webhook"
|
|
| "http"
|
|
| "websocket"
|
|
| "kafka"
|
|
| "email"
|
|
| "nats"
|
|
| "postgres"
|
|
| "sqs"
|
|
| "mqtt"
|
|
| "gcp";
|
|
body: any;
|
|
headers: Record<string, string>;
|
|
query: Record<string, string>;
|
|
};
|
|
|
|
export async function preprocessor(event: Event) {
|
|
return {
|
|
param1: event.body.field1,
|
|
param2: event.query.id,
|
|
};
|
|
}
|
|
\`\`\`
|
|
|
|
## S3 Object Operations
|
|
|
|
Windmill provides built-in support for S3-compatible storage operations. The \`wmill.S3Object\` type covers both the \`s3://storage/key\` URI form (\`s3:///key\` for the workspace default storage) and the \`{ s3, storage? }\` record form — always use it instead of redefining your own.
|
|
|
|
### Receiving an S3Object as a script parameter
|
|
|
|
\`\`\`typescript
|
|
import * as wmill from "windmill-client";
|
|
|
|
export async function main(file: wmill.S3Object) {
|
|
const content = await wmill.loadS3File(file);
|
|
// ...
|
|
}
|
|
\`\`\`
|
|
|
|
### S3 operations
|
|
|
|
\`\`\`typescript
|
|
import * as wmill from "windmill-client";
|
|
|
|
// Load file content from S3
|
|
const content: Uint8Array = await wmill.loadS3File(s3object);
|
|
|
|
// Load file as stream
|
|
const blob: Blob = await wmill.loadS3FileStream(s3object);
|
|
|
|
// Write file to S3
|
|
const result: wmill.S3Object = await wmill.writeS3File(
|
|
s3object, // Target path (or undefined to auto-generate)
|
|
fileContent, // string or Blob
|
|
s3ResourcePath // Optional: specific S3 resource to use
|
|
);
|
|
\`\`\`
|
|
`;
|
|
|
|
export const LANG_BUNNATIVE = `# TypeScript (Bun Native)
|
|
|
|
Native TypeScript execution. Native scripts are Bun scripts that run on the native worker — a lightweight V8 isolate that exposes \`fetch\` and the JavaScript standard library — and can be heavily parallelized. Every script MUST start with \`//native\` on its first line so Windmill routes it to the native worker; without it the exact same script runs on the regular Bun worker. You may import npm packages and other Windmill scripts (e.g. \`./helper.ts\`) — imports are resolved and bundled just like a regular Bun script — as long as everything (your code and its dependencies) relies only on \`fetch\` and the standard library. Libraries that need Node/Bun runtime APIs (filesystem, \`node:*\` modules, child processes, native addons) will not work on the native worker; use the regular \`bun\` language for those.
|
|
|
|
## Structure
|
|
|
|
Export a single **async** function called \`main\`:
|
|
|
|
\`\`\`typescript
|
|
//native
|
|
export async function main(param1: string, param2: number) {
|
|
// Your code here
|
|
return { result: param1, count: param2 };
|
|
}
|
|
\`\`\`
|
|
|
|
Do not call the main function.
|
|
|
|
## Resource Types
|
|
|
|
On Windmill, credentials and configuration are stored in resources and passed as parameters to main.
|
|
|
|
Use the \`RT\` namespace for resource types:
|
|
|
|
\`\`\`typescript
|
|
//native
|
|
export async function main(stripe: RT.Stripe) {
|
|
// stripe contains API key and config from the resource
|
|
}
|
|
\`\`\`
|
|
|
|
Only use resource types if you need them to satisfy the instructions. Always use the RT namespace.
|
|
|
|
Before using a resource type, check the \`rt.d.ts\` file in the project root to see all available resource types and their fields. This file is generated by \`wmill resource-type generate-namespace\`.
|
|
|
|
## Imports
|
|
|
|
**The constraint is the runtime, not the import list.** You may import npm packages and relative Windmill scripts; they are resolved and bundled exactly like a regular Bun script. But the native worker only provides \`fetch\` and the JavaScript standard library, so any imported code must work using only those. Anything requiring Node/Bun built-ins (\`node:fs\`, \`child_process\`, the \`Bun\` API, native modules) belongs in a regular \`bun\` script instead. Use the globally available \`fetch\` for HTTP:
|
|
|
|
\`\`\`typescript
|
|
//native
|
|
export async function main(url: string) {
|
|
const response = await fetch(url);
|
|
return await response.json();
|
|
}
|
|
\`\`\`
|
|
|
|
## Windmill Client
|
|
|
|
\`windmill-client\` works on the native worker (its calls go over \`fetch\`), so use it as the **preferred way to talk to Windmill** — reading resources/variables/states, running scripts and flows, and the S3 helpers below (\`loadS3File\`, \`loadS3FileStream\`, \`writeS3File\`, \`S3Object\`). It handles auth, the workspace, and the base URL for you. Reserve raw \`fetch\` for calling *external* HTTP APIs that aren't Windmill.
|
|
|
|
The full \`windmill-client\` API reference (every exported function and its signature) is included in this skill below — consult it for the exact method instead of hand-rolling a \`fetch\` against the Windmill API.
|
|
|
|
## Preprocessor Scripts
|
|
|
|
For preprocessor scripts, the function should be named \`preprocessor\` and receives an \`event\` parameter:
|
|
|
|
\`\`\`typescript
|
|
//native
|
|
type Event = {
|
|
kind:
|
|
| "webhook"
|
|
| "http"
|
|
| "websocket"
|
|
| "kafka"
|
|
| "email"
|
|
| "nats"
|
|
| "postgres"
|
|
| "sqs"
|
|
| "mqtt"
|
|
| "gcp";
|
|
body: any;
|
|
headers: Record<string, string>;
|
|
query: Record<string, string>;
|
|
};
|
|
|
|
export async function preprocessor(event: Event) {
|
|
return {
|
|
param1: event.body.field1,
|
|
param2: event.query.id,
|
|
};
|
|
}
|
|
\`\`\`
|
|
|
|
## S3 Object Operations
|
|
|
|
Windmill provides built-in support for S3-compatible storage operations. The \`wmill.S3Object\` type covers both the \`s3://storage/key\` URI form (\`s3:///key\` for the workspace default storage) and the \`{ s3, storage? }\` record form — always use it instead of redefining your own.
|
|
|
|
### Receiving an S3Object as a script parameter
|
|
|
|
\`\`\`typescript
|
|
//native
|
|
import * as wmill from "windmill-client";
|
|
|
|
export async function main(file: wmill.S3Object) {
|
|
const content = await wmill.loadS3File(file);
|
|
// ...
|
|
}
|
|
\`\`\`
|
|
|
|
### S3 operations
|
|
|
|
\`\`\`typescript
|
|
//native
|
|
import * as wmill from "windmill-client";
|
|
|
|
// Load file content from S3
|
|
const content: Uint8Array = await wmill.loadS3File(s3object);
|
|
|
|
// Load file as stream
|
|
const blob: Blob = await wmill.loadS3FileStream(s3object);
|
|
|
|
// Write file to S3
|
|
const result: wmill.S3Object = await wmill.writeS3File(
|
|
s3object, // Target path (or undefined to auto-generate)
|
|
fileContent, // string or Blob
|
|
s3ResourcePath // Optional: specific S3 resource to use
|
|
);
|
|
\`\`\`
|
|
`;
|
|
|
|
export const LANG_CSHARP = `# C#
|
|
|
|
The script must contain a public static \`Main\` method inside a class:
|
|
|
|
\`\`\`csharp
|
|
public class Script
|
|
{
|
|
public static object Main(string name, int count)
|
|
{
|
|
return new { Name = name, Count = count };
|
|
}
|
|
}
|
|
\`\`\`
|
|
|
|
**Important:**
|
|
- Class name is irrelevant
|
|
- Method must be \`public static\`
|
|
- Return type can be \`object\` or specific type
|
|
|
|
## NuGet Packages
|
|
|
|
Add packages using the \`#r\` directive at the top:
|
|
|
|
\`\`\`csharp
|
|
#r "nuget: Newtonsoft.Json, 13.0.3"
|
|
#r "nuget: RestSharp, 110.2.0"
|
|
|
|
using Newtonsoft.Json;
|
|
using RestSharp;
|
|
|
|
public class Script
|
|
{
|
|
public static object Main(string url)
|
|
{
|
|
var client = new RestClient(url);
|
|
var request = new RestRequest();
|
|
var response = client.Get(request);
|
|
return JsonConvert.DeserializeObject(response.Content);
|
|
}
|
|
}
|
|
\`\`\`
|
|
`;
|
|
|
|
export const LANG_DENO = `# TypeScript (Deno)
|
|
|
|
Deno runtime with npm support via \`npm:\` prefix and native Deno libraries.
|
|
|
|
**Prefer Bun (\`write-script-bun\`) for TypeScript.** Only use Deno when the script specifically requires the Deno runtime — Deno's standard library or \`deno.land\` URL imports that have no npm equivalent. For all other TypeScript, use Bun instead.
|
|
|
|
## Structure
|
|
|
|
Export a single **async** function called \`main\`:
|
|
|
|
\`\`\`typescript
|
|
export async function main(param1: string, param2: number) {
|
|
// Your code here
|
|
return { result: param1, count: param2 };
|
|
}
|
|
\`\`\`
|
|
|
|
Do not call the main function. Libraries are installed automatically.
|
|
|
|
## Resource Types
|
|
|
|
On Windmill, credentials and configuration are stored in resources and passed as parameters to main.
|
|
|
|
Use the \`RT\` namespace for resource types:
|
|
|
|
\`\`\`typescript
|
|
export async function main(stripe: RT.Stripe) {
|
|
// stripe contains API key and config from the resource
|
|
}
|
|
\`\`\`
|
|
|
|
Only use resource types if you need them to satisfy the instructions. Always use the RT namespace.
|
|
|
|
Before using a resource type, check the \`rt.d.ts\` file in the project root to see all available resource types and their fields. This file is generated by \`wmill resource-type generate-namespace\`.
|
|
|
|
## Imports
|
|
|
|
\`\`\`typescript
|
|
// npm packages use npm: prefix
|
|
import Stripe from "npm:stripe";
|
|
import { someFunction } from "npm:some-package";
|
|
|
|
// Deno standard library
|
|
import { serve } from "https://deno.land/std/http/server.ts";
|
|
\`\`\`
|
|
|
|
## Windmill Client
|
|
|
|
Import the windmill client for platform interactions:
|
|
|
|
\`\`\`typescript
|
|
import * as wmill from "windmill-client";
|
|
\`\`\`
|
|
|
|
**Prefer \`windmill-client\` over raw \`fetch\` for anything that talks to Windmill** — reading resources/variables/states, running scripts and flows, S3 object operations, etc. It handles auth, the workspace, and the base URL for you. Reserve \`fetch\` for calling *external* HTTP APIs that aren't Windmill.
|
|
|
|
The full \`windmill-client\` API reference (every exported function and its signature) is included in this skill below — consult it for the exact method instead of guessing or falling back to \`fetch\`.
|
|
|
|
## Preprocessor Scripts
|
|
|
|
For preprocessor scripts, the function should be named \`preprocessor\` and receives an \`event\` parameter:
|
|
|
|
\`\`\`typescript
|
|
type Event = {
|
|
kind:
|
|
| "webhook"
|
|
| "http"
|
|
| "websocket"
|
|
| "kafka"
|
|
| "email"
|
|
| "nats"
|
|
| "postgres"
|
|
| "sqs"
|
|
| "mqtt"
|
|
| "gcp";
|
|
body: any;
|
|
headers: Record<string, string>;
|
|
query: Record<string, string>;
|
|
};
|
|
|
|
export async function preprocessor(event: Event) {
|
|
return {
|
|
param1: event.body.field1,
|
|
param2: event.query.id,
|
|
};
|
|
}
|
|
\`\`\`
|
|
|
|
## S3 Object Operations
|
|
|
|
Windmill provides built-in support for S3-compatible storage operations. The \`wmill.S3Object\` type covers both the \`s3://storage/key\` URI form (\`s3:///key\` for the workspace default storage) and the \`{ s3, storage? }\` record form — always use it instead of redefining your own.
|
|
|
|
### Receiving an S3Object as a script parameter
|
|
|
|
\`\`\`typescript
|
|
import * as wmill from "windmill-client";
|
|
|
|
export async function main(file: wmill.S3Object) {
|
|
const content = await wmill.loadS3File(file);
|
|
// ...
|
|
}
|
|
\`\`\`
|
|
|
|
### S3 operations
|
|
|
|
\`\`\`typescript
|
|
import * as wmill from "windmill-client";
|
|
|
|
// Load file content from S3
|
|
const content: Uint8Array = await wmill.loadS3File(s3object);
|
|
|
|
// Load file as stream
|
|
const blob: Blob = await wmill.loadS3FileStream(s3object);
|
|
|
|
// Write file to S3
|
|
const result: wmill.S3Object = await wmill.writeS3File(
|
|
s3object, // Target path (or undefined to auto-generate)
|
|
fileContent, // string or Blob
|
|
s3ResourcePath // Optional: specific S3 resource to use
|
|
);
|
|
\`\`\`
|
|
`;
|
|
|
|
export const LANG_DUCKDB = `# DuckDB
|
|
|
|
Arguments are defined with comments and used with \`$name\` syntax:
|
|
|
|
\`\`\`sql
|
|
-- $name (text) = default
|
|
-- $age (integer)
|
|
SELECT * FROM users WHERE name = $name AND age > $age;
|
|
\`\`\`
|
|
|
|
## Ducklake Integration
|
|
|
|
Attach Ducklake for data lake operations:
|
|
|
|
\`\`\`sql
|
|
-- Main ducklake
|
|
ATTACH 'ducklake' AS dl;
|
|
|
|
-- Named ducklake
|
|
ATTACH 'ducklake://my_lake' AS dl;
|
|
|
|
-- Then query
|
|
SELECT * FROM dl.schema.table;
|
|
\`\`\`
|
|
|
|
## External Database Connections
|
|
|
|
Connect to external databases using resources:
|
|
|
|
\`\`\`sql
|
|
ATTACH '$res:path/to/resource' AS db (TYPE postgres);
|
|
SELECT * FROM db.schema.table;
|
|
\`\`\`
|
|
|
|
## S3 File Operations
|
|
|
|
Read files from S3 storage:
|
|
|
|
\`\`\`sql
|
|
-- Default storage
|
|
SELECT * FROM read_csv('s3:///path/to/file.csv');
|
|
|
|
-- Named storage
|
|
SELECT * FROM read_csv('s3://storage_name/path/to/file.csv');
|
|
|
|
-- Parquet files
|
|
SELECT * FROM read_parquet('s3:///path/to/file.parquet');
|
|
|
|
-- JSON files
|
|
SELECT * FROM read_json('s3:///path/to/file.json');
|
|
\`\`\`
|
|
|
|
### Receiving an S3Object as a script parameter
|
|
|
|
Declare the arg with type \`(s3object)\`. Windmill renders an S3 file picker for it
|
|
and binds the arg as the bare \`s3://storage/key\` URI, which DuckDB's reader
|
|
functions consume directly:
|
|
|
|
\`\`\`sql
|
|
-- $file (s3object)
|
|
SELECT * FROM read_parquet($file);
|
|
\`\`\`
|
|
|
|
Works with any DuckDB reader: \`read_csv($file)\`, \`read_json($file)\`, etc.
|
|
|
|
### Writing query results to S3
|
|
|
|
DuckDB writes to S3 natively via \`COPY ... TO\`:
|
|
|
|
\`\`\`sql
|
|
COPY (SELECT * FROM users) TO 's3:///exports/users.parquet' (FORMAT PARQUET);
|
|
\`\`\`
|
|
|
|
Use this instead of the \`-- s3\` streaming directive supported by the other SQL
|
|
dialects — that directive is not available in DuckDB.
|
|
`;
|
|
|
|
export const LANG_GO = `# Go
|
|
|
|
## Structure
|
|
|
|
The file package must be \`inner\` and export a function called \`main\`:
|
|
|
|
\`\`\`go
|
|
package inner
|
|
|
|
func main(param1 string, param2 int) (map[string]interface{}, error) {
|
|
return map[string]interface{}{
|
|
"result": param1,
|
|
"count": param2,
|
|
}, nil
|
|
}
|
|
\`\`\`
|
|
|
|
**Important:**
|
|
- Package must be \`inner\`
|
|
- Return type must be \`({return_type}, error)\`
|
|
- Function name is \`main\` (lowercase)
|
|
|
|
## Return Types
|
|
|
|
The return type can be any Go type that can be serialized to JSON:
|
|
|
|
\`\`\`go
|
|
package inner
|
|
|
|
type Result struct {
|
|
Name string \`json:"name"\`
|
|
Count int \`json:"count"\`
|
|
}
|
|
|
|
func main(name string, count int) (Result, error) {
|
|
return Result{
|
|
Name: name,
|
|
Count: count,
|
|
}, nil
|
|
}
|
|
\`\`\`
|
|
|
|
## Error Handling
|
|
|
|
Return errors as the second return value:
|
|
|
|
\`\`\`go
|
|
package inner
|
|
|
|
import "errors"
|
|
|
|
func main(value int) (string, error) {
|
|
if value < 0 {
|
|
return "", errors.New("value must be positive")
|
|
}
|
|
return "success", nil
|
|
}
|
|
\`\`\`
|
|
`;
|
|
|
|
export const LANG_GRAPHQL = `# GraphQL
|
|
|
|
## Structure
|
|
|
|
Write GraphQL queries or mutations. Arguments can be added as query parameters:
|
|
|
|
\`\`\`graphql
|
|
query GetUser($id: ID!) {
|
|
user(id: $id) {
|
|
id
|
|
name
|
|
email
|
|
}
|
|
}
|
|
\`\`\`
|
|
|
|
## Variables
|
|
|
|
Variables are passed as script arguments and automatically bound to the query:
|
|
|
|
\`\`\`graphql
|
|
query SearchProducts($query: String!, $limit: Int = 10) {
|
|
products(search: $query, first: $limit) {
|
|
edges {
|
|
node {
|
|
id
|
|
name
|
|
price
|
|
}
|
|
}
|
|
}
|
|
}
|
|
\`\`\`
|
|
|
|
## Mutations
|
|
|
|
\`\`\`graphql
|
|
mutation CreateUser($input: CreateUserInput!) {
|
|
createUser(input: $input) {
|
|
id
|
|
name
|
|
createdAt
|
|
}
|
|
}
|
|
\`\`\`
|
|
`;
|
|
|
|
export const LANG_JAVA = `# Java
|
|
|
|
The script must contain a Main public class with a \`public static main()\` method:
|
|
|
|
\`\`\`java
|
|
public class Main {
|
|
public static Object main(String name, int count) {
|
|
java.util.Map<String, Object> result = new java.util.HashMap<>();
|
|
result.put("name", name);
|
|
result.put("count", count);
|
|
return result;
|
|
}
|
|
}
|
|
\`\`\`
|
|
|
|
**Important:**
|
|
- Class must be named \`Main\`
|
|
- Method must be \`public static Object main(...)\`
|
|
- Return type is \`Object\` or \`void\`
|
|
|
|
## Maven Dependencies
|
|
|
|
Add dependencies using comments at the top:
|
|
|
|
\`\`\`java
|
|
//requirements:
|
|
//com.google.code.gson:gson:2.10.1
|
|
//org.apache.httpcomponents:httpclient:4.5.14
|
|
|
|
import com.google.gson.Gson;
|
|
|
|
public class Main {
|
|
public static Object main(String input) {
|
|
Gson gson = new Gson();
|
|
return gson.fromJson(input, Object.class);
|
|
}
|
|
}
|
|
\`\`\`
|
|
`;
|
|
|
|
export const LANG_MSSQL = `# Microsoft SQL Server (MSSQL)
|
|
|
|
Arguments use \`@P1\`, \`@P2\`, etc.
|
|
|
|
Name the parameters by adding comments before the statement:
|
|
|
|
\`\`\`sql
|
|
-- @P1 name1 (varchar)
|
|
-- @P2 name2 (int) = 0
|
|
SELECT * FROM users WHERE name = @P1 AND age > @P2;
|
|
\`\`\`
|
|
|
|
## Receiving an S3Object as a script parameter
|
|
|
|
Declare the arg with type \`(s3object)\`. Windmill renders an S3 file picker for
|
|
it, downloads the file, and binds it as \`nvarchar(max)\` JSON text — Parquet/CSV
|
|
files are decoded server-side into a JSON array of records, JSON/JSONL pass
|
|
through. Consume with \`OPENJSON\`:
|
|
|
|
\`\`\`sql
|
|
-- @P1 file (s3object)
|
|
SELECT id, name
|
|
FROM OPENJSON(@P1)
|
|
WITH (id INT, name NVARCHAR(200));
|
|
\`\`\`
|
|
|
|
## Streaming query results to S3
|
|
|
|
Add a \`-- s3\` directive at the top of the script to stream the result set to S3
|
|
instead of returning rows. Windmill writes the file and returns its \`S3Object\`
|
|
as the script result.
|
|
|
|
\`\`\`sql
|
|
-- s3 prefix=exports/users format=parquet
|
|
SELECT id, name FROM users;
|
|
\`\`\`
|
|
|
|
All keys are optional: \`prefix\` (object key prefix), \`storage\` (named storage —
|
|
omit to use the workspace default), \`format\` (\`json\` (default), \`parquet\`, or
|
|
\`csv\`). Use this for large result sets — rows stream directly to S3 instead of
|
|
being buffered as the script return value.
|
|
`;
|
|
|
|
export const LANG_MYSQL = `# MySQL
|
|
|
|
Arguments use \`?\` placeholders.
|
|
|
|
Name the parameters by adding comments before the statement:
|
|
|
|
\`\`\`sql
|
|
-- ? name1 (text)
|
|
-- ? name2 (int) = 0
|
|
SELECT * FROM users WHERE name = ? AND age > ?;
|
|
\`\`\`
|
|
|
|
## Receiving an S3Object as a script parameter
|
|
|
|
Declare the arg with type \`(s3object)\`. Windmill renders an S3 file picker for
|
|
it, downloads the file, and binds it as JSON text — Parquet/CSV files are
|
|
decoded server-side into a JSON array of records, JSON/JSONL pass through.
|
|
Consume with \`JSON_TABLE\`:
|
|
|
|
\`\`\`sql
|
|
-- ? file (s3object)
|
|
SELECT id, name
|
|
FROM JSON_TABLE(?, '$[*]'
|
|
COLUMNS (id INT PATH '$.id', name VARCHAR(200) PATH '$.name')
|
|
) AS r;
|
|
\`\`\`
|
|
|
|
## Streaming query results to S3
|
|
|
|
Add a \`-- s3\` directive at the top of the script to stream the result set to S3
|
|
instead of returning rows. Windmill writes the file and returns its \`S3Object\`
|
|
as the script result.
|
|
|
|
\`\`\`sql
|
|
-- s3 prefix=exports/users format=parquet
|
|
SELECT id, name FROM users;
|
|
\`\`\`
|
|
|
|
All keys are optional: \`prefix\` (object key prefix), \`storage\` (named storage —
|
|
omit to use the workspace default), \`format\` (\`json\` (default), \`parquet\`, or
|
|
\`csv\`). Use this for large result sets — rows stream directly to S3 instead of
|
|
being buffered as the script return value.
|
|
`;
|
|
|
|
export const LANG_PHP = `# PHP
|
|
|
|
## Structure
|
|
|
|
The script must start with \`<?php\` and contain at least one function called \`main\`:
|
|
|
|
\`\`\`php
|
|
<?php
|
|
|
|
function main(string $param1, int $param2) {
|
|
return ["result" => $param1, "count" => $param2];
|
|
}
|
|
\`\`\`
|
|
|
|
## Resource Types
|
|
|
|
On Windmill, credentials and configuration are stored in resources and passed as parameters to main.
|
|
|
|
You need to **redefine** the type of the resources that are needed before the main function. Always check if the class already exists using \`class_exists\`:
|
|
|
|
\`\`\`php
|
|
<?php
|
|
|
|
if (!class_exists('Postgresql')) {
|
|
class Postgresql {
|
|
public string $host;
|
|
public int $port;
|
|
public string $user;
|
|
public string $password;
|
|
public string $dbname;
|
|
}
|
|
}
|
|
|
|
function main(Postgresql $db) {
|
|
// $db contains the database connection details
|
|
}
|
|
\`\`\`
|
|
|
|
The resource type name has to be exactly as specified.
|
|
|
|
## Library Dependencies
|
|
|
|
Specify library dependencies as comments before the main function:
|
|
|
|
\`\`\`php
|
|
<?php
|
|
|
|
// require:
|
|
// guzzlehttp/guzzle
|
|
// stripe/stripe-php@^10.0
|
|
|
|
function main() {
|
|
// Libraries are available
|
|
}
|
|
\`\`\`
|
|
|
|
One dependency per line. No need to require autoload, it is already done.
|
|
`;
|
|
|
|
export const LANG_POSTGRESQL = `# PostgreSQL
|
|
|
|
Arguments are obtained directly in the statement with \`$1::{type}\`, \`$2::{type}\`, etc.
|
|
|
|
Name the parameters by adding comments at the beginning of the script (without specifying the type):
|
|
|
|
\`\`\`sql
|
|
-- $1 name1
|
|
-- $2 name2 = default_value
|
|
SELECT * FROM users WHERE name = $1::TEXT AND age > $2::INT;
|
|
\`\`\`
|
|
|
|
## Receiving an S3Object as a script parameter
|
|
|
|
Declare the arg with type \`(s3object)\`. Windmill renders an S3 file picker for
|
|
it, downloads the file, and binds it as a \`jsonb\` parameter — Parquet/CSV files
|
|
are decoded server-side into a JSON array of records, JSON/JSONL pass through.
|
|
Consume with \`jsonb_to_recordset\` (or any \`jsonb\` API):
|
|
|
|
\`\`\`sql
|
|
-- $1 file (s3object)
|
|
SELECT *
|
|
FROM jsonb_to_recordset($1::jsonb) AS r(id INT, name TEXT);
|
|
\`\`\`
|
|
|
|
## Streaming query results to S3
|
|
|
|
Add a \`-- s3\` directive at the top of the script to stream the result set to S3
|
|
instead of returning rows. Windmill writes the file and returns its \`S3Object\`
|
|
as the script result.
|
|
|
|
\`\`\`sql
|
|
-- s3 prefix=exports/users format=parquet
|
|
SELECT id, name FROM users;
|
|
\`\`\`
|
|
|
|
All keys are optional: \`prefix\` (object key prefix), \`storage\` (named storage —
|
|
omit to use the workspace default), \`format\` (\`json\` (default), \`parquet\`, or
|
|
\`csv\`). Use this for large result sets — rows stream directly to S3 instead of
|
|
being buffered as the script return value.
|
|
`;
|
|
|
|
export const LANG_POWERSHELL = `# PowerShell
|
|
|
|
## Structure
|
|
|
|
Arguments are obtained by calling the \`param\` function on the first line:
|
|
|
|
\`\`\`powershell
|
|
param($Name, $Count = 0, [int]$Age)
|
|
|
|
# Your code here
|
|
Write-Output "Processing $Name, count: $Count, age: $Age"
|
|
|
|
# Return object
|
|
@{
|
|
name = $Name
|
|
count = $Count
|
|
age = $Age
|
|
}
|
|
\`\`\`
|
|
|
|
## Parameter Types
|
|
|
|
You can specify types for parameters:
|
|
|
|
\`\`\`powershell
|
|
param(
|
|
[string]$Name,
|
|
[int]$Count = 0,
|
|
[bool]$Enabled = $true,
|
|
[array]$Items
|
|
)
|
|
|
|
@{
|
|
name = $Name
|
|
count = $Count
|
|
enabled = $Enabled
|
|
items = $Items
|
|
}
|
|
\`\`\`
|
|
|
|
## Return Values
|
|
|
|
Return values by outputting them at the end of the script:
|
|
|
|
\`\`\`powershell
|
|
param($Input)
|
|
|
|
$result = @{
|
|
processed = $true
|
|
data = $Input
|
|
timestamp = Get-Date -Format "o"
|
|
}
|
|
|
|
$result
|
|
\`\`\`
|
|
`;
|
|
|
|
export const LANG_PYTHON3 = `# Python
|
|
|
|
## Structure
|
|
|
|
The script must contain at least one function called \`main\`:
|
|
|
|
\`\`\`python
|
|
def main(param1: str, param2: int):
|
|
# Your code here
|
|
return {"result": param1, "count": param2}
|
|
\`\`\`
|
|
|
|
Do not call the main function. Libraries are installed automatically.
|
|
|
|
## Resource Types
|
|
|
|
On Windmill, credentials and configuration are stored in resources and passed as parameters to main.
|
|
|
|
You need to **redefine** the type of the resources that are needed before the main function as TypedDict:
|
|
|
|
\`\`\`python
|
|
from typing import TypedDict
|
|
|
|
class postgresql(TypedDict):
|
|
host: str
|
|
port: int
|
|
user: str
|
|
password: str
|
|
dbname: str
|
|
|
|
def main(db: postgresql):
|
|
# db contains the database connection details
|
|
pass
|
|
\`\`\`
|
|
|
|
**Important rules:**
|
|
|
|
- The resource type name must be **IN LOWERCASE**
|
|
- Only include resource types if they are actually needed
|
|
- If an import conflicts with a resource type name, **rename the imported object, not the type name**
|
|
- Make sure to import TypedDict from typing **if you're using it**
|
|
|
|
## Imports
|
|
|
|
Libraries are installed automatically. Do not show installation instructions.
|
|
|
|
\`\`\`python
|
|
import requests
|
|
import pandas as pd
|
|
from datetime import datetime
|
|
\`\`\`
|
|
|
|
If an import name conflicts with a resource type:
|
|
|
|
\`\`\`python
|
|
# Wrong - don't rename the type
|
|
import stripe as stripe_lib
|
|
class stripe_type(TypedDict): ...
|
|
|
|
# Correct - rename the import
|
|
import stripe as stripe_sdk
|
|
class stripe(TypedDict):
|
|
api_key: str
|
|
\`\`\`
|
|
|
|
## Windmill Client
|
|
|
|
Import the windmill client for platform interactions:
|
|
|
|
\`\`\`python
|
|
import wmill
|
|
\`\`\`
|
|
|
|
See the SDK documentation for available methods.
|
|
|
|
## Preprocessor Scripts
|
|
|
|
For preprocessor scripts, the function should be named \`preprocessor\` and receives an \`event\` parameter:
|
|
|
|
\`\`\`python
|
|
from typing import TypedDict, Literal, Any
|
|
|
|
class Event(TypedDict):
|
|
kind: Literal["webhook", "http", "websocket", "kafka", "email", "nats", "postgres", "sqs", "mqtt", "gcp"]
|
|
body: Any
|
|
headers: dict[str, str]
|
|
query: dict[str, str]
|
|
|
|
def preprocessor(event: Event):
|
|
# Transform the event into flow input parameters
|
|
return {
|
|
"param1": event["body"]["field1"],
|
|
"param2": event["query"]["id"]
|
|
}
|
|
\`\`\`
|
|
|
|
## S3 Object Operations
|
|
|
|
Windmill provides built-in support for S3-compatible storage operations.
|
|
|
|
### Receiving an S3Object as a script parameter
|
|
|
|
To accept a file from S3 as input to a script, type the parameter with \`S3Object\` (imported from \`wmill\`):
|
|
|
|
\`\`\`python
|
|
import wmill
|
|
from wmill import S3Object
|
|
|
|
def main(file: S3Object):
|
|
content = wmill.load_s3_file(file)
|
|
# ...
|
|
\`\`\`
|
|
|
|
### S3 operations
|
|
|
|
\`\`\`python
|
|
import wmill
|
|
|
|
# Load file content from S3
|
|
content: bytes = wmill.load_s3_file(s3object)
|
|
|
|
# Load file as stream reader
|
|
reader: BufferedReader = wmill.load_s3_file_reader(s3object)
|
|
|
|
# Write file to S3
|
|
result: S3Object = wmill.write_s3_file(
|
|
s3object, # Target path (or None to auto-generate)
|
|
file_content, # bytes or BufferedReader
|
|
s3_resource_path, # Optional: specific S3 resource
|
|
content_type, # Optional: MIME type
|
|
content_disposition # Optional: Content-Disposition header
|
|
)
|
|
\`\`\`
|
|
`;
|
|
|
|
export const LANG_RLANG = `# R
|
|
|
|
## Structure
|
|
|
|
Define a \`main\` function using \`<-\` or \`=\` assignment. Parameters become the script inputs:
|
|
|
|
\`\`\`r
|
|
library(dplyr)
|
|
library(jsonlite)
|
|
|
|
main <- function(x, name = "default", flag = TRUE) {
|
|
df <- tibble(x = x, name = name)
|
|
result <- df %>% mutate(greeting = paste("Hello", name))
|
|
return(toJSON(result, auto_unbox = TRUE))
|
|
}
|
|
\`\`\`
|
|
|
|
**Important:**
|
|
- The \`main\` function is required
|
|
- Use \`library()\` to load packages — they are resolved and installed automatically
|
|
- \`jsonlite\` is always available (used internally for argument parsing)
|
|
- Return values must be JSON-serializable
|
|
|
|
## Parameters
|
|
|
|
R types map to Windmill types:
|
|
- \`numeric\` → float/int
|
|
- \`character\` → string
|
|
- \`logical\` → bool (use \`TRUE\`/\`FALSE\`)
|
|
- \`list\` → object/dict
|
|
- \`NULL\` → null
|
|
|
|
Default values are inferred from the function signature:
|
|
|
|
\`\`\`r
|
|
main <- function(
|
|
name, # required string
|
|
count = 10, # optional int, default 10
|
|
verbose = FALSE # optional bool, default FALSE
|
|
) {
|
|
# ...
|
|
}
|
|
\`\`\`
|
|
|
|
## Resources and Variables
|
|
|
|
Use the built-in Windmill helpers (no import needed):
|
|
|
|
\`\`\`r
|
|
main <- function() {
|
|
# Get a variable
|
|
api_key <- get_variable("f/my_folder/api_key")
|
|
|
|
# Get a resource (returns a list)
|
|
db <- get_resource("f/my_folder/postgres_config")
|
|
host <- db$host
|
|
port <- db$port
|
|
|
|
return(list(host = host, port = port))
|
|
}
|
|
\`\`\`
|
|
|
|
## Output
|
|
|
|
Return any JSON-serializable value from \`main\`. The return value becomes the step result:
|
|
|
|
\`\`\`r
|
|
main <- function(x) {
|
|
# Return a scalar
|
|
return(x + 1)
|
|
|
|
# Or a list (becomes JSON object)
|
|
return(list(result = x + 1, status = "ok"))
|
|
}
|
|
\`\`\`
|
|
|
|
## Annotations
|
|
|
|
Control execution behavior with comment annotations:
|
|
|
|
\`\`\`r
|
|
#renv_verbose = true # Show verbose renv output during resolution
|
|
#renv_install_verbose = true # Show verbose output during package installation
|
|
#sandbox = true # Run in nsjail sandbox (requires nsjail)
|
|
\`\`\`
|
|
`;
|
|
|
|
export const LANG_RUST = `# Rust
|
|
|
|
## Structure
|
|
|
|
The script must contain a function called \`main\` with proper return type:
|
|
|
|
\`\`\`rust
|
|
use anyhow::anyhow;
|
|
use serde::Serialize;
|
|
|
|
#[derive(Serialize, Debug)]
|
|
struct ReturnType {
|
|
result: String,
|
|
count: i32,
|
|
}
|
|
|
|
fn main(param1: String, param2: i32) -> anyhow::Result<ReturnType> {
|
|
Ok(ReturnType {
|
|
result: param1,
|
|
count: param2,
|
|
})
|
|
}
|
|
\`\`\`
|
|
|
|
**Important:**
|
|
- Arguments should be owned types
|
|
- Return type must be serializable (\`#[derive(Serialize)]\`)
|
|
- Return type is \`anyhow::Result<T>\`
|
|
|
|
## Dependencies
|
|
|
|
Packages must be specified with a partial cargo.toml at the beginning of the script:
|
|
|
|
\`\`\`rust
|
|
//! \`\`\`cargo
|
|
//! [dependencies]
|
|
//! anyhow = "1.0.86"
|
|
//! reqwest = { version = "0.11", features = ["json"] }
|
|
//! tokio = { version = "1", features = ["full"] }
|
|
//! \`\`\`
|
|
|
|
use anyhow::anyhow;
|
|
// ... rest of the code
|
|
\`\`\`
|
|
|
|
**Note:** Serde is already included, no need to add it again.
|
|
|
|
## Async Functions
|
|
|
|
If you need to handle async functions (e.g., using tokio), keep the main function sync and create the runtime inside:
|
|
|
|
\`\`\`rust
|
|
//! \`\`\`cargo
|
|
//! [dependencies]
|
|
//! anyhow = "1.0.86"
|
|
//! tokio = { version = "1", features = ["full"] }
|
|
//! reqwest = { version = "0.11", features = ["json"] }
|
|
//! \`\`\`
|
|
|
|
use anyhow::anyhow;
|
|
use serde::Serialize;
|
|
|
|
#[derive(Serialize, Debug)]
|
|
struct Response {
|
|
data: String,
|
|
}
|
|
|
|
fn main(url: String) -> anyhow::Result<Response> {
|
|
let rt = tokio::runtime::Runtime::new()?;
|
|
rt.block_on(async {
|
|
let resp = reqwest::get(&url).await?.text().await?;
|
|
Ok(Response { data: resp })
|
|
})
|
|
}
|
|
\`\`\`
|
|
`;
|
|
|
|
export const LANG_SNOWFLAKE = `# Snowflake
|
|
|
|
Arguments use \`?\` placeholders.
|
|
|
|
Name the parameters by adding comments before the statement:
|
|
|
|
\`\`\`sql
|
|
-- ? name1 (text)
|
|
-- ? name2 (number) = 0
|
|
SELECT * FROM users WHERE name = ? AND age > ?;
|
|
\`\`\`
|
|
|
|
## Receiving an S3Object as a script parameter
|
|
|
|
Declare the arg with type \`(s3object)\`. Windmill renders an S3 file picker for
|
|
it, downloads the file, and binds it as JSON text — Parquet/CSV files are
|
|
decoded server-side into a JSON array of records, JSON/JSONL pass through.
|
|
Wrap the bind with \`PARSE_JSON(?)\` and walk it with \`LATERAL FLATTEN\`:
|
|
|
|
\`\`\`sql
|
|
-- ? file (s3object)
|
|
SELECT
|
|
v.value:id::NUMBER AS id,
|
|
v.value:name::STRING AS name
|
|
FROM LATERAL FLATTEN(input => PARSE_JSON(?)) v;
|
|
\`\`\`
|
|
|
|
## Streaming query results to S3
|
|
|
|
Add a \`-- s3\` directive at the top of the script to stream the result set to S3
|
|
instead of returning rows. Windmill writes the file and returns its \`S3Object\`
|
|
as the script result.
|
|
|
|
\`\`\`sql
|
|
-- s3 prefix=exports/users format=parquet
|
|
SELECT id, name FROM users;
|
|
\`\`\`
|
|
|
|
All keys are optional: \`prefix\` (object key prefix), \`storage\` (named storage —
|
|
omit to use the workspace default), \`format\` (\`json\` (default), \`parquet\`, or
|
|
\`csv\`). Use this for large result sets — rows stream directly to S3 instead of
|
|
being buffered, bypassing the 10000-row return cap.
|
|
`;
|
|
|