Files
windmill/system_prompts/auto-generated/script.md
T
0e807fb1dd feat: put a data table's connection under Postgres roles (#11020)
* feat(datatables): put a data table's connection under Postgres roles

A data table backed by the instance database resolved to exactly one Postgres connection,
`custom_instance_user`, for everyone who could reach it at all. There was no way to say
this job reads, that one writes, this one never sees the salaries table.

A data table role is now a real Postgres login on the cluster, defined once for the
instance by a superadmin and named exactly as they named it. A script that declares
`-- role analytics` connects as `analytics`, and Postgres decides what it may touch —
grants are ordinary SQL. Windmill answers only "may this caller ask for this role", from
the tenant lists on the data table entry: `u/alice`, `g/analysts`, `f/finance` or `*`.
A data table with no `permissions` block behaves exactly as before.

Everything that opens a connection on someone's behalf goes through one chokepoint,
`get_datatable_resource_from_db`, which takes the identity explicitly and fails closed when
there is none. The role logs in as itself — never `SET ROLE`, which a script could
`RESET ROLE` its way out of.

A fork's data table entry becomes a pointer at the workspace that governs it rather than a
copy of it. The settings clone used to hand a fork a byte-identical entry naming the
parent's database, which a fork admin could edit to grant themselves `admin` there; a
pointer has nothing local to edit, and its tenants are evaluated as a member of the
governing workspace, by email. `permissions` is stripped from the workspace export and
ignored on import: tenants name principals of one workspace, and a settings push is not
where an access decision should be made.

Operations that see the whole database whatever the roles grant stay with the governing
workspace's admins: editing the roles, a migration that declares none, and opening a
replication stream for a Postgres trigger or capture.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012ti5HyeTikPMYyW8YSdiHR

* fix(datatables): gate the paths that reach a whole database as admin

Auditing what still resolved through the unchecked resolver turned up three that act for a
caller and hand back the admin connection: `resolve_pg_source_checked` (behind schema
export, the full-schema read, database creation, import and the forked-database drop), the
connection test, and the schema snapshot a fork clone takes of its parent. On a data table
under roles each let any workspace member — or a fork admin who is nobody in the governing
workspace — read or copy the whole database whatever its roles grant.

All three now require admin reach on the governing workspace. A dump taken under a
restricted role would be a silently truncated copy rather than an error, so refusing is the
only right answer for the copy paths.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012ti5HyeTikPMYyW8YSdiHR

* fix(datatables): confine roles to the instance database, and stop a fork reaching the parent's bookkeeping

A data table role is a login on Windmill's own Postgres. Nothing stopped a workspace admin
putting a *resource-backed* data table under roles, at which point the executor dialled the
host that resource names — one the admin chose — with the role's real cluster password, and
`CONNECT` is granted to every registered instance database. Both ends now refuse: the
permissions endpoint rejects the save, and the chokepoint refuses to substitute credentials
on a non-instance entry rather than trusting the record it read.

Two more places reached the governing database without answering to it. The initial-migration
generator returned a `pg_dump` of the whole schema to any member. And the migration
rename/delete cascade followed a fork's pointer into the parent, so a fork admin renaming or
removing their own local entry relabelled or wiped the parent's `_wm_migrations` — after
which the parent re-runs every migration from zero. The remote half is now skipped when the
entry resolves into another workspace, which is also just correct: a fork renaming what it
calls a data table changes nothing about the data table.

Also: revoking a tenant now bounces the replication streams of every workspace holding an
entry that resolves here, not only the governing one, so a fork's trigger stops rather than
living on inside its open connection; the instance role catalog and the governing workspace's
tenant lists are no longer returned to someone who cannot edit them; and the tenant rename
dedup collapses non-adjacent duplicates, per role rather than once any role changed.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012ti5HyeTikPMYyW8YSdiHR

* fix(datatables): fail loudly where a role or a pointer can be left half-recorded

Three ways the feature could end up in a state nobody could see or undo.

Creating a role writes the cluster first and the catalog second, but the catalog write was an
`UPDATE` that matched nothing when the instance Postgres settings row was absent — leaving a
live login with a password nobody recorded: invisible to the catalog, un-recreatable because
the name is taken, and un-deletable because there is no entry to delete. It now errors, so
the operation is retryable once the row is restored.

Deleting a workspace only nulls the fork lineage; the data table entries pointing at it are
left resolving to nothing. Sweeping them is not an option — turning a pointer back into a copy
would hand each fork the database outright — so the delete now names the data tables it
stranded, and resolving one says which workspace is missing rather than reporting a data table
this workspace never had.

`InstanceDatatableRole` derived `Debug` while holding a Postgres password; it is now
hand-written so `{:?}` on the catalog cannot put a live credential in a log line.

Adds the two branches the reviews found unpinned: a caller who is not a member of the
governing workspace at all, and `NoIdentity` — the compatibility path for an agent worker that
predates this and sends no job id.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012ti5HyeTikPMYyW8YSdiHR

* fix(datatables): unbreak two operator messages and two comments that described other code

The two strings this branch added for states an operator hits once — the catalog write that
matched nothing, and the delete that stranded a pointer — were collapsed from their multi-line
form with the indentation left in, so both rendered with a fourteen-space gap mid-sentence.

`list_datatables` claimed to report a chain it cannot follow and then dropped it; it does drop
it, and the comment now says why that is the right place to stay quiet. The non-superadmin
check in `edit_datatable_config` was introduced as also covering references, which it does not
and need not: `reference` is overwritten from the stored entry for every caller before the
check runs.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012ti5HyeTikPMYyW8YSdiHR

* fix(datatables): serialize role catalog mutations, and state each helper's authorization contract

The catalog is one JSON document, so create, rename, enable and delete are all
read-modify-write. Two concurrent creates read the same snapshot, both succeed in the
cluster, and the second write drops the first — leaving a live Postgres login with a password
nobody recorded, which is the exact state the delete path exists to prevent. Every mutation
now runs in one transaction holding an advisory lock across the read, the cluster DDL and the
write, so a lost update cannot happen and a failure rolls the whole thing back. The DDL
helpers take that transaction rather than the pool, which is what makes the lock cover them.

Their statements moved off `sqlx::raw_sql`: the simple protocol is only needed for genuinely
multi-statement SQL, and its future is not `Send`, which an axum handler holding the
transaction requires. Each of these is one statement anyway.

The new cross-crate surface now says what callers must do. `read_role_catalog` returns
plaintext credentials; `create`/`rename`/`set_login`/`drop_instance_role` and
`converge_connect_grants` mutate cluster-wide state; `read_datatable_entry` reads a workspace's
raw config. All of them are superadmin-gated by their current handlers, but nothing said so at
the definition, which is where the next caller looks.

Also: the roles table reloads after a failed login toggle instead of leaving it claiming a flip
that did not land; the rename affordance is the design-system `Button`, not a raw one; and
`resolve_datatable_pg_as_caller` drops a `role` parameter no caller ever filled — browsing
resolves as the data table's default until the database manager grows a picker.

Why role passwords stay a plain `String` while the instance user's password beside them is a
`StringOrSecretRef`, asked three times across reviews: that one is a secret ref because an
operator supplies it and may want it from their own backend, while these are minted here and
never entered by anyone, so there is nothing for a ref to point at. Encrypting generated
secrets at rest is a separate change that would take the replication password with it. Now
said at the field.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012ti5HyeTikPMYyW8YSdiHR

* fix(datatables): give the role catalog its own row, out of reach of the config machinery

Putting it inside `custom_instance_pg_databases` was the wrong call, and it cost two ways.
The catalog serializes a generated Postgres password per role, and that row is the
operator-facing instance config, so the passwords reached `get_instance_config` and its YAML
editor — a live cluster credential in a response body, a UI field and any log of either.
Worse in the other direction: `to_settings_map` strips the catalog, so a full-row upsert of
that key writes the row back without it and the catalog is gone, while the cluster keeps every
login it described.

`custom_instance_replication_pwd` is the precedent and says exactly why — a generated secret,
written only by the server, never operator-authored, hidden so the config machinery cannot
read, rewrite or drop it. The catalog is the same thing, so it now has the same shape:
`datatable_roles`, in `HIDDEN_SETTINGS`, `PROTECTED_SETTINGS` and the agent-worker denylist.
No redaction to keep in step with three code paths, and no way for a neighbouring write to
take it out.

Two races on the same shared documents. `edit_datatable_config` read the stored data tables
outside its transaction and then wrote the whole `datatable` document, so a permissions save
committing in between was silently rolled back; it now reads under `FOR UPDATE`. And
`set_datatable_permissions` validated role ids against the catalog before opening its
transaction, so a deletion in between let it write a deleted role back — including as the
default, which every later job then fails on; it now holds the catalog lock and the settings
row across validation and write.

Completes the authorization contracts the previous commit claimed but did not finish:
`read_datatable_entry` (which it named and missed), `resolve_governing_datatable`, whose whole
job is to answer for a workspace the caller may not belong to, and
`converge_connect_grants_with`, which had not inherited its wrapper's.

Also the generic Python SDK reference: `_format_py_params` learned the bare `*` last time, but
`extract_py_functions` is a second formatter and still rendered `datatable(name, role)`, so
code written from that page passed a keyword-only argument positionally.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012ti5HyeTikPMYyW8YSdiHR

* fix(datatables): make the concurrency test pin the handlers, and the contracts describe what is enforced

The concurrency test reimplemented the read-modify-write inline, so deleting the lock from all
three handlers left it green — it pinned Postgres, not the code it was written for. It now
drives `create_datatable_role` twice concurrently and asserts the catalog kept both names.
Checked the way the last one should have been: removing the lock from the handler makes it
fail with "wmtest_a_… is a live cluster login the catalog forgot".

The contracts added last commit were stricter than this PR's own callers, which is worse than
none — the next reader sees a rule already broken and learns to ignore it.
`read_role_catalog` said superadmin-only while two of its four callers are open to any
workspace member, and `converge_connect_grants` said superadmin while
`set_datatable_permissions` reaches it as a workspace admin. Both were fine on substance: the
rule that actually holds is about the credential never reaching a response, log, audit record
or export, not about who may call. They now say that. `read_datatable_entry` gets the same
treatment rather than the one the earlier message claimed for it: it is the primitive every
resolution goes through, so it is deliberately open, and what must not escape is `permissions`
— it names the governing workspace's users, groups and folders.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012ti5HyeTikPMYyW8YSdiHR

* fix(datatables): close the last ways a role or a pointer can be left pointing at nothing

The raw settings readers hand back whatever is in the row, so moving the catalog into its own
`global_settings` key protected the config machinery and left `GET /settings/global/datatable_roles`
and the settings listing returning every live password. Both now filter that one key. The
neighbouring `custom_instance_replication_pwd` has the same shape and is not touched here: it
predates this and widening the fix to it is a decision about an operator workflow, not a
consequence of this change.

Three ways a save could leave something resolving to nothing:

A permissioned data table could be moved to a PostgreSQL resource. The block was carried across
as a server-owned field, the runtime refuses roles on a resource-backed table, so the save
succeeded and every job afterwards failed. Refused instead — turning roles off first is one step,
and it keeps discarding an access decision something somebody chose.

Renaming a governing data table left every fork pointing at the old name: the data table
disappears from their pickers and their jobs stop, with nothing in the renaming workspace to
suggest why. The rename now follows into the pointers in the same transaction.

Deleting one cannot be followed the same way, so it is reported instead — the response names what
it stranded, the way deleting a workspace does, and the fork's own error already says which
workspace is gone.

Also: `ensure_instance_db_grant_options_unchecked` claimed superadmin while the permissions
handler reaches it as a workspace admin (the same class fixed last commit, one instance missed);
the role entry kept an `instance_config_schema` derive it no longer needs; `write_role_catalog`
was the one writer of that table not stamping `updated_at`; and the concurrency test dropped its
roles only on success — a failing run is exactly the one that creates them without recording them.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012ti5HyeTikPMYyW8YSdiHR

* refactor(datatables): put the role catalog in its own table, not in global_settings

Five findings across three rounds were all the same choice. A set of live Postgres credentials
was living in `global_settings`, which has generic read, list, write, config-export and CLI
round-trip paths that know nothing about what they carry: the passwords reached the instance
config and its YAML editor, a full-row upsert of a neighbouring key erased the catalog,
`GET /settings/global/{key}` and the settings listing returned them raw, and this round the
redaction that fixed the last two turned `wmill instance push` into something that wipes every
password — a fix breaking the assumption the previous fix made. `POST /settings/global/datatable_roles`
could also empty it outside the lock.

The approved plan offered a table or `global_settings`, so this is the other option it already
allowed rather than a new design. `datatable_role` is a table: no generic settings path can read
it, list it, export it, write it or round-trip it, so none of the five needs a guard. The
redaction, the hidden/protected/agent-denylist entries and the JSON document all go with it.

One row per role also removes the read-modify-write the concurrency work was about: two
concurrent creates are two inserts, and the unique index on `name` is what settles a collision.
The advisory lock stays for the one window rows do not cover — `CREATE ROLE` is invisible to
another transaction until commit, so without it both creates pass their `pg_roles` check.

Also from this round: rename mappings are checked against the configuration they claim to
describe, since fork pointers are rewritten from them — a caller could otherwise submit
`main -> missing` against an unchanged config and repoint every fork of `main` at a name nothing
has, and `A -> B` plus `B -> C` moved what pointed at `A` all the way to `C`. And the warning
naming forks a delete stranded reached the response but not the screen: both the data table
settings save and the workspace delete now show it.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012ti5HyeTikPMYyW8YSdiHR

* fix(datatables): validate a rename against the save it describes, and re-check under the locks

Three from the round, all about deciding on state that could already have moved.

A permission save resolved the data table and checked it was instance-backed before taking any
lock, then wrote under one. A config save committing in between could move the table onto a
PostgreSQL resource — recreating exactly what the transition guard refuses — or rename it, in
which case the write targeted a key that no longer existed and reported success having changed
nothing. It now re-resolves and re-checks on the locked state.

Rename validation checked that the source existed before and the target existed after, which
still accepts `main -> decoy` against a save that keeps both: every fork of `main` then follows
onto a different data table, silently, because it keeps resolving. The rule is now the actual
old-to-new key transition — a source may only survive if another rename took its name, and a
target may only pre-exist if another rename freed it. That also stops two sources sharing one
target, and it admits a swap, which the previous guard refused: `datatables` is keyed by name, so
a swap cannot be done one save at a time, and refusing it was a regression against main. The
pointer cascade now runs in two passes through a temporary name, the way the migration cascade
one layer down already handles the same shape, so `A -> B` with `B -> C` moves each pointer once
from what it named before the save.

The tenant mutators say what they are for: they write an access decision for any workspace named,
with an arbitrary mutation, and exist for the transaction that frees or renames a principal.
Editing a decision on purpose belongs in the permissions endpoint.

Carried in the same change: the stranded-fork list is a field rather than a phrase to grep out of
a success string; the pointer cascade matches with `EXISTS` instead of a `LIKE` over the whole
document, so a workspace whose pointers name something else is not rewritten to a byte-identical
value under an exclusive lock; and `InstanceDatatableRole` drops the serde derives left over from
the JSON document, one of which would emit `pwd`.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012ti5HyeTikPMYyW8YSdiHR

* fix(datatables): cascade on the leave route that is used, gate migrations before the admin connection, and drop a role atomically

The tenant cascade on leaving went onto `/users/leave`. The UI and the generated client call
`/workspaces/leave` — a different handler in a different crate with the same name — which
deleted the membership and left `u/<username>` in the tenant lists. Leaving and rejoining
therefore restored the access the leave was supposed to end, and a later account taking the
username would have inherited it. The regression test drives the route the client actually
calls; without the fix it fails with "leaving kept the tenant".

The migration endpoints authorized too late. `run_datatable_migrations` opened the data table's
admin connection, created `_wm_migrations` and read it before reaching the per-migration role
check — so with nothing pending, nothing was checked at all. Rollback returned before its check
when nothing was applied, and the status endpoint had none. All three now ask, before any
connection is opened, whether the caller can reach the data table as any role at all; which role
a given migration runs as is still decided per migration, and by the executor after that.

Deleting a role committed the cluster drop and the catalog row, then swept the tenant lists in
separate transactions. A sweep failing part-way left workspaces naming a role nothing can connect
as, while the retry answered `NotFound` because the catalog entry was already gone. The sweep now
runs in the same transaction, so the drop, the row and every tenant list commit together.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012ti5HyeTikPMYyW8YSdiHR

* fix(datatables): refuse to copy a data table that is under roles

pg_dump carries no roles and the import runs with --no-privileges, so a copied
data table arrives owned by the admin connection with no GRANT for any role.
The settings clone brings `permissions` across, so the fork's tenants pass
Windmill's check, connect as the role they were given, and are denied by
Postgres on everything: an entry that reads as configured and answers nothing.

Refuse the copy — in the import endpoint before any data moves, and in the fork
path the CLI takes. Replaying the source's owners and ACLs into the clone is
what lifts this, and is a change of its own. Dropping `permissions` from the
copy instead would be the unsafe half, since the copy holds the parent's rows.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012ti5HyeTikPMYyW8YSdiHR

* fix(datatables): refuse the clone's database too, not only its data

A clone is two endpoints: `create_pg_database` then `import_pg_database`. Only
the second refused a data table under roles, so a fork asking to clone one
created and registered an empty `wm_fork_…` instance database and then failed —
and nothing collects it, since `drop_forked_datatable_databases` only drops
entries carrying `forked_from` and no entry names this one.

Refuse in both, so the clone stops before a database exists.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012ti5HyeTikPMYyW8YSdiHR

* nit worker error msg

* fix pg_dump stuck on version 17 on nix

* fix(datatables): refuse a malformed role annotation instead of ignoring it

`-- Role operator`, `-- role operator;` and `-- role operator -- why` all failed
the annotation parser's exact-match rule, so the query fell through to the data
table's default role and ran, silently, under a login the author did not choose.
Naming a role exists precisely to not do that.

A leading comment whose first word is `role` is now an annotation attempt: the
keyword matches case-insensitively, one trailing `;` is tolerated, and anything
else is an error naming the line. Only callers that already know the target is a
`datatable://` reference ever run this, so ordinary SQL keeps its comments.

Also bumps the dev shell's postgres client to 18 — it trailed the server the dev
database runs, which takes out every data table export, clone and fork-with-data.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012ti5HyeTikPMYyW8YSdiHR

* fix(datatables): refuse a malformed role query string instead of ignoring it

`?Role=analytics`, `?role=` and `?x=1&role=…` all fell through the reference
parser's exact-match rule, so the connection resolved to the data table's default
role and ran under a login the caller never asked for — the URI half of the same
trap as a malformed `-- role` annotation.

The key now matches case-insensitively, and anything else in the query string is
an error naming it; `role` is the only parameter a reference takes. Callers that
only need the entry keep a lenient `datatable_ref_name`, since they never act on
the role. The DuckDB `ATTACH` parser propagates it rather than attaching under
the default.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012ti5HyeTikPMYyW8YSdiHR

* fix(datatables): carry the role annotation into the row_to_json retry

The retry rebuilds its SQL from `pruneComments(code)`, so the leading comment
block never reached the second attempt — and with it the `-- role <name>` line
that decides which login the query runs as. The retry connected as the data
table's default role instead, so a query the first attempt was denied could
succeed on the second, reported as "recovered with the row_to_json fix".

Carry the leading comment block over. The retry itself is unchanged.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BjfMkJyKzodxkobqGZ6Lqb

* chore(datatables): don't mount the roles UI until the ACL editor lands

Enforcement ships first. The permissions drawer is what turns roles on, and the
catalog section is what creates them — both are only useful once there is a way
to grant a role the privileges it needs, which arrives with the ACL editor. Left
mounted they would offer a feature whose other half does not exist.

The two components are complete and reviewed; only their call sites here are
commented out, with a note pointing the follow-up PRs at them.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BjfMkJyKzodxkobqGZ6Lqb

* fix(datatables): honour `-- role: x`, and fix the DuckDB attach test

Two review findings, both real.

`attach_datatable_parses_name_and_role` never compiled: `parse_attach_datatable`
returns `Result<Option<_>>` now and one call site kept a single `unwrap`. Its
`?Role=analytics` case also asserted a refusal, contradicting the parser in the
same commit, which matches the key case-insensitively. Replaced with the cases
that are genuinely malformed, and a positive one for the cased key.

`-- role: analytics` fell through to the default role — the silent fallback the
strict parser exists to remove, for the spelling most likely to be typed. The
keyword now accepts an optional colon, attached or spaced, while a word that
merely starts with it (`rolebased`) is still not an attempt.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BjfMkJyKzodxkobqGZ6Lqb

* fix(datatables): clone a fork's pointer instead of failing after the copy

Forking a fork with cloning left an orphan database. The preflight resolves the
pointer and sees the governing entry, so both endpoints ran and filled the new
database; `apply_forked_datatable` then refused the inherited pointer and rolled
the fork back, stranding a registered `wm_fork_*` that no entry names and whose
name blocks the retry.

Refusing earlier would have been the smaller change, but forking a fork and
cloning worked before pointers existed, so it would trade an orphan for a
regression. Resolve what the pointer names and write the terminal entry the
clone needs: the whole `database` object rather than a patch of its
`resource_path`, since a pointer has none, and `reference` removed with it.

Also accepts `-- role=x` and `-- Role = x`, two more spellings that fell through
to the default role.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BjfMkJyKzodxkobqGZ6Lqb

* fix(datatables): refuse to roll back the catalog while roles exist

The down migration dropped the table and left every role behind: live Postgres
logins whose passwords only that table carried, so after a revert Windmill could
neither use, disable nor delete them, and re-applying could not recreate them
because the names were taken. Cleaning up here is not possible either — dropping
a role means reassigning what it owns in every instance database, and a
migration runs in one — so it now refuses while the catalog is non-empty and
says to delete the roles through instance settings, which does the cluster work.

Also enforces the instance-only invariant the resolved-pointer clone relies on
rather than only asserting it in a comment.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BjfMkJyKzodxkobqGZ6Lqb

* refactor(datatables): settle clonability in one place, before anything is created

A clone is three stages a workspace apart — `create_pg_database`, then
`import_pg_database`, then `apply_forked_datatable` inside the fork transaction.
Only the third can roll back, and `CREATE DATABASE` is not transactional, so any
refusal that lives there strands a registered `wm_fork_*` that no entry names
and whose name blocks the retry.

That orphan has now been fixed three times, most recently reintroduced by a
guard added one commit ago. Patching each new refusal into the first endpoint is
not the fix; having two places that can refuse is. `ensure_datatable_is_clonable`
now answers every reason a copy can be refused and returns what it resolved, and
the stage that writes the entry only does the work.

Also takes an ACCESS EXCLUSIVE lock before the rollback guard counts, so a role
created concurrently cannot slip between the check and the drop.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BjfMkJyKzodxkobqGZ6Lqb

* fix(datatables): let a retried clone reclaim its own leftover database

A clone creates its target database one request before it copies into it, and
the fork that would name it is written a request after that. Any failure in
between — a pg_dump error, a bad restore, a dropped connection, the source's
roles changing mid-flow — left a registered `wm_fork_*` that no entry names,
and every retry then failed on its name. This predates data table roles.

`create_pg_database` now reclaims such a leftover before creating: only a
`wm_fork_*` database Windmill registered as a data table database and that no
data table or ducklake entry names, in any workspace, archived ones included.
The drop never terminates connections, so a clone still copying into it makes
the reclaim fail instead of being cut off. It is limited to callers who
administer the source — reaching it is not enough, since on a data table
without roles every member reaches it — and anyone else gets the refusal an
existing database always got.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* Revert "fix(datatables): let a retried clone reclaim its own leftover database"

This reverts commit 7dd3275a10.

The reclaim tied the caller to the source they administer, but not to the
database it dropped. Between another workspace's import and its final fork
request, that workspace's target is full, registered, unnamed and has no open
connection, so an admin of any instance data table could name it and have it
dropped and recreated empty. The victim's fork would then commit pointing at
the empty copy. Safe reclaim needs durable clone ownership and serialization
with the request that names the database; until then the leftover stays, as it
did before this PR.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* docs(datatables): record the stale clone database as a known limitation

A clone is three requests and `CREATE DATABASE` is not transactional, so a
failure after the first leaves a registered `wm_fork_*` behind, as it did
before data table roles. Accepted for this PR: it is harmless to data and goes
away once the clone is a single server-side operation.

The comment also records why the obvious fix is wrong: reclaiming the leftover
on retry, without durable clone ownership, can drop another workspace's fully
copied database between its import and its final fork request.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* fix(datatables): bounce the streams reading a data table when it is deleted

Deleting a governing data table, or the workspace that holds it, only collected
the fork pointers it stranded, for the warning. A Postgres trigger or capture
already streaming through one of those pointers kept the replication connection
it opened while the pointer still resolved, so it went on dispatching the
governing database's rows after the fork lost access — until its connection
happened to restart. The governing workspace's own streams on a deleted entry
did the same.

Both deletion paths now bounce the affected listeners inside their own
transaction, through the helper a permission change already uses, so a
listener that reconnects re-resolves the entry and finds it gone. The helper is
split so a caller can pass the (workspace, local name) pairs it already holds.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* fix(datatables): keep the fork schema baseline, and bounce streams on every removal

Three fixes from review.

`edit_datatable_config` took `forked_from` wholesale from the stored entry, so
the fork schema diff's save of an advanced baseline was silently discarded and
an applied change was offered again. Whether an entry carries a clone stamp is
still carried from the store, since that is what marks its database droppable,
but the baseline inside it is now taken from the request.

The stranded-pointer warning and the stream bounce ran over the optional
`deleted_datatables` hint, which the settings-sync CLI never sends, so removing
a governing data table through `wmill` bounced nothing. Removals are now derived
from the stored configuration against the saved one.

`delete_workspace` read the pointers to bounce before its transaction, so a fork
committing a pointer during the deletion was missed. The read now happens inside
the transaction, after the workspace row is deleted: a fork's insert key-share
locks that row through its parent foreign key, so it is either seen or fails on
the missing parent.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* refactor(datatables): keep Postgres triggers and data table roles apart

A replication stream reads every row of every table whatever the data table's
roles grant, and its listener checks access only when it connects. Rather than
chase every way access can change and bounce the streams each one affects, a
data table now carries one or the other:

- a Postgres trigger or capture cannot be created on, or connect to, a data
  table under roles;
- roles cannot be turned on while an enabled trigger or a live capture reads
  the data table, its own or a fork's through its pointer. The refusal names
  each one to disable.

This removes the stream bounces on roles edits and on data table and workspace
deletion, and the trigger gate that admitted admins. The fork schema baseline
fix from the same review round is kept.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* fix(datatables): refuse a Postgres trigger on a data table under roles when it is saved

Creating or editing a trigger that points at a data table under roles was
accepted, and its listener then retried the refused connection every 30
seconds forever. The save is now refused, and a trigger that reaches such a
data table anyway (re-enabled, or cloned into a fork) is disabled by its
listener with the reason, as a missing replication slot is.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* fix(datatables): disable a data table role before deleting it

Deleting a role reassigns and drops what it owns in each registered
database on its own connection, and each of those passes commits as it
goes. A database failing part-way left the role enabled in the catalog and
able to log in, but already stripped in the databases reached before it.
The role is now disabled in its own commit first, so a failed delete
leaves a disabled role to retry.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* fix(datatables): serialize roles going on with a stream starting

Turning roles on looked for enabled triggers and live captures once,
without a lock anything starting a stream also took. A trigger enabled in
that window could have its listener connect before roles committed, and a
healthy listener never checks again. Both transitions now serialize on one
advisory lock: roles going on hold it exclusive while they look, and
trigger create, edit and enable, and capture setup and ping hold it shared
while they commit. Either the look sees the stream, or the listener
connects after roles are committed and refuses.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* fix(datatables): wait out live listeners, and resolve stored names containing `?`

Turning roles on counted a trigger as gone once disabled, and a capture
once its client stopped pinging, but the listener keeps its replication
connection until its next heartbeat notices. A trigger or capture whose
listener pinged in the last 15 seconds, the window a server holds a
listener for, now still counts as streaming.

Data table names could contain `?` before they were restricted, and such
entries are still stored. Splitting `?role=` off a reference misread them:
`a?b` became `a` with an unknown parameter, and the clone checks looked at
a different entry than the one copied. An entry stored under the whole
reference is now looked up first, in the Postgres executor, DuckDB ATTACH
and the clone checks. Agent workers cannot read the workspace and keep
the strict parse, which refuses such a name rather than misreading it.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* fix(datatables): warn when a settings sync strands fork pointers

A settings save reported the fork pointers left resolving to nothing only
for the names in `deleted_datatables`, which `wmill sync push` never sends.
The save now works out what it removed from the locked entries, and the
CLI prints the stranded pointers it returns.

Also correct the replication helper's contract: no role or admin check
makes a replication connection safe, so a data table under roles is
refused outright rather than gated as an admin operation.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BjfMkJyKzodxkobqGZ6Lqb

* fix(datatables): refuse a save that drops a data table's roles through an undeclared rename

A data table's roles follow its entry only through a declared rename. A
settings sync sends the whole map and never declares one, so renaming a
data table under roles there read as a delete and a new entry on the same
database: the new entry carried no roles, and every caller connected as
admin. Such a save is now refused, naming both entries.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BjfMkJyKzodxkobqGZ6Lqb

* fix(datatables): no entry without roles may newly reach a database under roles

The previous guard only caught a new name replacing an entry under roles.
A whole-map save could also repoint an existing entry without roles at
that database, or another workspace could point one there, and every
caller of that entry would connect as admin. The rule is now stated on
the saved entries: one that carries no roles and newly points at an
instance database any entry under roles uses, in this workspace or
another, is refused. A declared rename carries its roles and passes.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BjfMkJyKzodxkobqGZ6Lqb

* feat(datatables): move data table role catalog and resolution to the enterprise edition

Roles are an Enterprise Edition feature. The catalog, the Postgres logins,
CONNECT convergence, tenant evaluation and the role half of connection
resolution move to windmill-ee-private. Every public function keeps its path
and signature and forwards through datatable_roles_oss, which re-exports the
enterprise implementation or, without it, refuses.

Without the enterprise edition a data table under roles, or a caller naming a
role, is refused a connection rather than resolved as admin, and the reach and
admin-access checks refuse one under roles. A data table not under roles
resolves as before in every edition, and an instance database keeps the
CONNECT grants it was created with. The catalog lock, the stream lock, the
tenant cascades and the permissions stripping stay in OSS: they only restrict.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BjfMkJyKzodxkobqGZ6Lqb

* feat(datatables): move the data table permissions endpoints to the enterprise edition

The permissions read, save and usable-roles handlers move to
windmill-ee-private; the routes stay registered and, without the enterprise
edition, answer that data table roles are an Enterprise Edition feature.
ensure_governs_datatable and ensure_reaches_datatable keep their paths: the
first refuses, the second passes a data table not under roles.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BjfMkJyKzodxkobqGZ6Lqb

* feat(datatables): move the data table role catalog endpoints to the enterprise edition

The superadmin list, create, update and delete handlers move to
windmill-ee-private. The routes stay registered and, without the enterprise
edition, refuse after authentication.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BjfMkJyKzodxkobqGZ6Lqb

* test(datatables): run the roles tests on the enterprise edition, refusals without it

Each test that exercises roles runs with private and enterprise. Two tests run
without them: every roles route answers the Enterprise refusal, and a data
table saved under roles, or a named role, is refused a connection while one
not under roles resolves as before.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BjfMkJyKzodxkobqGZ6Lqb

* feat(datatables): gate the roles UI mount sites on an enterprise license

Both mount sites are still commented out; the gate travels with them.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BjfMkJyKzodxkobqGZ6Lqb

* test(datatables): run the tenant matcher test on the enterprise edition

The matcher it covers is enterprise code now, so without the enterprise
edition the test hit the stub and failed the default windmill-common run. It
runs with private and enterprise, and a counterpart without them asserts that
no tenant list covers anyone, the wildcard and a workspace admin included.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BjfMkJyKzodxkobqGZ6Lqb

* chore: update ee-repo-ref to a1873dbb67f2302b85ff5362f8387b48eccdb607

This commit updates the EE repository reference after PR #783 was merged in windmill-ee-private.

Previous ee-repo-ref: 5c853e2c20eca6b748415fc0d6862a6ebfb5fec4

New ee-repo-ref: a1873dbb67f2302b85ff5362f8387b48eccdb607

Automated by sync-ee-ref workflow.

* fix(datatables): refuse roles while a same-workspace alias reaches the database

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* fix(datatables): let CE migrations connect as an explicitly named admin

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* fix(datatables): serialize roles going on with aliases saved from other workspaces

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* docs(datatables): note that legacy names with ? cannot be migrated

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* fix(datatables): drop a DuckDB data table secret once its ATTACH has used it

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* perf(datatables): resolve a workspace's data tables per pointer hop, not per entry

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* fix(datatables): hold the parent's settings while a fork points at its data tables

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* chore: update ee-repo-ref to 7e338e4dabf91689bfd7fb0333c6534040b17b59

This commit updates the EE repository reference after PR #787 was merged in windmill-ee-private.

Previous ee-repo-ref: 38d6fcf2aeb39cfdac21814bbdbbcc02911e566a

New ee-repo-ref: 7e338e4dabf91689bfd7fb0333c6534040b17b59

Automated by sync-ee-ref workflow.

---------

Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
Co-authored-by: windmill-internal-app[bot] <windmill-internal-app[bot]@users.noreply.github.com>
2026-09-18 13:56:41 +02:00

92 KiB

Windmill Script Writing Guide

General Principles

  • Scripts must export a main function (do not call it)
  • Libraries are installed automatically - do not show installation instructions
  • Credentials and configuration are stored in resources and passed as parameters
  • The windmill client (wmill) provides APIs for interacting with the platform

Function Naming

  • Main function: main (or preprocessor for preprocessor scripts)
  • Must be async for TypeScript variants

Return Values

  • Scripts can return any JSON-serializable value
  • Return values become available to subsequent flow steps via results.step_id

Preprocessor Scripts

Preprocessor scripts process raw trigger data from various sources (webhook, custom HTTP route, SQS, WebSocket, Kafka, NATS, MQTT, AMQP, Postgres, or email) before passing it to the flow. This separates the trigger logic from the flow logic and keeps the auto-generated UI clean.

The returned object determines the parameter values passed to the flow. e.g., { b: 1, a: 2 } calls the flow with a = 2 and b = 1, assuming the flow has two inputs called a and b.

The preprocessor receives a single parameter called event.

Ansible

Windmill runs Ansible playbooks with ansible-playbook. A script is a single YAML document made of two parts separated by a --- line: a Windmill header and one or more standard Ansible plays.

Structure

---
# Windmill header: configures inventories, file resources, arguments and dependencies
extra_vars:
  world_qualifier:
    type: string
dependencies:
  galaxy:
    collections:
      - name: community.general
  python:
    - jmespath
---
# Standard Ansible plays
- name: Echo
  hosts: 127.0.0.1
  connection: local
  tasks:
    - name: Print debug message
      debug:
        msg: "Hello, {{ world_qualifier }} world!"

Header

The header is not standard Ansible — it is parsed by Windmill to build the script's inputs and runtime environment. Supported keys:

  • extra_vars: defines the script arguments. Each entry is passed to the playbook via --extra-vars and becomes a Jinja variable usable as {{ name }} in the plays. Give each argument a type (string, number, boolean, object, ...) so Windmill can generate the input form.
  • inventory: lists inventories. Use resource_type: ansible_inventory (optionally pinned with resource: u/user/your_resource) or resource_type: dynamic_inventory.
  • files: writes Windmill resources/variables to files before the run, e.g. - resource: u/user/template with target: ./config.j2, or - variable: u/user/ssh_key with target: ./ssh_key and mode: '0600'.
  • dependencies: galaxy collections/roles (installed with ansible-galaxy) and python pip packages available to the playbook.
  • options: extra ansible-playbook flags such as - verbosity: vvv.
  • vault_password: a Windmill variable path to use as the Ansible Vault password.

Arguments

Reference header extra_vars directly as Jinja variables in the plays:

extra_vars:
  name:
    type: string
  count:
    type: number
---
- hosts: localhost
  tasks:
    - debug:
        msg: "{{ name }} x {{ count }}"

Environment variables

Windmill contextual variables are available as environment variables and read with the env lookup:

- debug:
    msg: "Running in workspace {{ lookup('env', 'WM_WORKSPACE') }}"

Output

To return a result, write JSON to a result.json file in the job directory:

- hosts: localhost
  tasks:
    - name: Write result
      copy:
        content: "{{ { 'ok': true, 'value': 42 } | to_json }}"
        dest: result.json

Bash

Structure

Do not include #!/bin/bash. Arguments are obtained as positional parameters:

# Get arguments
var1="$1"
var2="$2"

echo "Processing $var1 and $var2"

# Return JSON by echoing to stdout
echo "{\"result\": \"$var1\", \"count\": $var2}"

Important:

  • Do not include shebang (#!/bin/bash)
  • Arguments are always strings
  • Access with $1, $2, etc.

Output

The script output is captured as the result. For structured data, output valid JSON:

name="$1"
count="$2"

# Output JSON result
cat << EOF
{
  "name": "$name",
  "count": $count,
  "timestamp": "$(date -Iseconds)"
}
EOF

Environment Variables

Environment variables set in Windmill are available:

# Access environment variable
echo "Workspace: $WM_WORKSPACE"
echo "Job ID: $WM_JOB_ID"

BigQuery

Arguments use @name syntax.

Name the parameters by adding comments before the statement:

-- @name1 (string)
-- @name2 (int64) = 0
SELECT * FROM users WHERE name = @name1 AND age > @name2;

Receiving an S3Object as a script parameter

Declare the arg with type (s3object). Windmill renders an S3 file picker for it, downloads the file, and binds it as a STRING JSON parameter — Parquet/CSV files are decoded server-side into a JSON array of records, JSON/JSONL pass through. Consume with JSON_EXTRACT_ARRAY / JSON_VALUE:

-- @file (s3object)
SELECT
  CAST(JSON_VALUE(row, '$.id') AS INT64) AS id,
  JSON_VALUE(row, '$.name') AS name
FROM UNNEST(JSON_EXTRACT_ARRAY(@file)) AS row;

Streaming query results to S3

Add a -- s3 directive at the top of the script to stream the result set to S3 instead of returning rows. Windmill writes the file and returns its S3Object as the script result.

-- s3 prefix=exports/users format=parquet
SELECT id, name FROM users;

All keys are optional: prefix (object key prefix), storage (named storage — omit to use the workspace default), format (json (default), parquet, or csv). Use this for large result sets — rows stream directly to S3 instead of being buffered, bypassing the 10000-row return cap.

TypeScript (Bun)

Bun runtime with full npm ecosystem and fastest execution. Bun is the default and preferred TypeScript runtime — choose it for any TypeScript script unless there is a major reason to use Deno for that specific use-case.

Structure

Export a single async function called main:

export async function main(param1: string, param2: number) {
  // Your code here
  return { result: param1, count: param2 };
}

Do not call the main function. Libraries are installed automatically.

Resource Types

On Windmill, credentials and configuration are stored in resources and passed as parameters to main.

Use the RT namespace for resource types:

export async function main(stripe: RT.Stripe) {
  // stripe contains API key and config from the resource
}

Only use resource types if you need them to satisfy the instructions. Always use the RT namespace.

Before using a resource type, check the rt.d.ts file in the project root to see all available resource types and their fields. This file is generated by wmill resource-type generate-namespace.

Imports

import Stripe from "stripe";
import { someFunction } from "some-package";

Prefer //native when the runtime allows it

If a script only needs fetch and the JavaScript standard library — including when it uses windmill-client — prefer making it a native script: add //native as the first line and write it with the write-script-bunnative skill. Native scripts run on a lightweight V8 isolate, start faster, and parallelize heavily. windmill-client works on the native worker (its calls go over fetch), so needing the Windmill client is not a reason to avoid //native. Use the regular bun language only when the code (or a dependency) needs Node/Bun runtime APIs — node:* modules, the filesystem, child processes, or native addons.

Windmill Client

Import the windmill client for platform interactions:

import * as wmill from "windmill-client";

Prefer windmill-client over raw fetch for anything that talks to Windmill — reading resources/variables/states, running scripts and flows, S3 object operations, etc. It handles auth, the workspace, and the base URL for you, so you don't hand-roll URLs or tokens. Reserve fetch for calling external HTTP APIs that aren't Windmill.

The full windmill-client API reference (every exported function and its signature) is included in this skill below — consult it for the exact method to use instead of guessing or falling back to fetch.

Preprocessor Scripts

For preprocessor scripts, the function should be named preprocessor and receives an event parameter:

type Event = {
  kind:
    | "webhook"
    | "http"
    | "websocket"
    | "kafka"
    | "email"
    | "nats"
    | "postgres"
    | "sqs"
    | "mqtt"
    | "gcp";
  body: any;
  headers: Record<string, string>;
  query: Record<string, string>;
};

export async function preprocessor(event: Event) {
  return {
    param1: event.body.field1,
    param2: event.query.id,
  };
}

S3 Object Operations

Windmill provides built-in support for S3-compatible storage operations. The wmill.S3Object type covers both the s3://storage/key URI form (s3:///key for the workspace default storage) and the { s3, storage? } record form — always use it instead of redefining your own.

Receiving an S3Object as a script parameter

import * as wmill from "windmill-client";

export async function main(file: wmill.S3Object) {
  const content = await wmill.loadS3File(file);
  // ...
}

S3 operations

import * as wmill from "windmill-client";

// Load file content from S3
const content: Uint8Array = await wmill.loadS3File(s3object);

// Load file as stream
const blob: Blob = await wmill.loadS3FileStream(s3object);

// Write file to S3
const result: wmill.S3Object = await wmill.writeS3File(
  s3object, // Target path (or undefined to auto-generate)
  fileContent, // string or Blob
  s3ResourcePath // Optional: specific S3 resource to use
);

TypeScript (Bun Native)

Native TypeScript execution. Native scripts are Bun scripts that run on the native worker — a lightweight V8 isolate that exposes fetch and the JavaScript standard library — and can be heavily parallelized. Every script MUST start with //native on its first line so Windmill routes it to the native worker; without it the exact same script runs on the regular Bun worker. You may import npm packages and other Windmill scripts (e.g. ./helper.ts) — imports are resolved and bundled just like a regular Bun script — as long as everything (your code and its dependencies) relies only on fetch and the standard library. Libraries that need Node/Bun runtime APIs (filesystem, node:* modules, child processes, native addons) will not work on the native worker; use the regular bun language for those.

Structure

Export a single async function called main:

//native
export async function main(param1: string, param2: number) {
  // Your code here
  return { result: param1, count: param2 };
}

Do not call the main function.

Resource Types

On Windmill, credentials and configuration are stored in resources and passed as parameters to main.

Use the RT namespace for resource types:

//native
export async function main(stripe: RT.Stripe) {
  // stripe contains API key and config from the resource
}

Only use resource types if you need them to satisfy the instructions. Always use the RT namespace.

Before using a resource type, check the rt.d.ts file in the project root to see all available resource types and their fields. This file is generated by wmill resource-type generate-namespace.

Imports

The constraint is the runtime, not the import list. You may import npm packages and relative Windmill scripts; they are resolved and bundled exactly like a regular Bun script. But the native worker only provides fetch and the JavaScript standard library, so any imported code must work using only those. Anything requiring Node/Bun built-ins (node:fs, child_process, the Bun API, native modules) belongs in a regular bun script instead. Use the globally available fetch for HTTP:

//native
export async function main(url: string) {
  const response = await fetch(url);
  return await response.json();
}

Windmill Client

windmill-client works on the native worker (its calls go over fetch), so use it as the preferred way to talk to Windmill — reading resources/variables/states, running scripts and flows, and the S3 helpers below (loadS3File, loadS3FileStream, writeS3File, S3Object). It handles auth, the workspace, and the base URL for you. Reserve raw fetch for calling external HTTP APIs that aren't Windmill.

The full windmill-client API reference (every exported function and its signature) is included in this skill below — consult it for the exact method instead of hand-rolling a fetch against the Windmill API.

Preprocessor Scripts

For preprocessor scripts, the function should be named preprocessor and receives an event parameter:

//native
type Event = {
  kind:
    | "webhook"
    | "http"
    | "websocket"
    | "kafka"
    | "email"
    | "nats"
    | "postgres"
    | "sqs"
    | "mqtt"
    | "gcp";
  body: any;
  headers: Record<string, string>;
  query: Record<string, string>;
};

export async function preprocessor(event: Event) {
  return {
    param1: event.body.field1,
    param2: event.query.id,
  };
}

S3 Object Operations

Windmill provides built-in support for S3-compatible storage operations. The wmill.S3Object type covers both the s3://storage/key URI form (s3:///key for the workspace default storage) and the { s3, storage? } record form — always use it instead of redefining your own.

Receiving an S3Object as a script parameter

//native
import * as wmill from "windmill-client";

export async function main(file: wmill.S3Object) {
  const content = await wmill.loadS3File(file);
  // ...
}

S3 operations

//native
import * as wmill from "windmill-client";

// Load file content from S3
const content: Uint8Array = await wmill.loadS3File(s3object);

// Load file as stream
const blob: Blob = await wmill.loadS3FileStream(s3object);

// Write file to S3
const result: wmill.S3Object = await wmill.writeS3File(
  s3object, // Target path (or undefined to auto-generate)
  fileContent, // string or Blob
  s3ResourcePath // Optional: specific S3 resource to use
);

C#

The script must contain a public static Main method inside a class:

public class Script
{
    public static object Main(string name, int count)
    {
        return new { Name = name, Count = count };
    }
}

Important:

  • Class name is irrelevant
  • Method must be public static
  • Return type can be object or specific type

NuGet Packages

Add packages using the #r directive at the top:

#r "nuget: Newtonsoft.Json, 13.0.3"
#r "nuget: RestSharp, 110.2.0"

using Newtonsoft.Json;
using RestSharp;

public class Script
{
    public static object Main(string url)
    {
        var client = new RestClient(url);
        var request = new RestRequest();
        var response = client.Get(request);
        return JsonConvert.DeserializeObject(response.Content);
    }
}

TypeScript (Deno)

Deno runtime with npm support via npm: prefix and native Deno libraries.

Prefer Bun (write-script-bun) for TypeScript. Only use Deno when the script specifically requires the Deno runtime — Deno's standard library or deno.land URL imports that have no npm equivalent. For all other TypeScript, use Bun instead.

Structure

Export a single async function called main:

export async function main(param1: string, param2: number) {
  // Your code here
  return { result: param1, count: param2 };
}

Do not call the main function. Libraries are installed automatically.

Resource Types

On Windmill, credentials and configuration are stored in resources and passed as parameters to main.

Use the RT namespace for resource types:

export async function main(stripe: RT.Stripe) {
  // stripe contains API key and config from the resource
}

Only use resource types if you need them to satisfy the instructions. Always use the RT namespace.

Before using a resource type, check the rt.d.ts file in the project root to see all available resource types and their fields. This file is generated by wmill resource-type generate-namespace.

Imports

// npm packages use npm: prefix
import Stripe from "npm:stripe";
import { someFunction } from "npm:some-package";

// Deno standard library
import { serve } from "https://deno.land/std/http/server.ts";

Windmill Client

Import the windmill client for platform interactions:

import * as wmill from "windmill-client";

Prefer windmill-client over raw fetch for anything that talks to Windmill — reading resources/variables/states, running scripts and flows, S3 object operations, etc. It handles auth, the workspace, and the base URL for you. Reserve fetch for calling external HTTP APIs that aren't Windmill.

The full windmill-client API reference (every exported function and its signature) is included in this skill below — consult it for the exact method instead of guessing or falling back to fetch.

Preprocessor Scripts

For preprocessor scripts, the function should be named preprocessor and receives an event parameter:

type Event = {
  kind:
    | "webhook"
    | "http"
    | "websocket"
    | "kafka"
    | "email"
    | "nats"
    | "postgres"
    | "sqs"
    | "mqtt"
    | "gcp";
  body: any;
  headers: Record<string, string>;
  query: Record<string, string>;
};

export async function preprocessor(event: Event) {
  return {
    param1: event.body.field1,
    param2: event.query.id,
  };
}

S3 Object Operations

Windmill provides built-in support for S3-compatible storage operations. The wmill.S3Object type covers both the s3://storage/key URI form (s3:///key for the workspace default storage) and the { s3, storage? } record form — always use it instead of redefining your own.

Receiving an S3Object as a script parameter

import * as wmill from "windmill-client";

export async function main(file: wmill.S3Object) {
  const content = await wmill.loadS3File(file);
  // ...
}

S3 operations

import * as wmill from "windmill-client";

// Load file content from S3
const content: Uint8Array = await wmill.loadS3File(s3object);

// Load file as stream
const blob: Blob = await wmill.loadS3FileStream(s3object);

// Write file to S3
const result: wmill.S3Object = await wmill.writeS3File(
  s3object, // Target path (or undefined to auto-generate)
  fileContent, // string or Blob
  s3ResourcePath // Optional: specific S3 resource to use
);

DuckDB

Arguments are defined with comments and used with $name syntax:

-- $name (text) = default
-- $age (integer)
SELECT * FROM users WHERE name = $name AND age > $age;

Ducklake Integration

Attach Ducklake for data lake operations:

-- Main ducklake
ATTACH 'ducklake' AS dl;

-- Named ducklake
ATTACH 'ducklake://my_lake' AS dl;

-- Then query
SELECT * FROM dl.schema.table;

External Database Connections

Connect to external databases using resources:

ATTACH '$res:path/to/resource' AS db (TYPE postgres);
SELECT * FROM db.schema.table;

S3 File Operations

Read files from S3 storage:

-- Default storage
SELECT * FROM read_csv('s3:///path/to/file.csv');

-- Named storage
SELECT * FROM read_csv('s3://storage_name/path/to/file.csv');

-- Parquet files
SELECT * FROM read_parquet('s3:///path/to/file.parquet');

-- JSON files
SELECT * FROM read_json('s3:///path/to/file.json');

Receiving an S3Object as a script parameter

Declare the arg with type (s3object). Windmill renders an S3 file picker for it and binds the arg as the bare s3://storage/key URI, which DuckDB's reader functions consume directly:

-- $file (s3object)
SELECT * FROM read_parquet($file);

Works with any DuckDB reader: read_csv($file), read_json($file), etc.

Writing query results to S3

DuckDB writes to S3 natively via COPY ... TO:

COPY (SELECT * FROM users) TO 's3:///exports/users.parquet' (FORMAT PARQUET);

Use this instead of the -- s3 streaming directive supported by the other SQL dialects — that directive is not available in DuckDB.

Go

Structure

The file package must be inner and export a function called main:

package inner

func main(param1 string, param2 int) (map[string]interface{}, error) {
    return map[string]interface{}{
        "result": param1,
        "count":  param2,
    }, nil
}

Important:

  • Package must be inner
  • Return type must be ({return_type}, error)
  • Function name is main (lowercase)

Return Types

The return type can be any Go type that can be serialized to JSON:

package inner

type Result struct {
    Name  string `json:"name"`
    Count int    `json:"count"`
}

func main(name string, count int) (Result, error) {
    return Result{
        Name:  name,
        Count: count,
    }, nil
}

Error Handling

Return errors as the second return value:

package inner

import "errors"

func main(value int) (string, error) {
    if value < 0 {
        return "", errors.New("value must be positive")
    }
    return "success", nil
}

GraphQL

Structure

Write GraphQL queries or mutations. Arguments can be added as query parameters:

query GetUser($id: ID!) {
  user(id: $id) {
    id
    name
    email
  }
}

Variables

Variables are passed as script arguments and automatically bound to the query:

query SearchProducts($query: String!, $limit: Int = 10) {
  products(search: $query, first: $limit) {
    edges {
      node {
        id
        name
        price
      }
    }
  }
}

Mutations

mutation CreateUser($input: CreateUserInput!) {
  createUser(input: $input) {
    id
    name
    createdAt
  }
}

Java

The script must contain a Main public class with a public static main() method:

public class Main {
    public static Object main(String name, int count) {
        java.util.Map<String, Object> result = new java.util.HashMap<>();
        result.put("name", name);
        result.put("count", count);
        return result;
    }
}

Important:

  • Class must be named Main
  • Method must be public static Object main(...)
  • Return type is Object or void

Maven Dependencies

Add dependencies using comments at the top:

//requirements:
//com.google.code.gson:gson:2.10.1
//org.apache.httpcomponents:httpclient:4.5.14

import com.google.gson.Gson;

public class Main {
    public static Object main(String input) {
        Gson gson = new Gson();
        return gson.fromJson(input, Object.class);
    }
}

Microsoft SQL Server (MSSQL)

Arguments use @P1, @P2, etc.

Name the parameters by adding comments before the statement:

-- @P1 name1 (varchar)
-- @P2 name2 (int) = 0
SELECT * FROM users WHERE name = @P1 AND age > @P2;

Receiving an S3Object as a script parameter

Declare the arg with type (s3object). Windmill renders an S3 file picker for it, downloads the file, and binds it as nvarchar(max) JSON text — Parquet/CSV files are decoded server-side into a JSON array of records, JSON/JSONL pass through. Consume with OPENJSON:

-- @P1 file (s3object)
SELECT id, name
FROM OPENJSON(@P1)
WITH (id INT, name NVARCHAR(200));

Streaming query results to S3

Add a -- s3 directive at the top of the script to stream the result set to S3 instead of returning rows. Windmill writes the file and returns its S3Object as the script result.

-- s3 prefix=exports/users format=parquet
SELECT id, name FROM users;

All keys are optional: prefix (object key prefix), storage (named storage — omit to use the workspace default), format (json (default), parquet, or csv). Use this for large result sets — rows stream directly to S3 instead of being buffered as the script return value.

MySQL

Arguments use ? placeholders.

Name the parameters by adding comments before the statement:

-- ? name1 (text)
-- ? name2 (int) = 0
SELECT * FROM users WHERE name = ? AND age > ?;

Receiving an S3Object as a script parameter

Declare the arg with type (s3object). Windmill renders an S3 file picker for it, downloads the file, and binds it as JSON text — Parquet/CSV files are decoded server-side into a JSON array of records, JSON/JSONL pass through. Consume with JSON_TABLE:

-- ? file (s3object)
SELECT id, name
FROM JSON_TABLE(?, '$[*]'
  COLUMNS (id INT PATH '$.id', name VARCHAR(200) PATH '$.name')
) AS r;

Streaming query results to S3

Add a -- s3 directive at the top of the script to stream the result set to S3 instead of returning rows. Windmill writes the file and returns its S3Object as the script result.

-- s3 prefix=exports/users format=parquet
SELECT id, name FROM users;

All keys are optional: prefix (object key prefix), storage (named storage — omit to use the workspace default), format (json (default), parquet, or csv). Use this for large result sets — rows stream directly to S3 instead of being buffered as the script return value.

PHP

Structure

The script must start with <?php and contain at least one function called main:

<?php

function main(string $param1, int $param2) {
    return ["result" => $param1, "count" => $param2];
}

Resource Types

On Windmill, credentials and configuration are stored in resources and passed as parameters to main.

You need to redefine the type of the resources that are needed before the main function. Always check if the class already exists using class_exists:

<?php

if (!class_exists('Postgresql')) {
    class Postgresql {
        public string $host;
        public int $port;
        public string $user;
        public string $password;
        public string $dbname;
    }
}

function main(Postgresql $db) {
    // $db contains the database connection details
}

The resource type name has to be exactly as specified.

Library Dependencies

Specify library dependencies as comments before the main function:

<?php

// require:
// guzzlehttp/guzzle
// stripe/stripe-php@^10.0

function main() {
    // Libraries are available
}

One dependency per line. No need to require autoload, it is already done.

PostgreSQL

Arguments are obtained directly in the statement with $1::{type}, $2::{type}, etc.

Name the parameters by adding comments at the beginning of the script (without specifying the type):

-- $1 name1
-- $2 name2 = default_value
SELECT * FROM users WHERE name = $1::TEXT AND age > $2::INT;

Receiving an S3Object as a script parameter

Declare the arg with type (s3object). Windmill renders an S3 file picker for it, downloads the file, and binds it as a jsonb parameter — Parquet/CSV files are decoded server-side into a JSON array of records, JSON/JSONL pass through. Consume with jsonb_to_recordset (or any jsonb API):

-- $1 file (s3object)
SELECT *
FROM jsonb_to_recordset($1::jsonb) AS r(id INT, name TEXT);

Streaming query results to S3

Add a -- s3 directive at the top of the script to stream the result set to S3 instead of returning rows. Windmill writes the file and returns its S3Object as the script result.

-- s3 prefix=exports/users format=parquet
SELECT id, name FROM users;

All keys are optional: prefix (object key prefix), storage (named storage — omit to use the workspace default), format (json (default), parquet, or csv). Use this for large result sets — rows stream directly to S3 instead of being buffered as the script return value.

PowerShell

Structure

Arguments are obtained by calling the param function on the first line:

param($Name, $Count = 0, [int]$Age)

# Your code here
Write-Output "Processing $Name, count: $Count, age: $Age"

# Return object
@{
    name = $Name
    count = $Count
    age = $Age
}

Parameter Types

You can specify types for parameters:

param(
    [string]$Name,
    [int]$Count = 0,
    [bool]$Enabled = $true,
    [array]$Items
)

@{
    name = $Name
    count = $Count
    enabled = $Enabled
    items = $Items
}

Return Values

Return values by outputting them at the end of the script:

param($Input)

$result = @{
    processed = $true
    data = $Input
    timestamp = Get-Date -Format "o"
}

$result

Python

Structure

The script must contain at least one function called main:

def main(param1: str, param2: int):
    # Your code here
    return {"result": param1, "count": param2}

Do not call the main function. Libraries are installed automatically.

Resource Types

On Windmill, credentials and configuration are stored in resources and passed as parameters to main.

You need to redefine the type of the resources that are needed before the main function as TypedDict:

from typing import TypedDict

class postgresql(TypedDict):
    host: str
    port: int
    user: str
    password: str
    dbname: str

def main(db: postgresql):
    # db contains the database connection details
    pass

Important rules:

  • The resource type name must be IN LOWERCASE
  • Only include resource types if they are actually needed
  • If an import conflicts with a resource type name, rename the imported object, not the type name
  • Make sure to import TypedDict from typing if you're using it

Imports

Libraries are installed automatically. Do not show installation instructions.

import requests
import pandas as pd
from datetime import datetime

If an import name conflicts with a resource type:

# Wrong - don't rename the type
import stripe as stripe_lib
class stripe_type(TypedDict): ...

# Correct - rename the import
import stripe as stripe_sdk
class stripe(TypedDict):
    api_key: str

Windmill Client

Import the windmill client for platform interactions:

import wmill

See the SDK documentation for available methods.

Preprocessor Scripts

For preprocessor scripts, the function should be named preprocessor and receives an event parameter:

from typing import TypedDict, Literal, Any

class Event(TypedDict):
    kind: Literal["webhook", "http", "websocket", "kafka", "email", "nats", "postgres", "sqs", "mqtt", "gcp"]
    body: Any
    headers: dict[str, str]
    query: dict[str, str]

def preprocessor(event: Event):
    # Transform the event into flow input parameters
    return {
        "param1": event["body"]["field1"],
        "param2": event["query"]["id"]
    }

S3 Object Operations

Windmill provides built-in support for S3-compatible storage operations.

Receiving an S3Object as a script parameter

To accept a file from S3 as input to a script, type the parameter with S3Object (imported from wmill):

import wmill
from wmill import S3Object

def main(file: S3Object):
    content = wmill.load_s3_file(file)
    # ...

S3 operations

import wmill

# Load file content from S3
content: bytes = wmill.load_s3_file(s3object)

# Load file as stream reader
reader: BufferedReader = wmill.load_s3_file_reader(s3object)

# Write file to S3
result: S3Object = wmill.write_s3_file(
    s3object,           # Target path (or None to auto-generate)
    file_content,       # bytes or BufferedReader
    s3_resource_path,   # Optional: specific S3 resource
    content_type,       # Optional: MIME type
    content_disposition # Optional: Content-Disposition header
)

R

Structure

Define a main function using <- or = assignment. Parameters become the script inputs:

library(dplyr)
library(jsonlite)

main <- function(x, name = "default", flag = TRUE) {
    df <- tibble(x = x, name = name)
    result <- df %>% mutate(greeting = paste("Hello", name))
    return(toJSON(result, auto_unbox = TRUE))
}

Important:

  • The main function is required
  • Use library() to load packages — they are resolved and installed automatically
  • jsonlite is always available (used internally for argument parsing)
  • Return values must be JSON-serializable

Parameters

R types map to Windmill types:

  • numeric → float/int
  • character → string
  • logical → bool (use TRUE/FALSE)
  • list → object/dict
  • NULL → null

Default values are inferred from the function signature:

main <- function(
    name,              # required string
    count = 10,        # optional int, default 10
    verbose = FALSE    # optional bool, default FALSE
) {
    # ...
}

Resources and Variables

Use the built-in Windmill helpers (no import needed):

main <- function() {
    # Get a variable
    api_key <- get_variable("f/my_folder/api_key")

    # Get a resource (returns a list)
    db <- get_resource("f/my_folder/postgres_config")
    host <- db$host
    port <- db$port

    return(list(host = host, port = port))
}

Output

Return any JSON-serializable value from main. The return value becomes the step result:

main <- function(x) {
    # Return a scalar
    return(x + 1)

    # Or a list (becomes JSON object)
    return(list(result = x + 1, status = "ok"))
}

Annotations

Control execution behavior with comment annotations:

#renv_verbose = true        # Show verbose renv output during resolution
#renv_install_verbose = true # Show verbose output during package installation
#sandbox = true              # Run in nsjail sandbox (requires nsjail)

Rust

Structure

The script must contain a function called main with proper return type:

use anyhow::anyhow;
use serde::Serialize;

#[derive(Serialize, Debug)]
struct ReturnType {
    result: String,
    count: i32,
}

fn main(param1: String, param2: i32) -> anyhow::Result<ReturnType> {
    Ok(ReturnType {
        result: param1,
        count: param2,
    })
}

Important:

  • Arguments should be owned types
  • Return type must be serializable (#[derive(Serialize)])
  • Return type is anyhow::Result<T>

Dependencies

Packages must be specified with a partial cargo.toml at the beginning of the script:

//! ```cargo
//! [dependencies]
//! anyhow = "1.0.86"
//! reqwest = { version = "0.11", features = ["json"] }
//! tokio = { version = "1", features = ["full"] }
//! ```

use anyhow::anyhow;
// ... rest of the code

Note: Serde is already included, no need to add it again.

Async Functions

If you need to handle async functions (e.g., using tokio), keep the main function sync and create the runtime inside:

//! ```cargo
//! [dependencies]
//! anyhow = "1.0.86"
//! tokio = { version = "1", features = ["full"] }
//! reqwest = { version = "0.11", features = ["json"] }
//! ```

use anyhow::anyhow;
use serde::Serialize;

#[derive(Serialize, Debug)]
struct Response {
    data: String,
}

fn main(url: String) -> anyhow::Result<Response> {
    let rt = tokio::runtime::Runtime::new()?;
    rt.block_on(async {
        let resp = reqwest::get(&url).await?.text().await?;
        Ok(Response { data: resp })
    })
}

Snowflake

Arguments use ? placeholders.

Name the parameters by adding comments before the statement:

-- ? name1 (text)
-- ? name2 (number) = 0
SELECT * FROM users WHERE name = ? AND age > ?;

Receiving an S3Object as a script parameter

Declare the arg with type (s3object). Windmill renders an S3 file picker for it, downloads the file, and binds it as JSON text — Parquet/CSV files are decoded server-side into a JSON array of records, JSON/JSONL pass through. Wrap the bind with PARSE_JSON(?) and walk it with LATERAL FLATTEN:

-- ? file (s3object)
SELECT
  v.value:id::NUMBER AS id,
  v.value:name::STRING AS name
FROM LATERAL FLATTEN(input => PARSE_JSON(?)) v;

Streaming query results to S3

Add a -- s3 directive at the top of the script to stream the result set to S3 instead of returning rows. Windmill writes the file and returns its S3Object as the script result.

-- s3 prefix=exports/users format=parquet
SELECT id, name FROM users;

All keys are optional: prefix (object key prefix), storage (named storage — omit to use the workspace default), format (json (default), parquet, or csv). Use this for large result sets — rows stream directly to S3 instead of being buffered, bypassing the 10000-row return cap.

TypeScript SDK (windmill-client)

Import: import * as wmill from 'windmill-client'

The client configures itself from the job's environment — base URL, token and credentials mode are all set before your code runs, so there is nothing to initialize and no reason to read WM_TOKEN or BASE_INTERNAL_URL and build an API URL yourself. Reconstructing that by hand only reintroduces details the client already handles. Call the SDK for anything Windmill, and use raw HTTP for third-party APIs.

The helpers below are the surface to prefer. For an endpoint none of them covers, import the generated service classes (JobService, ScriptService, ...) from 'windmill-client' — they are not listed here but they do exist. What does not exist is a helper name you guessed at: if it is neither listed below nor a service method, do not call it.

To know who is running the script, read the contextual variables rather than calling the API: process.env.WM_END_USER_EMAIL || process.env.WM_EMAIL. WM_END_USER_EMAIL is the app viewer when the run was triggered from an app and empty otherwise (both variables are always defined), WM_EMAIL is the user the job is permissioned as. WM_USERNAME is the matching username.

workerHasInternalServer(): boolean

/**

  • Initialize the Windmill client with authentication token and base URL
  • @param token - Authentication token (defaults to WM_TOKEN env variable)
  • @param baseUrl - API base URL (defaults to BASE_INTERNAL_URL or BASE_URL env variable) */ setClient(token?: string, baseUrl?: string): void

/**

  • Create a client configuration from env variables
  • @returns client configuration */ getWorkspace(): string

/**

  • Get a resource value by path
  • @param path path of the resource, default to internal state path
  • @param undefinedIfEmpty if the resource does not exist, return undefined instead of throwing an error
  • @returns resource value */ async getResource(path?: string, undefinedIfEmpty?: boolean): Promise

/**

  • Get the true root job id
  • @param jobId job id to get the root job id from (default to current job)
  • @returns root job id */ async getRootJobId(jobId?: string): Promise

/**

  • Run a script synchronously by its path and wait for the result
  • @param path - Script path in Windmill
  • @param args - Arguments to pass to the script
  • @param verbose - Enable verbose logging
  • @param tag - Override the worker tag the job runs on
  • @returns Script execution result */ async runScriptByPath(path: string, args: Record<string, any> | null = null, verbose: boolean = false, tag: string | null = null): Promise

/**

  • Run a script synchronously by its hash and wait for the result
  • @param hash_ - Script hash in Windmill
  • @param args - Arguments to pass to the script
  • @param verbose - Enable verbose logging
  • @param tag - Override the worker tag the job runs on
  • @returns Script execution result */ async runScriptByHash(hash_: string, args: Record<string, any> | null = null, verbose: boolean = false, tag: string | null = null): Promise

/**

  • Append a text to the result stream
  • @param text text to append to the result stream */ appendToResultStream(text: string): void

/**

  • Stream to the result stream
  • @param stream stream to stream to the result stream */ async streamResult(stream: AsyncIterable): Promise

/**

  • Run a flow synchronously by its path and wait for the result
  • @param path - Flow path in Windmill
  • @param args - Arguments to pass to the flow
  • @param verbose - Enable verbose logging
  • @param tag - Override the worker tag the job runs on
  • @returns Flow execution result */ async runFlow(path: string | null = null, args: Record<string, any> | null = null, verbose: boolean = false, tag: string | null = null): Promise

/**

  • Wait for a job to complete and return its result
  • @param jobId - ID of the job to wait for
  • @param verbose - Enable verbose logging
  • @returns Job result when completed */ async waitJob(jobId: string, verbose: boolean = false): Promise

/**

  • Get the result of a completed job
  • @param jobId - ID of the completed job
  • @returns Job result */ async getResult(jobId: string): Promise

/**

  • Get the result of a job if completed, or its current status
  • @param jobId - ID of the job
  • @returns Object with started, completed, success, and result properties */ async getResultMaybe(jobId: string): Promise

/**

  • Cancel a queued or running job by ID.
  • @param jobId - UUID of the job to cancel
  • @param reason - Optional reason for cancellation
  • @returns Response message from the cancel endpoint */ async cancelJob(jobId: string, reason: string | undefined = undefined): Promise

/**

  • Run a script asynchronously by its path
  • @param path - Script path in Windmill
  • @param args - Arguments to pass to the script
  • @param scheduledInSeconds - Schedule execution for a future time (in seconds)
  • @param tag - Override the worker tag the job runs on
  • @returns Job ID of the created job */ async runScriptByPathAsync(path: string, args: Record<string, any> | null = null, scheduledInSeconds: number | null = null, tag: string | null = null): Promise

/**

  • Run a script asynchronously by its hash
  • @param hash_ - Script hash in Windmill
  • @param args - Arguments to pass to the script
  • @param scheduledInSeconds - Schedule execution for a future time (in seconds)
  • @param tag - Override the worker tag the job runs on
  • @returns Job ID of the created job */ async runScriptByHashAsync(hash_: string, args: Record<string, any> | null = null, scheduledInSeconds: number | null = null, tag: string | null = null): Promise

/**

  • Run a flow asynchronously by its path
  • @param path - Flow path in Windmill
  • @param args - Arguments to pass to the flow
  • @param scheduledInSeconds - Schedule execution for a future time (in seconds)
  • @param doNotTrackInParent - If false, tracks state in parent job (only use when fully awaiting the job)
  • @param tag - Override the worker tag the job runs on
  • @returns Job ID of the created job */ async runFlowAsync(path: string | null, args: Record<string, any> | null, scheduledInSeconds: number | null = null, // can only be set to false if this the job will be fully await and not concurrent with any other job // as otherwise the child flow and its own child will store their state in the parent job which will // lead to incorrectness and failures doNotTrackInParent: boolean = true, tag: string | null = null): Promise

/**

  • Resolve a resource value in case the default value was picked because the input payload was undefined
  • @param obj resource value or path of the resource under the format $res:path
  • @returns resource value */ async resolveDefaultResource(obj: any): Promise

/**

  • Get the state file path from environment variables
  • @returns State path string */ getStatePath(): string

/**

  • Set a resource value by path
  • @param path path of the resource to set, default to state path
  • @param value new value of the resource to set
  • @param initializeToTypeIfNotExist if the resource does not exist, initialize it with this type */ async setResource(value: any, path?: string, initializeToTypeIfNotExist?: string): Promise

/**

  • Set the state
  • @param state state to set
  • @param path Optional state resource path override. Defaults to getStatePath(). */ async setState(state: any, path?: string): Promise

/**

  • Set the progress
  • Progress cannot go back and limited to 0% to 99% range
  • @param percent Progress to set in %
  • @param jobId? Job to set progress for */ async setProgress(percent: number, jobId?: any): Promise

/**

  • Get the progress
  • @param jobId? Job to get progress from
  • @returns Optional clamped between 0 and 100 progress value */ async getProgress(jobId?: any): Promise<number | null>

/**

  • Set a flow user state
  • @param key key of the state
  • @param value value of the state */ async setFlowUserState(key: string, value: any, errorIfNotPossible?: boolean): Promise

/**

  • Get a flow user state
  • @param path path of the variable */ async getFlowUserState(key: string, errorIfNotPossible?: boolean): Promise

/**

  • Get the state shared across executions
  • @param path Optional state resource path override. Defaults to getStatePath(). */ async getState(path?: string): Promise

/**

  • Get a variable by path
  • @param path path of the variable
  • @returns variable value */ async getVariable(path: string): Promise

/**

  • Set a variable by path, create if not exist
  • @param path path of the variable
  • @param value value of the variable
  • @param isSecretIfNotExist if the variable does not exist, create it as secret or not (default: false)
  • @param descriptionIfNotExist if the variable does not exist, create it with this description (default: "") */ async setVariable(path: string, value: string, isSecretIfNotExist?: boolean, descriptionIfNotExist?: string): Promise

/**

  • Build a PostgreSQL connection URL from a database resource
  • @param path - Path to the database resource
  • @returns PostgreSQL connection URL string */ async databaseUrlFromResource(path: string): Promise

async polarsConnectionSettings(s3_resource_path: string | undefined): Promise

async duckdbConnectionSettings(s3_resource_path: string | undefined): Promise

/**

  • Get S3 client settings from a resource or workspace default
  • @param s3_resource_path - Path to S3 resource (uses workspace default if undefined)
  • @param workspace - Workspace to read from (defaults to the WM_WORKSPACE env var)
  • @returns S3 client configuration settings */ async denoS3LightClientSettings(s3_resource_path: string | undefined, workspace: string | undefined = undefined): Promise

/**

  • Load the content of a file stored in S3. If the s3ResourcePath is undefined, it will default to the workspace S3 resource.
  • let fileContent = await wmill.loadS3FileContent(inputFile)
  • // if the file is a raw text file, it can be decoded and printed directly:
  • const text = new TextDecoder().decode(fileContentStream)
  • console.log(text);
  • @param workspace - Workspace to read from (defaults to the WM_WORKSPACE env var) */ async loadS3File(s3object: S3Object, s3ResourcePath: string | undefined = undefined, workspace: string | undefined = undefined): Promise<Uint8Array | undefined>

/**

  • Load the content of a file stored in S3 as a stream. If the s3ResourcePath is undefined, it will default to the workspace S3 resource.
  • let fileContentBlob = await wmill.loadS3FileStream(inputFile)
  • // if the content is plain text, the blob can be read directly:
  • console.log(await fileContentBlob.text());
  • @param workspace - Workspace to read from (defaults to the WM_WORKSPACE env var) */ async loadS3FileStream(s3object: S3Object, s3ResourcePath: string | undefined = undefined, workspace: string | undefined = undefined): Promise<Blob | undefined>

/**

  • Persist a file to the S3 bucket. If the s3ResourcePath is undefined, it will default to the workspace S3 resource.
  • const s3object = await writeS3File(s3Object, "Hello Windmill!")
  • const fileContentAsUtf8Str = (await s3object.toArray()).toString('utf-8')
  • console.log(fileContentAsUtf8Str)
  • @param workspace - Workspace to write to (defaults to the WM_WORKSPACE env var) */ async writeS3File(s3object: S3Object | undefined, fileContent: string | Blob, s3ResourcePath: string | undefined = undefined, contentType: string | undefined = undefined, contentDisposition: string | undefined = undefined, workspace: string | undefined = undefined): Promise

/**

  • Permanently delete a file from S3 by key.
  • await wmill.deleteS3File({ s3: "path/to/file.txt" })
  • @param s3object - S3 object identifying the file to delete (must have s3 set)
  • @param workspace - Workspace to delete from (defaults to the WM_WORKSPACE env var) */ async deleteS3File(s3object: S3Object, workspace: string | undefined = undefined): Promise

/**

  • Sign S3 objects to be used by anonymous users in public apps
  • @param s3objects s3 objects to sign
  • @param expirySecs how long the signature stays valid, in seconds (default 43200 = 12h, clamped to [60, 604800])
  • @returns signed s3 objects */ async signS3Objects(s3objects: S3Object[], { expirySecs }: { expirySecs?: number } = {}): Promise<S3Object[]>

/**

  • Sign S3 object to be used by anonymous users in public apps
  • @param s3object s3 object to sign
  • @param expirySecs how long the signature stays valid, in seconds (default 43200 = 12h, clamped to [60, 604800])
  • @returns signed s3 object */ async signS3Object(s3object: S3Object, { expirySecs }: { expirySecs?: number } = {}): Promise

/**

  • Generate a presigned public URL for an array of S3 objects.
  • If an S3 object is not signed yet, it will be signed first.
  • @param s3Objects s3 objects to sign
  • @param expirySecs how long the signature stays valid, in seconds (default 43200 = 12h, clamped to [60, 604800])
  • @returns list of signed public URLs */ async getPresignedS3PublicUrls(s3Objects: S3Object[], { baseUrl, expirySecs }: { baseUrl?: string; expirySecs?: number } = {}): Promise<string[]>

/**

  • Generate a presigned public URL for an S3 object. If the S3 object is not signed yet, it will be signed first.
  • @param s3Object s3 object to sign
  • @param expirySecs how long the signature stays valid, in seconds (default 43200 = 12h, clamped to [60, 604800])
  • @returns signed public URL */ async getPresignedS3PublicUrl(s3Objects: S3Object, { baseUrl, expirySecs }: { baseUrl?: string; expirySecs?: number } = {}): Promise

/**

  • Get URLs needed for resuming a flow after this step
  • @param approver approver name
  • @param flowLevel if true, generate resume URLs for the parent flow instead of the specific step.
  •              This allows pre-approvals that can be consumed by any later suspend step in the same flow.
    
  • @returns approval page UI URL, resume and cancel API URLs for resuming the flow */ async getResumeUrls(approver?: string, flowLevel?: boolean): Promise<{ approvalPage: string; resume: string; cancel: string; }>

/**

  • Get an OIDC jwt token for auth to external services (e.g: Vault, AWS) (ee only)
  • @param audience audience of the token
  • @param expiresIn Optional number of seconds until the token expires
  • @returns jwt token */ async getIdToken(audience: string, expiresIn?: number): Promise

/**

  • Convert a base64-encoded string to Uint8Array
  • @param data - Base64-encoded string
  • @returns Decoded Uint8Array */ base64ToUint8Array(data: string): Uint8Array

/**

  • Convert a Uint8Array to base64-encoded string
  • @param arrayBuffer - Uint8Array to encode
  • @returns Base64-encoded string */ uint8ArrayToBase64(arrayBuffer: Uint8Array): string

/**

  • Sends an interactive approval request via Slack, allowing optional customization of the message, approver, and form fields.
  • [Enterprise Edition Only] To include form fields in the Slack approval request, go to Advanced -> Suspend -> Form
  • and define a form. Learn more at Windmill Documentation.
  • @param {Object} options - The configuration options for the Slack approval request.
  • @param {string} options.slackResourcePath - The path to the Slack resource in Windmill.
  • @param {string} options.channelId - The Slack channel ID where the approval request will be sent.
  • @param {string} [options.message] - Optional custom message to include in the Slack approval request.
  • @param {string} [options.approver] - Optional user ID or name of the approver for the request.
  • @param {DefaultArgs} [options.defaultArgsJson] - Optional object defining or overriding the default arguments to a form field.
  • @param {Enums} [options.dynamicEnumsJson] - Optional object overriding the enum default values of an enum form field.
  • @param {string} [options.resumeButtonText] - Optional text for the resume button.
  • @param {string} [options.cancelButtonText] - Optional text for the cancel button.
  • @returns {Promise} Resolves when the Slack approval request is successfully sent.
  • @throws {Error} If the function is not called within a flow or flow preview.
  • @throws {Error} If the JobService.getSlackApprovalPayload call fails.
  • Usage Example:
  • await requestInteractiveSlackApproval({
  • slackResourcePath: "/u/alex/my_slack_resource",
  • channelId: "admins-slack-channel",
  • message: "Please approve this request",
  • approver: "approver123",
  • defaultArgsJson: { key1: "value1", key2: 42 },
  • dynamicEnumsJson: { foo: ["choice1", "choice2"], bar: ["optionA", "optionB"] },
  • resumeButtonText: "Resume",
  • cancelButtonText: "Cancel",
  • });
  • Note: This function requires execution within a Windmill flow or flow preview. */ async requestInteractiveSlackApproval({ slackResourcePath, channelId, message, approver, defaultArgsJson, dynamicEnumsJson, resumeButtonText, cancelButtonText, }: SlackApprovalOptions): Promise

/**

  • Sends an interactive approval request via Teams, allowing optional customization of the message, approver, and form fields.
  • [Enterprise Edition Only] To include form fields in the Teams approval request, go to Advanced -> Suspend -> Form
  • and define a form. Learn more at Windmill Documentation.
  • @param {Object} options - The configuration options for the Teams approval request.
  • @param {string} options.teamName - The Teams team name where the approval request will be sent.
  • @param {string} options.channelName - The Teams channel name where the approval request will be sent.
  • @param {string} [options.message] - Optional custom message to include in the Teams approval request.
  • @param {string} [options.approver] - Optional user ID or name of the approver for the request.
  • @param {DefaultArgs} [options.defaultArgsJson] - Optional object defining or overriding the default arguments to a form field.
  • @param {Enums} [options.dynamicEnumsJson] - Optional object overriding the enum default values of an enum form field.
  • @returns {Promise} Resolves when the Teams approval request is successfully sent.
  • @throws {Error} If the function is not called within a flow or flow preview.
  • @throws {Error} If the JobService.getTeamsApprovalPayload call fails.
  • Usage Example:
  • await requestInteractiveTeamsApproval({
  • teamName: "admins-teams",
  • channelName: "admins-teams-channel",
  • message: "Please approve this request",
  • approver: "approver123",
  • defaultArgsJson: { key1: "value1", key2: 42 },
  • dynamicEnumsJson: { foo: ["choice1", "choice2"], bar: ["optionA", "optionB"] },
  • });
  • Note: This function requires execution within a Windmill flow or flow preview. */ async requestInteractiveTeamsApproval({ teamName, channelName, message, approver, defaultArgsJson, dynamicEnumsJson, }: TeamsApprovalOptions): Promise

setWorkflowCtx(ctx: WorkflowCtx | null): void

async sleep(seconds: number): Promise

/**

  • Execute fn inline and checkpoint the result. On replay the cached value is
  • returned without re-executing fn.
  • fn's result is encoded as JSON and decoded back before it is returned, so
  • the round that runs the body sees the same types every replay sees: a Date
  • comes back as a string, a Map as {}. {@link Jsonified} is that shape. */ async step(name: string, fn: () => T | Promise,): Promise<Jsonified<Awaited>>

/**

  • Create a task that dispatches to a separate Windmill script.
  • @example
  • const extract = taskScript("f/data/extract");
  • // inside workflow: await extract({ url: "https://..." }) */ taskScript(path: string, options?: TaskOptions): (...args: any[]) => PromiseLike

/**

  • Create a task that dispatches to a separate Windmill flow.
  • @example
  • const pipeline = taskFlow("f/etl/pipeline");
  • // inside workflow: await pipeline({ input: data }) */ taskFlow(path: string, options?: TaskOptions): (...args: any[]) => PromiseLike

/**

  • Mark an async function as a workflow-as-code entry point.
  • The function must be deterministic: given the same inputs it must call
  • tasks in the same order on every replay. Branching on task results is fine
  • (results are replayed from checkpoint), but branching on external state
  • (current time, random values, external API calls) must use step() to
  • checkpoint the value so replays see the same result. */ workflow(fn: (...args: any[]) => Promise): void

/**

  • Suspend the workflow and wait for an external approval.
  • Pass key to name the step, then getApprovalUrls(key) yields the URLs that
  • resume exactly this approval — route them through your own channel. Without a
  • key the steps are named approval, approval_2, ...
  • skin: "minimal" shows approvers only the request (form and approve/reject)
  • instead of the detailed page with the workflow's details. description is
  • shown above the form: a string, or a rich value such as { markdown: "..." }.
  • @example
  • const urls = await step("urls", () => getApprovalUrls("manager"));
  • await step("notify", () => sendEmail(urls.resume, urls.cancel));
  • const { value, approver } = await waitForApproval({ key: "manager", timeout: 3600 }); */ waitForApproval(options?: { timeout?: number; form?: object; selfApproval?: boolean; key?: string; skin?: "detailed" | "minimal"; description?: string | object; }): PromiseLike<{ value: any; approver: string; approved: boolean }>

/**

  • Resume/cancel/approval-page URLs bound to one waitForApproval step.
  • Unlike getResumeUrls(), which signs a random nonce, these address the very
  • resume_job record the step's built-in approval buttons use, so they are
  • stable across replays and safe to embed in a custom notification.
  • stepKey must match the key given to waitForApproval. Keys must be unique
  • within a workflow; reusing one throws rather than silently renaming it. The URL
  • only resumes while that step is awaiting approval; used at any other moment it is
  • rejected rather than banking a row a different approval would consume. Send it
  • ahead of time — approvers just cannot act before the workflow reaches the step.
  • resume and cancel are step-bound; approvalPage is not — it opens the job's
  • approval page, which acts on whichever approval is pending when it is used.
  • @example
  • const urls = await step("urls", () => getApprovalUrls("manager"));
  • await step("notify", () => sendEmail(urls.resume, urls.cancel));
  • await waitForApproval({ key: "manager" }); */ async getApprovalUrls(stepKey: string = "approval", approver?: string): Promise<{ approvalPage: string; resume: string; cancel: string; }>

/**

  • Process items in parallel with optional concurrency control.
  • Each item is processed by calling fn(item), which should be a task().
  • Items are dispatched in batches of concurrency (default: all at once).
  • @example
  • const process = task(async (item: string) => { ... });
  • const results = await parallel(items, process, { concurrency: 5 }); */ async parallel<T, R>(items: T[], fn: (item: T) => PromiseLike | R, options?: { concurrency?: number },): Promise<R[]>

/**

  • Commit Kafka offsets for a trigger with auto_commit disabled.
  • @param triggerPath - Path to the Kafka trigger (from event.wm_trigger.trigger_path)
  • @param topic - Kafka topic name (from event.topic)
  • @param partition - Partition number (from event.partition)
  • @param offset - Message offset to commit (from event.offset) */ async commitKafkaOffsets(triggerPath: string, topic: string, partition: number, offset: number,): Promise

/**

  • Parse an S3 object from URI string or record format
  • @param s3Object - S3 object as URI string (s3://storage/key, s3:///key
  • for the default storage) or record. Any other string throws rather than
  • falling back to an auto-generated key: an auto key is requested by
  • omitting the object, and a fallback would silently misplace the upload
  • on any typo.
  • @returns S3 object record with storage and s3 key */ parseS3Object(s3Object: S3Object): S3ObjectRecord

/**

  • Create a SQL template function for PostgreSQL/datatable queries
  • @param name - Database/datatable name (default: "main")
  • @param opts.role - Connect as this data table role instead of the data table's default one.
  • Only meaningful on a data table under roles, and only for a role you are a tenant of.
  • @returns SQL template function for building parameterized queries
  • @example
  • let sql = wmill.datatable()
  • let name = 'Robin'
  • let age = 21
  • await sql`
  • SELECT * FROM friends
  • WHERE name = ${name} AND age = ${age}::int
    
  • `.fetch()
  • @example
  • // Read through a restricted role
  • let sql = wmill.datatable("main", { role: "analytics" }) */ datatable(name: string = "main", opts?: DatatableOptions): DatatableSqlTemplateFunction

/**

  • Create a SQL template function for DuckDB/ducklake queries
  • @param name - DuckDB database name, optionally with a schema as name:schema (default: "main")
  • @returns SQL template function for building parameterized queries
  • @example
  • let sql = wmill.ducklake()
  • let name = 'Robin'
  • let age = 21
  • await sql`
  • SELECT * FROM friends
  • WHERE name = ${name} AND age = ${age}
    
  • `.fetch()
  • @example
  • // Target a specific schema within the ducklake
  • let sql = wmill.ducklake("my_lake:analytics") */ ducklake(name: string = "main"): SqlTemplateFunction

/**

  • Idempotently materialize selectSql into a ducklake table for one
  • partition (or the whole table when partition is omitted) — the client-side
  • equivalent of the // materialize engine.
  • With uniqueKey it upserts the slice (delete-by-key + insert); otherwise it
  • replaces it (whole table → CREATE OR REPLACE; partition → delete + insert).
  • Safe to re-run for the same partition (backfill / failure-recovery).
  • Returns a lazy statement — call .execute() to run it:
  • await wmill.upsertPartition({ table, selectSql, partition }).execute(). */ upsertPartition(opts: DucklakeMaterializeOptions): SqlStatement

/**

  • INSERT-only materialization (no dedup/replace) for append-only tables.
  • Re-running the same partition duplicates rows — use only for immutable
  • event-log sources.
  • Returns a lazy statement — call .execute() to run it:
  • await wmill.appendPartition({ table, selectSql, partition }).execute(). */ appendPartition(opts: Omit<DucklakeMaterializeOptions, "uniqueKey">,): SqlStatement

Python SDK (wmill)

Import: import wmill

The client configures itself from the job's environment — base URL, token and credentials mode are all set before your code runs, so there is nothing to initialize and no reason to read WM_TOKEN or BASE_INTERNAL_URL and build an API URL yourself. Reconstructing that by hand only reintroduces details the client already handles. Call the SDK for anything Windmill, and use raw HTTP for third-party APIs.

The functions below are the surface to prefer. For an endpoint none of them covers, wmill.Windmill().get(endpoint) and .post(endpoint) issue an authenticated request against this instance. What does not exist is a function name you guessed at: if it is not listed below, do not call it.

To know who is running the script, read the contextual variables rather than calling the API: os.environ.get("WM_END_USER_EMAIL") or os.environ.get("WM_EMAIL"). WM_END_USER_EMAIL is the app viewer when the run was triggered from an app and empty otherwise (both variables are always defined), WM_EMAIL is the user the job is permissioned as. WM_USERNAME is the matching username.

def worker_has_internal_server() -> bool

def get_mocked_api() -> Optional[dict]

Get the HTTP client instance.

Returns:

Configured httpx.Client for API requests

def get_client() -> httpx.Client

Make an HTTP GET request to the Windmill API.

Args:

endpoint: API endpoint path

raise_for_status: Whether to raise an exception on HTTP errors

**kwargs: Additional arguments passed to httpx.get

Returns:

HTTP response object

def get(endpoint, raise_for_status = True, **kwargs) -> httpx.Response

Make an HTTP POST request to the Windmill API.

Args:

endpoint: API endpoint path

raise_for_status: Whether to raise an exception on HTTP errors

**kwargs: Additional arguments passed to httpx.post

Returns:

HTTP response object

def post(endpoint, raise_for_status = True, **kwargs) -> httpx.Response

Create a new authentication token.

Args:

duration: Token validity duration (default: 1 day)

Returns:

New authentication token string

def create_token(duration = dt.timedelta(days=1)) -> str

Create a script job by path and return its job id.

def run_script_by_path_async(path: str, args: dict = None, scheduled_in_secs: int = None, tag: str = None) -> str

Create a script job by hash and return its job id.

def run_script_by_hash_async(hash_: str, args: dict = None, scheduled_in_secs: int = None, tag: str = None) -> str

Create a flow job and return its job id.

def run_flow_async(path: str, args: dict = None, scheduled_in_secs: int = None, do_not_track_in_parent: bool = True, tag: str = None) -> str

Run script by path synchronously and return its result.

def run_script_by_path(path: str, args: dict = None, timeout: dt.timedelta | int | float | None = None, verbose: bool = False, cleanup: bool = True, assert_result_is_not_none: bool = False, tag: str = None) -> Any

Run script by hash synchronously and return its result.

def run_script_by_hash(hash_: str, args: dict = None, timeout: dt.timedelta | int | float | None = None, verbose: bool = False, cleanup: bool = True, assert_result_is_not_none: bool = False, tag: str = None) -> Any

Run a script on the current worker without creating a job.

On agent workers (no internal server), falls back to running a normal

preview job and waiting for the result.

def run_inline_script_preview(content: str, language: str, args: dict = None) -> Any

Wait for a job to complete and return its result.

Args:

job_id: ID of the job to wait for

timeout: Maximum time to wait (seconds or timedelta)

verbose: Enable verbose logging

cleanup: Register cleanup handler to cancel job on exit

assert_result_is_not_none: Raise exception if result is None

Returns:

Job result when completed

Raises:

TimeoutError: If timeout is reached

Exception: If job fails

def wait_job(job_id, timeout: dt.timedelta | int | float | None = None, verbose: bool = False, cleanup: bool = True, assert_result_is_not_none: bool = False)

Cancel a specific job by ID.

Args:

job_id: UUID of the job to cancel

reason: Optional reason for cancellation

Returns:

Response message from the cancel endpoint

def cancel_job(job_id: str, reason: str = None) -> str

Cancel currently running executions of the same script.

def cancel_running() -> dict

Get job details by ID.

Args:

job_id: UUID of the job

Returns:

Job details dictionary

def get_job(job_id: str) -> dict

Get the root job ID for a flow hierarchy.

Args:

job_id: Job ID (defaults to current WM_JOB_ID)

Returns:

Root job ID

def get_root_job_id(job_id: str | None = None) -> dict

Get an OIDC JWT token for authentication to external services.

Args:

audience: Token audience (e.g., "vault", "aws")

expires_in: Optional expiration time in seconds

Returns:

JWT token string

def get_id_token(audience: str, expires_in: int | None = None) -> str

Get the status of a job.

Args:

job_id: UUID of the job

Returns:

Job status: "RUNNING", "WAITING", or "COMPLETED"

def get_job_status(job_id: str) -> JobStatus

Get the result of a completed job.

Args:

job_id: UUID of the completed job

assert_result_is_not_none: Raise exception if result is None

Returns:

Job result

def get_result(job_id: str, assert_result_is_not_none: bool = True) -> Any

Get a variable value by path.

Args:

path: Variable path in Windmill

Returns:

Variable value as string

def get_variable(path: str) -> str

Set a variable value by path, creating it if it doesn't exist.

Args:

path: Variable path in Windmill

value: Variable value to set

is_secret: Whether the variable should be secret (default: False)

def set_variable(path: str, value: str, is_secret: bool = False) -> None

Get a resource value by path.

Args:

path: Resource path in Windmill

none_if_undefined: Return None instead of raising if not found

interpolated: if variables and resources are fully unrolled

Returns:

Resource value dictionary or None

def get_resource(path: str, none_if_undefined: bool = False, interpolated: bool = True) -> dict | None

Set a resource value by path, creating it if it doesn't exist.

Args:

value: Resource value to set

path: Resource path in Windmill

resource_type: Resource type for creation

def set_resource(value: Any, path: str, resource_type: str)

List resources from Windmill workspace.

Args:

resource_type: Optional resource type to filter by (e.g., "postgresql", "mysql", "s3")

page: Optional page number for pagination

per_page: Optional number of results per page

Returns:

List of resource dictionaries

def list_resources(resource_type: str = None, page: int = None, per_page: int = None) -> list[dict]

Set the workflow state.

Args:

value: State value to set

path: Optional state resource path override.

def set_state(value: Any, path: str | None = None) -> None

Get the workflow state.

Args:

path: Optional state resource path override.

Returns:

State value or None if not set

def get_state(path: str | None = None) -> Any

Set job progress percentage (0-99).

Args:

value: Progress percentage

job_id: Job ID (defaults to current WM_JOB_ID)

def set_progress(value: int, job_id: Optional[str] = None)

Get job progress percentage.

Args:

job_id: Job ID (defaults to current WM_JOB_ID)

Returns:

Progress value (0-100) or None if not set

def get_progress(job_id: Optional[str] = None) -> Any

Set the user state of a flow at a given key

def set_flow_user_state(key: str, value: Any) -> None

Get the user state of a flow at a given key

def get_flow_user_state(key: str) -> Any

Get the Windmill server version.

Returns:

Version string

def version()

Convenient helpers that takes an S3 resource as input and returns the settings necessary to

initiate an S3 connection from DuckDB

def get_duckdb_connection_settings(s3_resource_path: str = '') -> DuckDbConnectionSettings | None

Convenient helpers that takes an S3 resource as input and returns the settings necessary to

initiate an S3 connection from Polars

def get_polars_connection_settings(s3_resource_path: str = '') -> PolarsConnectionSettings

Convenient helpers that takes an S3 resource as input and returns the settings necessary to

initiate an S3 connection using boto3

def get_boto3_connection_settings(s3_resource_path: str = '') -> Boto3ConnectionSettings

Load a file from the workspace s3 bucket and returns its content as bytes.

'''python

from wmill import S3Object

s3_obj = S3Object(s3="/path/to/my_file.txt")

my_obj_content = client.load_s3_file(s3_obj)

file_content = my_obj_content.decode("utf-8")

'''

def load_s3_file(s3object: S3Object | str, s3_resource_path: str | None) -> bytes

Load a file from the workspace s3 bucket and returns the bytes stream.

'''python

from wmill import S3Object

s3_obj = S3Object(s3="/path/to/my_file.txt")

with wmill.load_s3_file_reader(s3object, s3_resource_path) as file_reader:

print(file_reader.read())

'''

def load_s3_file_reader(s3object: S3Object | str, s3_resource_path: str | None) -> BufferedReader

Write a file to the workspace S3 bucket

'''python

from wmill import S3Object

s3_obj = S3Object(s3="/path/to/my_file.txt")

# for an in memory bytes array:

file_content = b'Hello Windmill!'

client.write_s3_file(s3_obj, file_content)

# for a file:

with open("my_file.txt", "rb") as my_file:

client.write_s3_file(s3_obj, my_file)

'''

def write_s3_file(s3object: S3Object | str | None, file_content: BufferedReader | bytes, s3_resource_path: str | None, content_type: str | None = None, content_disposition: str | None = None) -> S3Object

Permanently delete a file from the workspace S3 bucket.

'''python

from wmill import S3Object

s3_obj = S3Object(s3="/path/to/my_file.txt")

client.delete_s3_object(s3_obj)

'''

def delete_s3_object(s3object: S3Object | str, s3_resource_path: str | None = None) -> None

Sign S3 objects for use by anonymous users in public apps.

Args:

s3_objects: List of S3 objects to sign

expiry_secs: How long the signature stays valid, in seconds

(defaults to 43200 = 12h, clamped to [60, 604800])

Returns:

List of signed S3 objects

def sign_s3_objects(s3_objects: list[S3Object | str], expiry_secs: int | None = None) -> list[S3Object]

Sign a single S3 object for use by anonymous users in public apps.

Args:

s3_object: S3 object to sign

expiry_secs: How long the signature stays valid, in seconds

(defaults to 43200 = 12h, clamped to [60, 604800])

Returns:

Signed S3 object

def sign_s3_object(s3_object: S3Object | str, expiry_secs: int | None = None) -> S3Object

Generate presigned public URLs for an array of S3 objects.

If an S3 object is not signed yet, it will be signed first.

Args:

s3_objects: List of S3 objects to sign

base_url: Optional base URL for the presigned URLs (defaults to WM_BASE_URL)

expiry_secs: How long the signatures stay valid, in seconds

(defaults to 43200 = 12h, clamped to [60, 604800])

Returns:

List of signed public URLs

Example:

>>> s3_objs = [S3Object(s3="/path/to/file1.txt"), S3Object(s3="/path/to/file2.txt")]

>>> urls = client.get_presigned_s3_public_urls(s3_objs)

def get_presigned_s3_public_urls(s3_objects: list[S3Object | str], base_url: str | None = None, expiry_secs: int | None = None) -> list[str]

Generate a presigned public URL for an S3 object.

If the S3 object is not signed yet, it will be signed first.

Args:

s3_object: S3 object to sign

base_url: Optional base URL for the presigned URL (defaults to WM_BASE_URL)

expiry_secs: How long the signature stays valid, in seconds

(defaults to 43200 = 12h, clamped to [60, 604800])

Returns:

Signed public URL

Example:

>>> s3_obj = S3Object(s3="/path/to/file.txt")

>>> url = client.get_presigned_s3_public_url(s3_obj)

def get_presigned_s3_public_url(s3_object: S3Object | str, base_url: str | None = None, expiry_secs: int | None = None) -> str

Get the current user information.

Returns:

User details dictionary

def whoami() -> dict

Get the current user information (alias for whoami).

Returns:

User details dictionary

def user() -> dict

Get the state resource path from environment.

Returns:

State path string

def state_path() -> str

Get the workflow state.

Returns:

State value or None if not set

def state() -> Any

Set the state in the shared folder using pickle

def set_shared_state_pickle(value: Any, path: str = 'state.pickle') -> None

Get the state in the shared folder using pickle

def get_shared_state_pickle(path: str = 'state.pickle') -> Any

Set the state in the shared folder using pickle

def set_shared_state(value: Any, path: str = 'state.json') -> None

Get the state in the shared folder using pickle

def get_shared_state(path: str = 'state.json') -> None

Get URLs needed for resuming a flow after suspension.

Args:

approver: Optional approver name

flow_level: If True, generate resume URLs for the parent flow instead of the

specific step. This allows pre-approvals that can be consumed by any later

suspend step in the same flow.

Returns:

Dictionary with approvalPage, resume, and cancel URLs

def get_resume_urls(approver: str = None, flow_level: bool = None) -> dict

Get the resume URLs bound to one wait_for_approval step of this workflow.

Args:

step_key: Checkpoint key of the approval step, as passed to

wait_for_approval(key=...)

approver: Optional approver name

Returns:

Dictionary with approvalPage, resume, and cancel URLs

def get_approval_urls(step_key: str = 'approval', approver: str = None) -> dict

Sends an interactive approval request via Slack, allowing optional customization of the message, approver, and form fields.

[Enterprise Edition Only] To include form fields in the Slack approval request, use the "Advanced -> Suspend -> Form" functionality.

Learn more at: https://www.windmill.dev/docs/flows/flow_approval#form

:param slack_resource_path: The path to the Slack resource in Windmill.

:type slack_resource_path: str

:param channel_id: The Slack channel ID where the approval request will be sent.

:type channel_id: str

:param message: Optional custom message to include in the Slack approval request.

:type message: str, optional

:param approver: Optional user ID or name of the approver for the request.

:type approver: str, optional

:param default_args_json: Optional dictionary defining or overriding the default arguments for form fields.

:type default_args_json: dict, optional

:param dynamic_enums_json: Optional dictionary overriding the enum default values of enum form fields.

:type dynamic_enums_json: dict, optional

:raises Exception: If the function is not called within a flow or flow preview.

:raises Exception: If the required flow job or flow step environment variables are not set.

:return: None

Usage Example:

>>> client.request_interactive_slack_approval(

... slack_resource_path="/u/alex/my_slack_resource",

... channel_id="admins-slack-channel",

... message="Please approve this request",

... approver="approver123",

... default_args_json={"key1": "value1", "key2": 42},

... dynamic_enums_json={"foo": ["choice1", "choice2"], "bar": ["optionA", "optionB"]},

... )

Notes:

- This function must be executed within a Windmill flow or flow preview.

- The function checks for required environment variables (WM_FLOW_JOB_ID, WM_FLOW_STEP_ID) to ensure it is run in the appropriate context.

def request_interactive_slack_approval(slack_resource_path: str, channel_id: str, message: str = None, approver: str = None, default_args_json: dict = None, dynamic_enums_json: dict = None) -> None

Send a message to a Microsoft Teams conversation with conversation_id, where success is used to style the message

def send_teams_message(conversation_id: str, text: str, success: bool = True, card_block: dict = None)

Get a DataTable client for SQL queries.

Args:

name: Database name (default: "main")

role: Connect as this data table role instead of the data table's default one.

Only meaningful on a data table under roles, and only for a role you are a

tenant of.

Returns:

DataTableClient instance

def datatable(name: str = 'main', *, role: Optional[str] = None)

Get a DuckLake client for DuckDB queries.

Args:

name: Database name (default: "main")

Returns:

DucklakeClient instance

def ducklake(name: str = 'main')

def init_global_client(f)

def deprecate(in_favor_of: str)

Get the current workspace ID.

Returns:

Workspace ID string

def get_workspace() -> str

def get_version() -> str

Create a script job and return its job ID.

Args:

hash_or_path: Script hash or path (determined by presence of '/')

args: Script arguments

scheduled_in_secs: Delay before execution in seconds

tag: Override the worker tag the job runs on

Returns:

Job ID string

def run_script_async(hash_or_path: str, args: Dict[str, Any] = None, scheduled_in_secs: int = None, tag: str = None) -> str

Run a script synchronously by hash and return its result.

Args:

hash: Script hash

args: Script arguments

verbose: Enable verbose logging

assert_result_is_not_none: Raise exception if result is None

cleanup: Register cleanup handler to cancel job on exit

timeout: Maximum time to wait

tag: Override the worker tag the job runs on

Returns:

Script result

def run_script_sync(hash: str, args: Dict[str, Any] = None, verbose: bool = False, assert_result_is_not_none: bool = True, cleanup: bool = True, timeout: dt.timedelta = None, tag: str = None) -> Any

Run a script synchronously by path and return its result.

Args:

path: Script path

args: Script arguments

verbose: Enable verbose logging

assert_result_is_not_none: Raise exception if result is None

cleanup: Register cleanup handler to cancel job on exit

timeout: Maximum time to wait

tag: Override the worker tag the job runs on

Returns:

Script result

def run_script_by_path_sync(path: str, args: Dict[str, Any] = None, verbose: bool = False, assert_result_is_not_none: bool = True, cleanup: bool = True, timeout: dt.timedelta = None, tag: str = None) -> Any

Convenient helpers that takes an S3 resource as input and returns the settings necessary to

initiate an S3 connection from DuckDB

def duckdb_connection_settings(s3_resource_path: str = '') -> DuckDbConnectionSettings

Convenient helpers that takes an S3 resource as input and returns the settings necessary to

initiate an S3 connection from Polars

def polars_connection_settings(s3_resource_path: str = '') -> PolarsConnectionSettings

Convenient helpers that takes an S3 resource as input and returns the settings necessary to

initiate an S3 connection using boto3

def boto3_connection_settings(s3_resource_path: str = '') -> Boto3ConnectionSettings

Get the state resource path from environment.

Returns:

State path string

def get_state_path() -> str

Parse resource syntax from string.

def parse_resource_syntax(s: str) -> Optional[str]

Parse S3 object from a s3://<storage>/<key> URI string (s3:///<key>

for the default storage) or S3Object format. Any other string raises

rather than falling back to an auto-generated key: an auto key is

requested by omitting the object, and a fallback would silently misplace

the upload on any typo.

def parse_s3_object(s3_object: S3Object | str) -> S3Object

Parse variable syntax from string.

def parse_variable_syntax(s: str) -> Optional[str]

Append a text to the result stream.

Args:

text: text to append to the result stream

def append_to_result_stream(text: str) -> None

Stream to the result stream.

Args:

stream: stream to stream to the result stream

def stream_result(stream) -> None

Execute a SQL query against the DataTable.

Args:

sql: SQL query string with $1, $2, etc. placeholders

*args: Positional arguments to bind to query placeholders

Returns:

SqlQuery instance for fetching results

def query(sql: str, *args) -> SqlQuery

Idempotently materialize the rows of select_sql into ducklake

table for one partition (or the whole table when partition is

None). Client-side equivalent of the // materialize engine: with

unique_key it upserts within the slice (delete-by-key + insert);

without it, it replaces (whole table → CREATE OR REPLACE; partition →

delete the partition + insert). Re-running the same slice is safe — the

backfill / failure-recovery contract.

The partition value is bound as a DuckDB arg (never string-interpolated)

so it cannot inject SQL. select_sql is trusted (your own query).

def upsert_partition(table: str, select_sql: str, partition: str = None, unique_key: str = None, partition_col: str = '_wm_partition', schema: str = None)

INSERT-only materialization (no dedup / no replace) for an immutable

event-log table — for one partition, or the whole table when

partition is None. NOTE: unlike upsert_partition, re-running the same

slice duplicates rows — use only for append-only sources.

def append_partition(table: str, select_sql: str, partition: str = None, partition_col: str = '_wm_partition', schema: str = None)

Read a materialized ducklake table, optionally a single partition.

def read(table: str, partition: str = None, partition_col: str = '_wm_partition', schema: str = None)

Execute query and fetch results.

Args:

result_collection: Optional result collection mode

Returns:

Query results

def fetch(result_collection: str | None = None)

Execute query and fetch first row of results.

Returns:

First row of query results

def fetch_one()

Execute query and fetch first row of results. Return result as a scalar value.

Returns:

First row of query result as a scalar value

def fetch_one_scalar()

Execute query and don't return any results.

def execute()

DuckDB executor requires explicit argument types at declaration

These types exist in both DuckDB and Postgres

Check that the types exist if you plan to extend this function for other SQL engines.

def infer_sql_type(value) -> str

def parse_sql_client_name(name: str) -> tuple[str, Optional[str]]

Decorator that marks a function as a workflow task.

Works in both WAC v1 (sync, HTTP-based dispatch) and WAC v2

(async, checkpoint/replay) modes:

- v2 (inside @workflow): dispatches as a checkpoint step.

- v1 (WM_JOB_ID set, no @workflow): dispatches via HTTP API.

- Standalone: executes the function body directly.

A task runs as its own job, so its result is always encoded as JSON and

decoded back before the caller sees it: a datetime comes back as a

string, a tuple as a list.

retry re-dispatches the task after a failure, inside @workflow only.

Every attempt is a step of its own (call_api, call_api#2, ...) and

the wait between two of them is a durable sleep, so a retrying task holds no

worker while it backs off. Keys: attempts (retries after the first

failure, a whole number from 0 to 100), delay (seconds before the first

retry, sub-second delays dropped), multiplier (applied to the delay

after each attempt, 1 keeps it constant), max_delay (ceiling in

seconds). attempts is required, and an out-of-range or unknown key is

rejected where the policy is written.

A workflow sleeps once per round, so tasks backing off in the same fan-out

wait one after another rather than together: the delay before a fan-out

retries is the sum of every backoff pending in it, not the longest one, and

it grows with both the width of the fan-out and attempts. Retries with

no delay all go out in a single round.

cache_ttl serves a previous result of the task for that many seconds

instead of running it again. A task is keyed on its step key (its name and

call order) and the workflow's input, not on the arguments it is called

with, so cache one only when whether it runs, and what it receives, follow

from the workflow's input alone. A task_script target is keyed on the

arguments it is called with. It has no effect on a task_flow target,

which keeps its flow's own cache policy.

Usage::

@task

async def extract_data(url: str): ...

@task(path="f/external_script", timeout=600, tag="gpu")

async def run_external(x: int): ...

@task(retry={"attempts": 3, "delay": 30, "multiplier": 2})

async def call_api(payload: dict): ...

def task(_func = None, *, path: Optional[str] = None, tag: Optional[str] = None, timeout: Optional[int] = None, cache_ttl: Optional[int] = None, priority: Optional[int] = None, concurrency_limit: Optional[int] = None, concurrency_key: Optional[str] = None, concurrency_time_window_s: Optional[int] = None, retry: Optional[dict] = None)

Create a task that dispatches to a separate Windmill script.

retry takes the same policy as :func:task.

Usage::

extract = task_script("f/data/extract", timeout=600)

@workflow

async def main():

data = await extract(url="https://...")

def task_script(path: str, *, timeout: Optional[int] = None, tag: Optional[str] = None, cache_ttl: Optional[int] = None, priority: Optional[int] = None, concurrency_limit: Optional[int] = None, concurrency_key: Optional[str] = None, concurrency_time_window_s: Optional[int] = None, retry: Optional[dict] = None)

Create a task that dispatches to a separate Windmill flow.

retry takes the same policy as :func:task.

Usage::

pipeline = task_flow("f/etl/pipeline", priority=10)

@workflow

async def main():

result = await pipeline(input=data)

def task_flow(path: str, *, timeout: Optional[int] = None, tag: Optional[str] = None, cache_ttl: Optional[int] = None, priority: Optional[int] = None, concurrency_limit: Optional[int] = None, concurrency_key: Optional[str] = None, concurrency_time_window_s: Optional[int] = None, retry: Optional[dict] = None)

Decorator marking an async function as a workflow-as-code entry point.

The function must be deterministic: given the same inputs it must call

tasks in the same order on every replay. Branching on task results is fine

(results are replayed from checkpoint), but branching on external state

(current time, random values, external API calls) must use step() to

checkpoint the value so replays see the same result.

def workflow(func)

Execute fn inline and checkpoint the result.

On replay the cached value is returned without re-executing fn.

Use for lightweight deterministic operations (timestamps, random IDs,

config reads) that should not incur the overhead of a child job.

fn's result is encoded as JSON and decoded back before it is returned,

so the round that runs the body sees the same types every replay sees:

a datetime comes back as a string, a tuple as a list.

async def step(name: str, fn)

Server-side sleep — suspend the workflow for the given duration without holding a worker.

Inside a @workflow, the parent job suspends and auto-resumes after seconds.

Outside a workflow, falls back to asyncio.sleep.

async def sleep(seconds: int)

Suspend the workflow and wait for an external approval.

Pass key to name the step, then get_approval_urls(key) yields the URLs

that resume exactly this approval — route them through your own channel.

Without a key the steps are named approval, approval_2, ...

Returns a dict with value (form data), approver, and approved.

Args:

timeout: Approval timeout in seconds (default 1800).

form: Optional form schema for the approval page.

self_approval: Whether the user who triggered the flow can approve it (default True).

key: Optional checkpoint key naming this approval step.

skin: "minimal" shows approvers only the request (form and approve/reject)

instead of the detailed page with the workflow's details.

description: Shown to approvers above the form: a string, or a rich value such as

{"markdown": "..."}.

Example::

urls = await step("urls", lambda: get_approval_urls("manager"))

await step("notify", lambda: send_email(urls["resume"], urls["cancel"]))

result = await wait_for_approval(key="manager", timeout=3600)

async def wait_for_approval(timeout: int = 1800, form: dict | None = None, self_approval: bool = True, key: str | None = None, skin: Literal['detailed', 'minimal'] | None = None, description: str | dict | None = None) -> dict

Process items in parallel with optional concurrency control.

Each item is processed by calling fn(item), which should be a @task.

Items are dispatched in batches of concurrency (default: all at once).

Example::

@task

async def process(item: str):

...

results = await parallel(items, process, concurrency=5)

async def parallel(items, fn, *, concurrency: Optional[int] = None)

Commit Kafka offsets for a trigger with auto_commit disabled.

Args:

trigger_path: Path to the Kafka trigger (from event['wm_trigger']['trigger_path'])

topic: Kafka topic name (from event['topic'])

partition: Partition number (from event['partition'])

offset: Message offset to commit (from event['offset'])

def commit_kafka_offsets(trigger_path: str, topic: str, partition: int, offset: int) -> None