48 Commits

Include renames

Author	SHA1	Message	Date
3 people	0e807fb1dd	feat: put a data table's connection under Postgres roles (#11020) ... * feat(datatables): put a data table's connection under Postgres roles A data table backed by the instance database resolved to exactly one Postgres connection, `custom_instance_user`, for everyone who could reach it at all. There was no way to say this job reads, that one writes, this one never sees the salaries table. A data table role is now a real Postgres login on the cluster, defined once for the instance by a superadmin and named exactly as they named it. A script that declares `-- role analytics` connects as `analytics`, and Postgres decides what it may touch — grants are ordinary SQL. Windmill answers only "may this caller ask for this role", from the tenant lists on the data table entry: `u/alice`, `g/analysts`, `f/finance` or `*`. A data table with no `permissions` block behaves exactly as before. Everything that opens a connection on someone's behalf goes through one chokepoint, `get_datatable_resource_from_db`, which takes the identity explicitly and fails closed when there is none. The role logs in as itself — never `SET ROLE`, which a script could `RESET ROLE` its way out of. A fork's data table entry becomes a pointer at the workspace that governs it rather than a copy of it. The settings clone used to hand a fork a byte-identical entry naming the parent's database, which a fork admin could edit to grant themselves `admin` there; a pointer has nothing local to edit, and its tenants are evaluated as a member of the governing workspace, by email. `permissions` is stripped from the workspace export and ignored on import: tenants name principals of one workspace, and a settings push is not where an access decision should be made. Operations that see the whole database whatever the roles grant stay with the governing workspace's admins: editing the roles, a migration that declares none, and opening a replication stream for a Postgres trigger or capture. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_012ti5HyeTikPMYyW8YSdiHR * fix(datatables): gate the paths that reach a whole database as admin Auditing what still resolved through the unchecked resolver turned up three that act for a caller and hand back the admin connection: `resolve_pg_source_checked` (behind schema export, the full-schema read, database creation, import and the forked-database drop), the connection test, and the schema snapshot a fork clone takes of its parent. On a data table under roles each let any workspace member — or a fork admin who is nobody in the governing workspace — read or copy the whole database whatever its roles grant. All three now require admin reach on the governing workspace. A dump taken under a restricted role would be a silently truncated copy rather than an error, so refusing is the only right answer for the copy paths. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_012ti5HyeTikPMYyW8YSdiHR * fix(datatables): confine roles to the instance database, and stop a fork reaching the parent's bookkeeping A data table role is a login on Windmill's own Postgres. Nothing stopped a workspace admin putting a *resource-backed* data table under roles, at which point the executor dialled the host that resource names — one the admin chose — with the role's real cluster password, and `CONNECT` is granted to every registered instance database. Both ends now refuse: the permissions endpoint rejects the save, and the chokepoint refuses to substitute credentials on a non-instance entry rather than trusting the record it read. Two more places reached the governing database without answering to it. The initial-migration generator returned a `pg_dump` of the whole schema to any member. And the migration rename/delete cascade followed a fork's pointer into the parent, so a fork admin renaming or removing their own local entry relabelled or wiped the parent's `_wm_migrations` — after which the parent re-runs every migration from zero. The remote half is now skipped when the entry resolves into another workspace, which is also just correct: a fork renaming what it calls a data table changes nothing about the data table. Also: revoking a tenant now bounces the replication streams of every workspace holding an entry that resolves here, not only the governing one, so a fork's trigger stops rather than living on inside its open connection; the instance role catalog and the governing workspace's tenant lists are no longer returned to someone who cannot edit them; and the tenant rename dedup collapses non-adjacent duplicates, per role rather than once any role changed. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_012ti5HyeTikPMYyW8YSdiHR * fix(datatables): fail loudly where a role or a pointer can be left half-recorded Three ways the feature could end up in a state nobody could see or undo. Creating a role writes the cluster first and the catalog second, but the catalog write was an `UPDATE` that matched nothing when the instance Postgres settings row was absent — leaving a live login with a password nobody recorded: invisible to the catalog, un-recreatable because the name is taken, and un-deletable because there is no entry to delete. It now errors, so the operation is retryable once the row is restored. Deleting a workspace only nulls the fork lineage; the data table entries pointing at it are left resolving to nothing. Sweeping them is not an option — turning a pointer back into a copy would hand each fork the database outright — so the delete now names the data tables it stranded, and resolving one says which workspace is missing rather than reporting a data table this workspace never had. `InstanceDatatableRole` derived `Debug` while holding a Postgres password; it is now hand-written so `{:?}` on the catalog cannot put a live credential in a log line. Adds the two branches the reviews found unpinned: a caller who is not a member of the governing workspace at all, and `NoIdentity` — the compatibility path for an agent worker that predates this and sends no job id. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_012ti5HyeTikPMYyW8YSdiHR * fix(datatables): unbreak two operator messages and two comments that described other code The two strings this branch added for states an operator hits once — the catalog write that matched nothing, and the delete that stranded a pointer — were collapsed from their multi-line form with the indentation left in, so both rendered with a fourteen-space gap mid-sentence. `list_datatables` claimed to report a chain it cannot follow and then dropped it; it does drop it, and the comment now says why that is the right place to stay quiet. The non-superadmin check in `edit_datatable_config` was introduced as also covering references, which it does not and need not: `reference` is overwritten from the stored entry for every caller before the check runs. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_012ti5HyeTikPMYyW8YSdiHR * fix(datatables): serialize role catalog mutations, and state each helper's authorization contract The catalog is one JSON document, so create, rename, enable and delete are all read-modify-write. Two concurrent creates read the same snapshot, both succeed in the cluster, and the second write drops the first — leaving a live Postgres login with a password nobody recorded, which is the exact state the delete path exists to prevent. Every mutation now runs in one transaction holding an advisory lock across the read, the cluster DDL and the write, so a lost update cannot happen and a failure rolls the whole thing back. The DDL helpers take that transaction rather than the pool, which is what makes the lock cover them. Their statements moved off `sqlx::raw_sql`: the simple protocol is only needed for genuinely multi-statement SQL, and its future is not `Send`, which an axum handler holding the transaction requires. Each of these is one statement anyway. The new cross-crate surface now says what callers must do. `read_role_catalog` returns plaintext credentials; `create`/`rename`/`set_login`/`drop_instance_role` and `converge_connect_grants` mutate cluster-wide state; `read_datatable_entry` reads a workspace's raw config. All of them are superadmin-gated by their current handlers, but nothing said so at the definition, which is where the next caller looks. Also: the roles table reloads after a failed login toggle instead of leaving it claiming a flip that did not land; the rename affordance is the design-system `Button`, not a raw one; and `resolve_datatable_pg_as_caller` drops a `role` parameter no caller ever filled — browsing resolves as the data table's default until the database manager grows a picker. Why role passwords stay a plain `String` while the instance user's password beside them is a `StringOrSecretRef`, asked three times across reviews: that one is a secret ref because an operator supplies it and may want it from their own backend, while these are minted here and never entered by anyone, so there is nothing for a ref to point at. Encrypting generated secrets at rest is a separate change that would take the replication password with it. Now said at the field. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_012ti5HyeTikPMYyW8YSdiHR * fix(datatables): give the role catalog its own row, out of reach of the config machinery Putting it inside `custom_instance_pg_databases` was the wrong call, and it cost two ways. The catalog serializes a generated Postgres password per role, and that row is the operator-facing instance config, so the passwords reached `get_instance_config` and its YAML editor — a live cluster credential in a response body, a UI field and any log of either. Worse in the other direction: `to_settings_map` strips the catalog, so a full-row upsert of that key writes the row back without it and the catalog is gone, while the cluster keeps every login it described. `custom_instance_replication_pwd` is the precedent and says exactly why — a generated secret, written only by the server, never operator-authored, hidden so the config machinery cannot read, rewrite or drop it. The catalog is the same thing, so it now has the same shape: `datatable_roles`, in `HIDDEN_SETTINGS`, `PROTECTED_SETTINGS` and the agent-worker denylist. No redaction to keep in step with three code paths, and no way for a neighbouring write to take it out. Two races on the same shared documents. `edit_datatable_config` read the stored data tables outside its transaction and then wrote the whole `datatable` document, so a permissions save committing in between was silently rolled back; it now reads under `FOR UPDATE`. And `set_datatable_permissions` validated role ids against the catalog before opening its transaction, so a deletion in between let it write a deleted role back — including as the default, which every later job then fails on; it now holds the catalog lock and the settings row across validation and write. Completes the authorization contracts the previous commit claimed but did not finish: `read_datatable_entry` (which it named and missed), `resolve_governing_datatable`, whose whole job is to answer for a workspace the caller may not belong to, and `converge_connect_grants_with`, which had not inherited its wrapper's. Also the generic Python SDK reference: `_format_py_params` learned the bare `*` last time, but `extract_py_functions` is a second formatter and still rendered `datatable(name, role)`, so code written from that page passed a keyword-only argument positionally. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_012ti5HyeTikPMYyW8YSdiHR * fix(datatables): make the concurrency test pin the handlers, and the contracts describe what is enforced The concurrency test reimplemented the read-modify-write inline, so deleting the lock from all three handlers left it green — it pinned Postgres, not the code it was written for. It now drives `create_datatable_role` twice concurrently and asserts the catalog kept both names. Checked the way the last one should have been: removing the lock from the handler makes it fail with "wmtest_a_… is a live cluster login the catalog forgot". The contracts added last commit were stricter than this PR's own callers, which is worse than none — the next reader sees a rule already broken and learns to ignore it. `read_role_catalog` said superadmin-only while two of its four callers are open to any workspace member, and `converge_connect_grants` said superadmin while `set_datatable_permissions` reaches it as a workspace admin. Both were fine on substance: the rule that actually holds is about the credential never reaching a response, log, audit record or export, not about who may call. They now say that. `read_datatable_entry` gets the same treatment rather than the one the earlier message claimed for it: it is the primitive every resolution goes through, so it is deliberately open, and what must not escape is `permissions` — it names the governing workspace's users, groups and folders. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_012ti5HyeTikPMYyW8YSdiHR * fix(datatables): close the last ways a role or a pointer can be left pointing at nothing The raw settings readers hand back whatever is in the row, so moving the catalog into its own `global_settings` key protected the config machinery and left `GET /settings/global/datatable_roles` and the settings listing returning every live password. Both now filter that one key. The neighbouring `custom_instance_replication_pwd` has the same shape and is not touched here: it predates this and widening the fix to it is a decision about an operator workflow, not a consequence of this change. Three ways a save could leave something resolving to nothing: A permissioned data table could be moved to a PostgreSQL resource. The block was carried across as a server-owned field, the runtime refuses roles on a resource-backed table, so the save succeeded and every job afterwards failed. Refused instead — turning roles off first is one step, and it keeps discarding an access decision something somebody chose. Renaming a governing data table left every fork pointing at the old name: the data table disappears from their pickers and their jobs stop, with nothing in the renaming workspace to suggest why. The rename now follows into the pointers in the same transaction. Deleting one cannot be followed the same way, so it is reported instead — the response names what it stranded, the way deleting a workspace does, and the fork's own error already says which workspace is gone. Also: `ensure_instance_db_grant_options_unchecked` claimed superadmin while the permissions handler reaches it as a workspace admin (the same class fixed last commit, one instance missed); the role entry kept an `instance_config_schema` derive it no longer needs; `write_role_catalog` was the one writer of that table not stamping `updated_at`; and the concurrency test dropped its roles only on success — a failing run is exactly the one that creates them without recording them. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_012ti5HyeTikPMYyW8YSdiHR * refactor(datatables): put the role catalog in its own table, not in global_settings Five findings across three rounds were all the same choice. A set of live Postgres credentials was living in `global_settings`, which has generic read, list, write, config-export and CLI round-trip paths that know nothing about what they carry: the passwords reached the instance config and its YAML editor, a full-row upsert of a neighbouring key erased the catalog, `GET /settings/global/{key}` and the settings listing returned them raw, and this round the redaction that fixed the last two turned `wmill instance push` into something that wipes every password — a fix breaking the assumption the previous fix made. `POST /settings/global/datatable_roles` could also empty it outside the lock. The approved plan offered a table or `global_settings`, so this is the other option it already allowed rather than a new design. `datatable_role` is a table: no generic settings path can read it, list it, export it, write it or round-trip it, so none of the five needs a guard. The redaction, the hidden/protected/agent-denylist entries and the JSON document all go with it. One row per role also removes the read-modify-write the concurrency work was about: two concurrent creates are two inserts, and the unique index on `name` is what settles a collision. The advisory lock stays for the one window rows do not cover — `CREATE ROLE` is invisible to another transaction until commit, so without it both creates pass their `pg_roles` check. Also from this round: rename mappings are checked against the configuration they claim to describe, since fork pointers are rewritten from them — a caller could otherwise submit `main -> missing` against an unchanged config and repoint every fork of `main` at a name nothing has, and `A -> B` plus `B -> C` moved what pointed at `A` all the way to `C`. And the warning naming forks a delete stranded reached the response but not the screen: both the data table settings save and the workspace delete now show it. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_012ti5HyeTikPMYyW8YSdiHR * fix(datatables): validate a rename against the save it describes, and re-check under the locks Three from the round, all about deciding on state that could already have moved. A permission save resolved the data table and checked it was instance-backed before taking any lock, then wrote under one. A config save committing in between could move the table onto a PostgreSQL resource — recreating exactly what the transition guard refuses — or rename it, in which case the write targeted a key that no longer existed and reported success having changed nothing. It now re-resolves and re-checks on the locked state. Rename validation checked that the source existed before and the target existed after, which still accepts `main -> decoy` against a save that keeps both: every fork of `main` then follows onto a different data table, silently, because it keeps resolving. The rule is now the actual old-to-new key transition — a source may only survive if another rename took its name, and a target may only pre-exist if another rename freed it. That also stops two sources sharing one target, and it admits a swap, which the previous guard refused: `datatables` is keyed by name, so a swap cannot be done one save at a time, and refusing it was a regression against main. The pointer cascade now runs in two passes through a temporary name, the way the migration cascade one layer down already handles the same shape, so `A -> B` with `B -> C` moves each pointer once from what it named before the save. The tenant mutators say what they are for: they write an access decision for any workspace named, with an arbitrary mutation, and exist for the transaction that frees or renames a principal. Editing a decision on purpose belongs in the permissions endpoint. Carried in the same change: the stranded-fork list is a field rather than a phrase to grep out of a success string; the pointer cascade matches with `EXISTS` instead of a `LIKE` over the whole document, so a workspace whose pointers name something else is not rewritten to a byte-identical value under an exclusive lock; and `InstanceDatatableRole` drops the serde derives left over from the JSON document, one of which would emit `pwd`. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_012ti5HyeTikPMYyW8YSdiHR * fix(datatables): cascade on the leave route that is used, gate migrations before the admin connection, and drop a role atomically The tenant cascade on leaving went onto `/users/leave`. The UI and the generated client call `/workspaces/leave` — a different handler in a different crate with the same name — which deleted the membership and left `u/<username>` in the tenant lists. Leaving and rejoining therefore restored the access the leave was supposed to end, and a later account taking the username would have inherited it. The regression test drives the route the client actually calls; without the fix it fails with "leaving kept the tenant". The migration endpoints authorized too late. `run_datatable_migrations` opened the data table's admin connection, created `_wm_migrations` and read it before reaching the per-migration role check — so with nothing pending, nothing was checked at all. Rollback returned before its check when nothing was applied, and the status endpoint had none. All three now ask, before any connection is opened, whether the caller can reach the data table as any role at all; which role a given migration runs as is still decided per migration, and by the executor after that. Deleting a role committed the cluster drop and the catalog row, then swept the tenant lists in separate transactions. A sweep failing part-way left workspaces naming a role nothing can connect as, while the retry answered `NotFound` because the catalog entry was already gone. The sweep now runs in the same transaction, so the drop, the row and every tenant list commit together. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_012ti5HyeTikPMYyW8YSdiHR * fix(datatables): refuse to copy a data table that is under roles pg_dump carries no roles and the import runs with --no-privileges, so a copied data table arrives owned by the admin connection with no GRANT for any role. The settings clone brings `permissions` across, so the fork's tenants pass Windmill's check, connect as the role they were given, and are denied by Postgres on everything: an entry that reads as configured and answers nothing. Refuse the copy — in the import endpoint before any data moves, and in the fork path the CLI takes. Replaying the source's owners and ACLs into the clone is what lifts this, and is a change of its own. Dropping `permissions` from the copy instead would be the unsafe half, since the copy holds the parent's rows. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_012ti5HyeTikPMYyW8YSdiHR * fix(datatables): refuse the clone's database too, not only its data A clone is two endpoints: `create_pg_database` then `import_pg_database`. Only the second refused a data table under roles, so a fork asking to clone one created and registered an empty `wm_fork_…` instance database and then failed — and nothing collects it, since `drop_forked_datatable_databases` only drops entries carrying `forked_from` and no entry names this one. Refuse in both, so the clone stops before a database exists. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_012ti5HyeTikPMYyW8YSdiHR * nit worker error msg * fix pg_dump stuck on version 17 on nix * fix(datatables): refuse a malformed role annotation instead of ignoring it `-- Role operator`, `-- role operator;` and `-- role operator -- why` all failed the annotation parser's exact-match rule, so the query fell through to the data table's default role and ran, silently, under a login the author did not choose. Naming a role exists precisely to not do that. A leading comment whose first word is `role` is now an annotation attempt: the keyword matches case-insensitively, one trailing `;` is tolerated, and anything else is an error naming the line. Only callers that already know the target is a `datatable://` reference ever run this, so ordinary SQL keeps its comments. Also bumps the dev shell's postgres client to 18 — it trailed the server the dev database runs, which takes out every data table export, clone and fork-with-data. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_012ti5HyeTikPMYyW8YSdiHR * fix(datatables): refuse a malformed role query string instead of ignoring it `?Role=analytics`, `?role=` and `?x=1&role=…` all fell through the reference parser's exact-match rule, so the connection resolved to the data table's default role and ran under a login the caller never asked for — the URI half of the same trap as a malformed `-- role` annotation. The key now matches case-insensitively, and anything else in the query string is an error naming it; `role` is the only parameter a reference takes. Callers that only need the entry keep a lenient `datatable_ref_name`, since they never act on the role. The DuckDB `ATTACH` parser propagates it rather than attaching under the default. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_012ti5HyeTikPMYyW8YSdiHR * fix(datatables): carry the role annotation into the row_to_json retry The retry rebuilds its SQL from `pruneComments(code)`, so the leading comment block never reached the second attempt — and with it the `-- role <name>` line that decides which login the query runs as. The retry connected as the data table's default role instead, so a query the first attempt was denied could succeed on the second, reported as "recovered with the row_to_json fix". Carry the leading comment block over. The retry itself is unchanged. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01BjfMkJyKzodxkobqGZ6Lqb * chore(datatables): don't mount the roles UI until the ACL editor lands Enforcement ships first. The permissions drawer is what turns roles on, and the catalog section is what creates them — both are only useful once there is a way to grant a role the privileges it needs, which arrives with the ACL editor. Left mounted they would offer a feature whose other half does not exist. The two components are complete and reviewed; only their call sites here are commented out, with a note pointing the follow-up PRs at them. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01BjfMkJyKzodxkobqGZ6Lqb * fix(datatables): honour `-- role: x`, and fix the DuckDB attach test Two review findings, both real. `attach_datatable_parses_name_and_role` never compiled: `parse_attach_datatable` returns `Result<Option<_>>` now and one call site kept a single `unwrap`. Its `?Role=analytics` case also asserted a refusal, contradicting the parser in the same commit, which matches the key case-insensitively. Replaced with the cases that are genuinely malformed, and a positive one for the cased key. `-- role: analytics` fell through to the default role — the silent fallback the strict parser exists to remove, for the spelling most likely to be typed. The keyword now accepts an optional colon, attached or spaced, while a word that merely starts with it (`rolebased`) is still not an attempt. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01BjfMkJyKzodxkobqGZ6Lqb * fix(datatables): clone a fork's pointer instead of failing after the copy Forking a fork with cloning left an orphan database. The preflight resolves the pointer and sees the governing entry, so both endpoints ran and filled the new database; `apply_forked_datatable` then refused the inherited pointer and rolled the fork back, stranding a registered `wm_fork_*` that no entry names and whose name blocks the retry. Refusing earlier would have been the smaller change, but forking a fork and cloning worked before pointers existed, so it would trade an orphan for a regression. Resolve what the pointer names and write the terminal entry the clone needs: the whole `database` object rather than a patch of its `resource_path`, since a pointer has none, and `reference` removed with it. Also accepts `-- role=x` and `-- Role = x`, two more spellings that fell through to the default role. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01BjfMkJyKzodxkobqGZ6Lqb * fix(datatables): refuse to roll back the catalog while roles exist The down migration dropped the table and left every role behind: live Postgres logins whose passwords only that table carried, so after a revert Windmill could neither use, disable nor delete them, and re-applying could not recreate them because the names were taken. Cleaning up here is not possible either — dropping a role means reassigning what it owns in every instance database, and a migration runs in one — so it now refuses while the catalog is non-empty and says to delete the roles through instance settings, which does the cluster work. Also enforces the instance-only invariant the resolved-pointer clone relies on rather than only asserting it in a comment. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01BjfMkJyKzodxkobqGZ6Lqb * refactor(datatables): settle clonability in one place, before anything is created A clone is three stages a workspace apart — `create_pg_database`, then `import_pg_database`, then `apply_forked_datatable` inside the fork transaction. Only the third can roll back, and `CREATE DATABASE` is not transactional, so any refusal that lives there strands a registered `wm_fork_*` that no entry names and whose name blocks the retry. That orphan has now been fixed three times, most recently reintroduced by a guard added one commit ago. Patching each new refusal into the first endpoint is not the fix; having two places that can refuse is. `ensure_datatable_is_clonable` now answers every reason a copy can be refused and returns what it resolved, and the stage that writes the entry only does the work. Also takes an ACCESS EXCLUSIVE lock before the rollback guard counts, so a role created concurrently cannot slip between the check and the drop. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01BjfMkJyKzodxkobqGZ6Lqb * fix(datatables): let a retried clone reclaim its own leftover database A clone creates its target database one request before it copies into it, and the fork that would name it is written a request after that. Any failure in between — a pg_dump error, a bad restore, a dropped connection, the source's roles changing mid-flow — left a registered `wm_fork_*` that no entry names, and every retry then failed on its name. This predates data table roles. `create_pg_database` now reclaims such a leftover before creating: only a `wm_fork_*` database Windmill registered as a data table database and that no data table or ducklake entry names, in any workspace, archived ones included. The drop never terminates connections, so a clone still copying into it makes the reclaim fail instead of being cut off. It is limited to callers who administer the source — reaching it is not enough, since on a data table without roles every member reaches it — and anyone else gets the refusal an existing database always got. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * Revert "fix(datatables): let a retried clone reclaim its own leftover database" This reverts commit 7dd3275a10. The reclaim tied the caller to the source they administer, but not to the database it dropped. Between another workspace's import and its final fork request, that workspace's target is full, registered, unnamed and has no open connection, so an admin of any instance data table could name it and have it dropped and recreated empty. The victim's fork would then commit pointing at the empty copy. Safe reclaim needs durable clone ownership and serialization with the request that names the database; until then the leftover stays, as it did before this PR. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * docs(datatables): record the stale clone database as a known limitation A clone is three requests and `CREATE DATABASE` is not transactional, so a failure after the first leaves a registered `wm_fork_*` behind, as it did before data table roles. Accepted for this PR: it is harmless to data and goes away once the clone is a single server-side operation. The comment also records why the obvious fix is wrong: reclaiming the leftover on retry, without durable clone ownership, can drop another workspace's fully copied database between its import and its final fork request. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix(datatables): bounce the streams reading a data table when it is deleted Deleting a governing data table, or the workspace that holds it, only collected the fork pointers it stranded, for the warning. A Postgres trigger or capture already streaming through one of those pointers kept the replication connection it opened while the pointer still resolved, so it went on dispatching the governing database's rows after the fork lost access — until its connection happened to restart. The governing workspace's own streams on a deleted entry did the same. Both deletion paths now bounce the affected listeners inside their own transaction, through the helper a permission change already uses, so a listener that reconnects re-resolves the entry and finds it gone. The helper is split so a caller can pass the (workspace, local name) pairs it already holds. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix(datatables): keep the fork schema baseline, and bounce streams on every removal Three fixes from review. `edit_datatable_config` took `forked_from` wholesale from the stored entry, so the fork schema diff's save of an advanced baseline was silently discarded and an applied change was offered again. Whether an entry carries a clone stamp is still carried from the store, since that is what marks its database droppable, but the baseline inside it is now taken from the request. The stranded-pointer warning and the stream bounce ran over the optional `deleted_datatables` hint, which the settings-sync CLI never sends, so removing a governing data table through `wmill` bounced nothing. Removals are now derived from the stored configuration against the saved one. `delete_workspace` read the pointers to bounce before its transaction, so a fork committing a pointer during the deletion was missed. The read now happens inside the transaction, after the workspace row is deleted: a fork's insert key-share locks that row through its parent foreign key, so it is either seen or fails on the missing parent. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * refactor(datatables): keep Postgres triggers and data table roles apart A replication stream reads every row of every table whatever the data table's roles grant, and its listener checks access only when it connects. Rather than chase every way access can change and bounce the streams each one affects, a data table now carries one or the other: - a Postgres trigger or capture cannot be created on, or connect to, a data table under roles; - roles cannot be turned on while an enabled trigger or a live capture reads the data table, its own or a fork's through its pointer. The refusal names each one to disable. This removes the stream bounces on roles edits and on data table and workspace deletion, and the trigger gate that admitted admins. The fork schema baseline fix from the same review round is kept. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix(datatables): refuse a Postgres trigger on a data table under roles when it is saved Creating or editing a trigger that points at a data table under roles was accepted, and its listener then retried the refused connection every 30 seconds forever. The save is now refused, and a trigger that reaches such a data table anyway (re-enabled, or cloned into a fork) is disabled by its listener with the reason, as a missing replication slot is. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix(datatables): disable a data table role before deleting it Deleting a role reassigns and drops what it owns in each registered database on its own connection, and each of those passes commits as it goes. A database failing part-way left the role enabled in the catalog and able to log in, but already stripped in the databases reached before it. The role is now disabled in its own commit first, so a failed delete leaves a disabled role to retry. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix(datatables): serialize roles going on with a stream starting Turning roles on looked for enabled triggers and live captures once, without a lock anything starting a stream also took. A trigger enabled in that window could have its listener connect before roles committed, and a healthy listener never checks again. Both transitions now serialize on one advisory lock: roles going on hold it exclusive while they look, and trigger create, edit and enable, and capture setup and ping hold it shared while they commit. Either the look sees the stream, or the listener connects after roles are committed and refuses. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix(datatables): wait out live listeners, and resolve stored names containing `?` Turning roles on counted a trigger as gone once disabled, and a capture once its client stopped pinging, but the listener keeps its replication connection until its next heartbeat notices. A trigger or capture whose listener pinged in the last 15 seconds, the window a server holds a listener for, now still counts as streaming. Data table names could contain `?` before they were restricted, and such entries are still stored. Splitting `?role=` off a reference misread them: `a?b` became `a` with an unknown parameter, and the clone checks looked at a different entry than the one copied. An entry stored under the whole reference is now looked up first, in the Postgres executor, DuckDB ATTACH and the clone checks. Agent workers cannot read the workspace and keep the strict parse, which refuses such a name rather than misreading it. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix(datatables): warn when a settings sync strands fork pointers A settings save reported the fork pointers left resolving to nothing only for the names in `deleted_datatables`, which `wmill sync push` never sends. The save now works out what it removed from the locked entries, and the CLI prints the stranded pointers it returns. Also correct the replication helper's contract: no role or admin check makes a replication connection safe, so a data table under roles is refused outright rather than gated as an admin operation. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01BjfMkJyKzodxkobqGZ6Lqb * fix(datatables): refuse a save that drops a data table's roles through an undeclared rename A data table's roles follow its entry only through a declared rename. A settings sync sends the whole map and never declares one, so renaming a data table under roles there read as a delete and a new entry on the same database: the new entry carried no roles, and every caller connected as admin. Such a save is now refused, naming both entries. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01BjfMkJyKzodxkobqGZ6Lqb * fix(datatables): no entry without roles may newly reach a database under roles The previous guard only caught a new name replacing an entry under roles. A whole-map save could also repoint an existing entry without roles at that database, or another workspace could point one there, and every caller of that entry would connect as admin. The rule is now stated on the saved entries: one that carries no roles and newly points at an instance database any entry under roles uses, in this workspace or another, is refused. A declared rename carries its roles and passes. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01BjfMkJyKzodxkobqGZ6Lqb * feat(datatables): move data table role catalog and resolution to the enterprise edition Roles are an Enterprise Edition feature. The catalog, the Postgres logins, CONNECT convergence, tenant evaluation and the role half of connection resolution move to windmill-ee-private. Every public function keeps its path and signature and forwards through datatable_roles_oss, which re-exports the enterprise implementation or, without it, refuses. Without the enterprise edition a data table under roles, or a caller naming a role, is refused a connection rather than resolved as admin, and the reach and admin-access checks refuse one under roles. A data table not under roles resolves as before in every edition, and an instance database keeps the CONNECT grants it was created with. The catalog lock, the stream lock, the tenant cascades and the permissions stripping stay in OSS: they only restrict. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01BjfMkJyKzodxkobqGZ6Lqb * feat(datatables): move the data table permissions endpoints to the enterprise edition The permissions read, save and usable-roles handlers move to windmill-ee-private; the routes stay registered and, without the enterprise edition, answer that data table roles are an Enterprise Edition feature. ensure_governs_datatable and ensure_reaches_datatable keep their paths: the first refuses, the second passes a data table not under roles. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01BjfMkJyKzodxkobqGZ6Lqb * feat(datatables): move the data table role catalog endpoints to the enterprise edition The superadmin list, create, update and delete handlers move to windmill-ee-private. The routes stay registered and, without the enterprise edition, refuse after authentication. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01BjfMkJyKzodxkobqGZ6Lqb * test(datatables): run the roles tests on the enterprise edition, refusals without it Each test that exercises roles runs with private and enterprise. Two tests run without them: every roles route answers the Enterprise refusal, and a data table saved under roles, or a named role, is refused a connection while one not under roles resolves as before. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01BjfMkJyKzodxkobqGZ6Lqb * feat(datatables): gate the roles UI mount sites on an enterprise license Both mount sites are still commented out; the gate travels with them. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01BjfMkJyKzodxkobqGZ6Lqb * test(datatables): run the tenant matcher test on the enterprise edition The matcher it covers is enterprise code now, so without the enterprise edition the test hit the stub and failed the default windmill-common run. It runs with private and enterprise, and a counterpart without them asserts that no tenant list covers anyone, the wildcard and a workspace admin included. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01BjfMkJyKzodxkobqGZ6Lqb * chore: update ee-repo-ref to a1873dbb67f2302b85ff5362f8387b48eccdb607 This commit updates the EE repository reference after PR #783 was merged in windmill-ee-private. Previous ee-repo-ref: 5c853e2c20eca6b748415fc0d6862a6ebfb5fec4 New ee-repo-ref: a1873dbb67f2302b85ff5362f8387b48eccdb607 Automated by sync-ee-ref workflow. * fix(datatables): refuse roles while a same-workspace alias reaches the database Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix(datatables): let CE migrations connect as an explicitly named admin Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix(datatables): serialize roles going on with aliases saved from other workspaces Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * docs(datatables): note that legacy names with ? cannot be migrated Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix(datatables): drop a DuckDB data table secret once its ATTACH has used it Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * perf(datatables): resolve a workspace's data tables per pointer hop, not per entry Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix(datatables): hold the parent's settings while a fork points at its data tables Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * chore: update ee-repo-ref to 7e338e4dabf91689bfd7fb0333c6534040b17b59 This commit updates the EE repository reference after PR #787 was merged in windmill-ee-private. Previous ee-repo-ref: 38d6fcf2aeb39cfdac21814bbdbbcc02911e566a New ee-repo-ref: 7e338e4dabf91689bfd7fb0333c6534040b17b59 Automated by sync-ee-ref workflow. --------- Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com> Co-authored-by: windmill-internal-app[bot] <windmill-internal-app[bot]@users.noreply.github.com>	2026-09-18 13:56:41 +02:00
Ruben FiszelandClaude Opus 4.8	770ac2be9e	fix(self-host): resolve caddy-l4 "unrecognized global option: layer4" error (#10106) ... The self-hosted Caddy image relied on the abandoned RussellLuo/caddy-ext/layer4 shim to provide the `layer4` Caddyfile global option, alongside an old (May 2024) pin of mholt/caddy-l4 that predated native Caddyfile support. This combination is fragile: - If the image is ever built without the RussellLuo shim, the `layer4` global option disappears and Caddy fails with "unrecognized global option: layer4" — the reported bug. - Bumping mholt/caddy-l4 to any version with native Caddyfile support makes both modules register `layer4`, panicking at startup with "global option 'layer4' already registered". mholt/caddy-l4 now natively registers the `layer4` global option, so drop the RussellLuo dependency entirely and switch the Caddyfile to the native `route { proxy { upstream ... } }` syntax. The adapted layer4 JSON is byte-identical to the previous output, so runtime behavior is unchanged. Also bump the Caddy base image to 2.11.4 (required by current caddy-l4) and add a path-filtered push trigger so the published `:latest` image is rebuilt whenever the Caddy Dockerfile changes, instead of only on manual dispatch (which is how `:latest` drifted out of sync with the checked-in Caddyfile in the first place). Fixes GIT-903 Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>	2026-07-15 01:08:55 +02:00
Ruben Fiszel	3cd0eac8c1	deps: bump deno_core / deno_ast / swc to the goldilocks pin set; drop serde ceiling (#9111) ... * deps: bump deno_core / deno_ast / swc to the goldilocks pin set; drop serde ceiling Bumps every deno_* and swc_* workspace dep to a hand-picked "goldilocks" combination that drops the serde =1.0.220 ceiling without crashing into the rustls / aws-sdk resolver wall that the obvious deno v2.6.0 target hits. ## What's the goldilocks set | crate | old | new | source | |------------------|----------|-----------|---------------------------------------| | deno_core | 0.336.0 | 0.352.0 | deno v2.4.0 | | deno_fetch | 0.214.0 | 0.233.0 | deno v2.4.0 | | deno_tls | 0.177.0 | 0.196.0 | deno v2.4.0 (last permissive-rustls) | | deno_console | 0.190.0 | 0.209.0 | deno v2.4.0 | | deno_url | 0.190.0 | 0.209.0 | deno v2.4.0 | | deno_webidl | 0.190.0 | 0.209.0 | deno v2.4.0 | | deno_web | 0.221.0 | 0.240.0 | deno v2.4.0 | | deno_io | 0.100.0 | 0.119.0 | deno v2.4.0 | | deno_net | 0.182.0 | 0.201.0 | deno v2.4.0 | | deno_permissions | 0.49.0 | 0.68.0 | deno v2.4.0 | | deno_telemetry | 0.12.0 | 0.31.0 | deno v2.4.0 | | deno_error | =0.5.5 | =0.6.1 | deno v2.4.0 | | deno_ast | =0.44.0 | =0.51.0 | **override** — see "load-bearing" below | | deno_fs | (new) | 0.119.0 | new workspace dep — FetchPermissions exposes deno_fs::CheckedPath / GetPath as public API | | v8 | =130.0.7 | =137.1.0 | deno_core 0.352 transitive | | swc_common | =0.37.5 | =14.0.4 | **the load-bearing pin** | | swc_ecma_ast | =0.118.2 | =15.0.0 | matched set with swc_common 14.0.4 | | swc_ecma_parser | =0.149.1 | =24.0.3 | matched set | | swc_ecma_visit | =0.104.8 | =15.0.0 | matched set | | serde | =1.0.220 | ^1 | **freed** (resolves to 1.0.228+) | ## Why this combination and not v2.6.0 The obvious target was deno v2.6.0 (with deno_ast 0.52 → swc_common 17, well past the `__private` ceiling). That hits three resolver collisions: 1. libsqlite3-sys: deno_cache → rusqlite 0.37 → libsqlite3-sys 0.35 vs sqlx → libsqlite3-sys 0.30. **Already killed by PR #9110** — we dropped deno_runtime, which is what pulled in deno_cache. 2. fqdn 0.4.6/0.4.7 yanked, required by deno_permissions 0.81.0. Solvable by injecting the yanked entry into Cargo.lock manually but ugly. 3. rustls: deno_tls 0.198+ hard-pins `=0.23.28`, but aws-sdk-bedrockruntime 1.122.0 → aws-smithy-http-client 1.1.5 wants `^0.23.31`. Within-major conflict, no resolver path. The unbeatable wall. Goldilocks-set choice sidesteps (2) and (3) entirely: - `deno_tls 0.196.0` was the last version before deno tightened `rustls ^0.23.11` (range, accepts 0.23.31) to exact `=0.23.28`. With ^0.23.11, the resolver picks rustls 0.23.35 (latest 0.23 patch) which satisfies both deno_tls's `>=0.23.11` and aws-sdk's `>=0.23.31`. Verified empirically: lockfile has rustls 0.23.35 after this bump. - `deno_permissions 0.68.0` (v2.4.0's pin) doesn't depend on fqdn at all. The fqdn dep was added in a later deno_permissions release. ## Why deno_ast =0.51.0 specifically (not 0.48.0 from v2.4.0) `swc_common 14.0.4` is the first patch that **drops the `pub use serde::__private as serde;` line** in `src/private/mod.rs`. Older 14.0.x and all 0.37.5–13.x revisions still have it, and that line is what was capping `serde = "=1.0.220"` (the workspace pin's "stuck because of swc" comment). Empirically verified by inspecting the tarballs of 14.0.0 / 14.0.1 / 14.0.2 / 14.0.3 / 14.0.4: 14.0.0: has hack 14.0.1: has hack 14.0.2: has hack 14.0.3: has hack 14.0.4: NO HACK ← inflection point `deno_ast 0.51.0` pins `swc_common =14.0.4` exactly — older deno_ast versions pin earlier swc_common patches that still have the hack. Notably, deno v2.4.0 itself pins `deno_ast =0.48.0` (swc_common 9.2.0, still has hack) — we deliberately deviate from v2.4.0's deno_ast pin to escape the swc serde wall, while keeping the rest of v2.4.0's pin set for resolver compatibility with aws-sdk. deno_ast 0.51 was never shipped in any deno release (v2.4.5 used 0.49, v2.5.0 jumped to 0.50, v2.6.0 to 0.52), but it's published on crates.io and compatible with v2.4.0's deno_core 0.352. ## What this unblocks - PR #9106's `serde = "=1.0.224"` bump variant can rebase onto this and resolve cleanly (MaterializeInc/rust-postgres' `postgres-types` needs `serde_core ^1.0.221`, which is satisfied now that we're on serde 1.0.228). - Future deno_* / swc_* bumps no longer need to argue about the serde ceiling — it's gone. ## What changes in source code This commit is Cargo.toml + Cargo.lock only. Source changes that the new deno_core / deno_fetch API requires live in the follow-up commits: - `parsers/windmill-parser-{ts,ts-asset,wac}`: swc 0.37 → 14 (`code.into()` ambiguity fix at 5 sites) - `windmill-runtime-nativets/build.rs` + `src/lib.rs`: deno_core 0.336 → 0.352 API moves (`init_ops_and_esm()` → `init()`, `FetchPermissions` / `NetPermissions` trait signature updates, `deno_tls::Proxy` enum shape change) A companion change in windmill-ee-private adjusts `otel_tracing_proxy_ee.rs:521` for `deno_telemetry::init`'s second arg becoming by-value (was `&OtelConfig`). * fix(parsers): adapt to swc_common 14 BytesStr ambiguity swc 0.37.5 → 14.0.4 changed `SourceMap::new_source_file`'s `src` argument from `String` to `impl Into<BytesStr>`. With `BytesStr` available, the existing call sites' `code.into()` on a `&str` becomes ambiguous between `Into<Bytes>` (from the bytes crate) and `Into<BytesStr>` (from bytes_str). Switch to `code.to_string()` to produce an owned `String` that satisfies `From<String> for BytesStr` unambiguously. Five call sites across three crates: - windmill-parser-ts/src/lib.rs (3 sites) - windmill-parser-ts-asset/src/lib.rs (1 site) - windmill-parser-wac/src/typescript.rs (1 site) * fix(nativets): adapt to deno_core 0.352 / deno_fetch 0.233 API changes The goldilocks deno bump (deno_core 0.336 → 0.352, deno_fetch 0.214 → 0.233, etc.) ripples through nativets' build.rs and src/lib.rs. Source-level changes required: ## 1. `extension!` macro: `init_ops_and_esm()` and `init_ops()` removed deno_core 0.352's `extension!` macro now generates a single `init()` function on the extension struct (full: ops + esm), plus `lazy_init()` (ops only, with `needs_lazy_init = true` and a contract that the caller invokes `JsRuntime::lazy_init_extensions` after construction). - `build.rs` (snapshot creation, wants both ops and esm baked in): `X::init_ops_and_esm(...)` → `X::init(...)`. - `src/lib.rs:create_nativets_runtime` (runtime, was using `init_ops()` because the snapshot already provides esm): also → `X::init(...)`. deno_core's snapshot path skips esm re-execution when the snapshot provides them, so the esm registration is a no-op at runtime. This is how deno's own v2.4.0 runtime works. Avoided `lazy_init` because it requires plumbing `JsRuntime::lazy_init_extensions(ext_args_vec)` correctly across the codebase, which is invasive for no behavioural benefit. ## 2. Local `fetch` extension now declared in both build.rs and lib.rs deno_core 0.352 validates extension order between snapshot and runtime. Our snapshot's last extension is the local `fetch` ext (which provides ext:fetch/src/runtime.js). To avoid a runtime panic: "Extensions from snapshot loaded in wrong order: expected fetch but got windmill" …the runtime extension list now ends with `fetch::init()` matching the snapshot order. The macro requires the same `esm` argument to type-check, even though the ESM is not re-executed at runtime (it's in the snapshot). ## 3. `FetchPermissions` and `NetPermissions` trait shape `deno_fetch::FetchPermissions` (deno_fetch 0.233.0) added new methods and changed signatures: - `check_read` / `check_write`: now take `path: Cow<'a, Path>` plus a new `get_path: &'a dyn deno_fs::GetPath` parameter, and return `Result<deno_fs::CheckedPath<'a>, FsError>` instead of `Result<Cow<Path>, FsError>`. - New `check_write` (didn't exist) and `check_net_vsock` methods. `deno_net::NetPermissions` (deno_net 0.201.0) gained `check_vsock` and `check_write_path` now takes `Cow<'_, Path>`. For `build.rs`'s `PermissionsContainer` (used only during snapshot creation, where permissions are never actually checked): all methods `unreachable!("snapshotting")`. For `src/lib.rs`'s `PermissionsContainer` (used at runtime — the nativets policy is "allow everything"): `check_read` / `check_write` return `Ok(CheckedPath::Unresolved(path))`, `check_*_vsock` return `Ok(())`. Smoke tests confirm fetch/net/url/web/blob/timers/structuredClone behaviour is intact end-to-end. ## 4. `deno_tls::Proxy` is now an enum `deno_tls::Proxy` was a struct, is now an enum with `Http`, `Https`, `Socks5` variants. Our call site uses HTTP proxies — switched the struct literal `deno_tls::Proxy { url, basic_auth }` to `deno_tls::Proxy::Http { url, basic_auth }`. ## 5. New `deno_fs` direct workspace dep `FetchPermissions` exposes `deno_fs::CheckedPath` and `deno_fs::GetPath` as part of its public API. We can't avoid naming `deno_fs` directly any more. Pinned to 0.119.0 (v2.4.0's matched version, transitively present already through deno_fetch). Added to workspace `[dependencies]` plus nativets's `[dependencies]` and `[build-dependencies]`. ## Validation `cargo check --features enterprise,deno_core,duckdb,license,python,rust,scoped_cache,parquet,private,private_registry_test,csharp,php,ruby,mysql,quickjs,mcp,run_inline` → clean. `cargo test -p windmill-runtime-nativets smoke -- --ignored --skip smoke_net_` → 8 passed; 0 failed (the full local smoke suite covering fetch, setTimeout/Promise.all, URL/SearchParams, Blob/btoa/atob, large payload roundtrip, error propagation, concurrent isolates, TS enum/union transpile). Network smoke tests (`smoke_net_fetch_example_com`, `smoke_net_fetch_json_and_headers`) not run as part of the validation gate but expected to pass — the change preserves deno_fetch behaviour through the trait surface. * chore: update ee-repo-ref to pick up deno_telemetry::init by-value fix Points at windmill-ee-private branch deps/bump-deno-and-swc-goldilocks which contains the companion otel_tracing_proxy_ee.rs adjustment for deno_telemetry 0.12 → 0.31 (second arg of `init` is now by-value). EE-only file, doesn't affect OSS build. * chore(nix): bump rusty_v8 in flake.nix to 137.1.0 to match Cargo.toml Cargo.toml's v8 pin moved from =130.0.7 to =137.1.0 as part of the deno_core 0.336 → 0.352 bump, but I missed the comment directly above the version pin: # Exact version NOTE: Do not forget to update version and hash in flake.nix flake.nix provides the prebuilt librusty_v8 binary that the v8 crate links against. A version mismatch would either fail to fetch (if the 137.1.0 release didn't exist) or cause link-time symbol mismatches. Nix is used by rust-client-check.yml and rust_on_release.yml in CI, plus the dev shell — stale flake pin breaks all of those. Updates x86_64-linux's sha256 to match the actual hash of librusty_v8_release_x86_64-unknown-linux-gnu.a.gz at the 137.1.0 tag. Other targets (aarch64-linux, x86_64-darwin, aarch64-darwin) remain as lib.fakeHash — they were already placeholders in the previous pin, so we don't regress on them. Caught by both cubic and Pi reviewers on PR #9111. * docs(nativets): clarify snapshot-prefix rule in extension-order comment Claude reviewer caught that the doc comment claimed the runtime extension list matches the snapshot's order — implying an exact match. The truth is more permissive: deno_core 0.352 requires the snapshot's extension list to be a *prefix* of the runtime's, not an exact match. Runtime is allowed to append extra extensions (which we do — the windmill `ext` carrying our ops is the last entry at runtime but absent from the snapshot). The code is correct as-is; only the comment wording was misleading. Also fixes the same wording in PR description.	2026-05-11 21:13:24 +00:00
Diego Imbert	7f2486bdba	fix: Update duckdb to 1.5.2 (Ducklake 1.0.0) (#8848)	2026-04-16 06:23:57 -07:00
3 people	a46aa641f9	feat: add R language support (#8263) ... * feat: add R language support Add R as a new supported scripting language in Windmill, following the same pattern used for Ruby. Includes: - Backend: ScriptLang::Rlang enum variant, DB migration, tree-sitter-r parser crate with tests, WASM parser binding, R executor with NSJail sandboxing, job dispatch and signature parsing - Frontend: language picker, R icon, syntax highlighting, editor bar insertions (Sys.getenv, get_variable, get_resource), schema inference, init code template, BETA badge - CLI: .r extension mapping, sync support, bootstrap template R scripts use `main <- function(...)` syntax, jsonlite for JSON serialization, and system curl for the Windmill client helper. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * feat: add R package resolution and installation Parse library()/require() calls from R scripts to extract dependencies. Resolve versions from CRAN, cache lockfiles in pip_resolution_cache, and install packages to a shared R library cache. The run step sets R_LIBS_USER so installed packages are available to the script. - Parser: parse_r_requirements() extracts package names from AST - Executor: resolve() generates lockfile, install() installs from CRAN - Worker lockfiles: wire up R resolve for dependency jobs Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * feat: add nsjail sandboxing for R resolve and install phases Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * fix: fix R get_variable/get_resource and add sandbox annotation + e2e tests Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * fix: fix R arg inference with JS fallback parser and get_variable/get_resource Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * fix flake * nsjail * nits * fix: R install improvements - suppress verbose output, flat lockfile logging, Dockerfile R support, rlimits - Suppress renv verbose output during resolve and install (controlled by #verbose annotation) - Filter renv from install list (already loaded, causes noisy restart message) - Log compact "resolved N packages" instead of full renv.lock JSON - Add R (r-base, r-cran-renv) to DockerfileFull and DockerfileFullEe - Use disable_rl for nsjail install config (R compiles from source) - Reduce default concurrency from 20 to 5 - Add rlang to openflow.openapi.yaml - Fix MainArgSignature (no_main_func -> auto_kind) after main merge Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * final * fix: remove accidental R install from multiplayer Dockerfile Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * fix: remove R from Windows build and DockerfileExtra Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * fix: rename R migration to avoid timestamp collision with trigger_filter_logic Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * all * fix: R install improvements - suppress verbose output, flat lockfile logging, Dockerfile R support, rlimits Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * fix: add clear error when Rscript binary is missing Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * fix: fix type errors in R fallback parser, use format! in wrap(), add R system prompts Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com> Co-authored-by: pyranota <pyra@duck.com>	2026-04-01 06:11:37 +00:00
centdixandClaude Opus 4.6	020de59fcf	make flake dev env vars respect per-worktree overrides (#8374) ... Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>	2026-03-16 14:12:52 +00:00
centdixandClaude Opus 4.6	89f835727b	chore: use Nix profiles in sandbox Docker image (#8140) ... * feat: use Nix profiles in sandbox Docker image Replace manual tool installs (rustup, nodesource, curl installers) in sandbox-image/Dockerfile.sandbox with a single `nix profile install .#sandbox`. All tools (Rust, Node, Bun, Deno, Go, gh, sqlx-cli, cargo-watch, Chromium, Playwright, etc.) are now managed declaratively via flake.nix. - Add `packages.sandbox` and `packages.sandbox-full` buildEnv outputs to flake.nix - Add `sandbox-env` helper script for browser tooling env vars - Update playwrightWrapper to export PLAYWRIGHT_BROWSERS_PATH - Rewrite Dockerfile.sandbox: Nix replaces ~50 lines of manual installs - Update entrypoint.sh to source Nix profile PATH - Delete deprecated root Dockerfile.sandbox Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * fix: sandbox image runs as non-root user with wmdev - Rewrite entrypoint.sh to start PostgreSQL as current user (no chown/su needed), fixing "Operation not permitted" when wmdev runs containers with --user - Add chmod -R 777 /root and passwd entry for UID 1000 so non-root containers can access bashrc, nix-profile, and tool configs - Remove apt postgresql server (Nix profile provides it) - Fix bash history expansion errors from literal `!` in system prompt - Fix asciinema path reference (available on PATH, not hardcoded) Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * fix: wrap pkg-config in sandbox profiles to bake in Nix search path Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * fix: add openssh-client and sudo to sandbox image for full root access Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * fix: use useradd instead of manual passwd entry for sandbox agent user Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>	2026-03-01 17:42:33 +00:00
centdixandClaude Opus 4.6	7728475fc9	refactor: rewrite flake.nix for clarity and modularity (#8137) ... * refactor: rewrite flake.nix from scratch for clarity and modularity Rewrite the Nix flake with clean separation of concerns, organized let-bindings, and 4 purpose-specific devShells instead of a monolithic default shell with broken package outputs. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * feat: add CLI tools to default devShell (gh, aws, playwright, mermaid, asciinema) Add tools needed for AI agent workflows and dev tooling: - gh (GitHub CLI) - awscli2 - asciinema (terminal recording) - playwright-driver with Nix-managed browsers - mermaid-cli (diagram generation) Playwright browsers are provided via nixpkgs' playwright-driver.browsers. Mermaid/Puppeteer reuses the headless_shell from the same browser set. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * fix: move wm-minio scripts to default devShell MinIO (local S3) is needed for regular development, not just the full profile. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * fix: use playwright wrapper + chromium for browser tools Replace playwright-driver (library, no CLI) with: - A `playwright` wrapper script that calls the Nix playwright-core CLI (version-matched to its own Nix-provided browsers) - pkgs.chromium for Mermaid/Puppeteer (which respects PUPPETEER_EXECUTABLE_PATH) This fixes playwright screenshot and mermaid diagram generation. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * feat: auto-load .env.local from main worktree in all devShells Gitignored files like .env.local don't exist in git worktrees. Add a shared shellHook that resolves back to the main tree via git-common-dir and sources .env.local if present. This ensures AWS credentials and other secrets are available in worktrees. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * fix: replace deprecated pkgs.hostPlatform with stdenv.hostPlatform Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * chore: remove AWS CLI from flake and sandbox images Pastebin is sufficient for screenshot sharing; AWS credentials add unnecessary complexity. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * fix: address PR review — ruby mismatch, quoting, shell dedup - Fix pkgs.ruby → pkgs.ruby_3_4 in extraRuntimeVars to match extraRuntimes - Replace $* with "$@" in all helper scripts (wm, wm-build, wm-caddy, wm-bench, wm-cli) to correctly preserve argument boundaries - Extract coreBuildInputs, browserVars, and playwrightWrapper as shared let-bindings to eliminate duplication between default and full shells Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * chore: remove .env.local auto-loading from devShells Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>	2026-02-28 06:51:02 +00:00
centdix	238a573650	remove claude from flake (#7913)	2026-02-11 16:30:19 +00:00
wendrul	900c76ccad	fix: nix flake libz.so for deno_core (#7905)	2026-02-11 13:13:36 +00:00
wendrul	25255f9526	Fix nix flake (missing curl for compiling all features) (#7881)	2026-02-10 16:46:06 +00:00
centdixandClaude Opus 4.6	76377a00a6	internal: flake nix devshell clang/mold/openssl compatibility (#7855) ... * fix: flake nix devshell clang/mold/openssl compatibility - Add mold linker to buildInputs - Pin cargo linker to clang 18 (stdenv's clang 21 causes SIGSEGV with mold) - Embed OpenSSL rpath via rustflags instead of LD_LIBRARY_PATH to avoid leaking into git/ssh Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * more fixes * fix --------- Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>	2026-02-09 14:03:22 +00:00
Pyra	95df7b9a6a	feat: otel REST tracing (#7571)	2026-01-20 13:38:02 +00:00
Pyraandwindmill-internal-app[bot]	fe56191422	feat(internal): runnable settings (#7298) ... * rework everything again Signed-off-by: pyranota <pyra@duck.com> * updcate sqlx Signed-off-by: pyranota <pyra@duck.com> * update ref Signed-off-by: pyranota <pyra@duck.com> * fix things Signed-off-by: pyranota <pyra@duck.com> * fix function Signed-off-by: pyranota <pyra@duck.com> * final fixes Signed-off-by: pyranota <pyra@duck.com> * update sqlx Signed-off-by: pyranota <pyra@duck.com> * fix script creation Signed-off-by: pyranota <pyra@duck.com> * address todo Signed-off-by: pyranota <pyra@duck.com> * cleanup Signed-off-by: pyranota <pyra@duck.com> * remove dbg Signed-off-by: pyranota <pyra@duck.com> * cleanup Signed-off-by: pyranota <pyra@duck.com> * fix Signed-off-by: pyranota <pyra@duck.com> * fixups Signed-off-by: pyranota <pyra@duck.com> * fix cargo.toml Signed-off-by: pyranota <pyra@duck.com> * update ee repo Signed-off-by: pyranota <pyra@duck.com> * fix ci Signed-off-by: pyranota <pyra@duck.com> * nit Signed-off-by: pyranota <pyra@duck.com> * ref Signed-off-by: pyranota <pyra@duck.com> * fix Signed-off-by: pyranota <pyra@duck.com> * ee repo Signed-off-by: pyranota <pyra@duck.com> * sqlx Signed-off-by: pyranota <pyra@duck.com> * ee ref Signed-off-by: pyranota <pyra@duck.com> * remove dbg Signed-off-by: pyranota <pyra@duck.com> * sqlx Signed-off-by: pyranota <pyra@duck.com> * chore: update ee-repo-ref to 505eadbff32d102ea5245a2bef88ce6f1bb95395 This commit updates the EE repository reference after PR #348 was merged in windmill-ee-private. Previous ee-repo-ref: 195243e56cc0eab55f8890fa57297206bfe2c18c New ee-repo-ref: 505eadbff32d102ea5245a2bef88ce6f1bb95395 Automated by sync-ee-ref workflow. * ci: force runnable settings Signed-off-by: pyranota <pyra@duck.com> --------- Signed-off-by: pyranota <pyra@duck.com> Co-authored-by: windmill-internal-app[bot] <windmill-internal-app[bot]@users.noreply.github.com>	2025-12-16 21:17:06 +00:00
Pyra	3050d42683	chore(flake): add gsed (#7283) ... * chore(flake): add `gsed` Signed-off-by: pyranota <pyra@duck.com> * gsed -> gnused Signed-off-by: pyranota <pyra@duck.com> --------- Signed-off-by: pyranota <pyra@duck.com>	2025-12-03 13:48:25 +00:00
PyraandRuben Fiszel	d38c96db36	feat: workspace dependencies (#7124) ... * commit raw requirements Signed-off-by: pyranota <pyra@duck.com> * raw requirements Signed-off-by: pyranota <pyra@duck.com> * implement `parse_annotation` Signed-off-by: pyranota <pyra@duck.com> * more progress on wdeps Signed-off-by: pyranota <pyra@duck.com> * more progress Signed-off-by: pyranota <pyra@duck.com> * fixes Signed-off-by: pyranota <pyra@duck.com> * more progress Signed-off-by: pyranota <pyra@duck.com> * fixes Signed-off-by: pyranota <pyra@duck.com> * cli improvements + raw deps Signed-off-by: pyranota <pyra@duck.com> * cleanup Signed-off-by: pyranota <pyra@duck.com> * fix python versions Signed-off-by: pyranota <pyra@duck.com> * progress Signed-off-by: pyranota <pyra@duck.com> * update :) Signed-off-by: pyranota <pyra@duck.com> * add MaybeLock Signed-off-by: pyranota <pyra@duck.com> * go WIP Signed-off-by: pyranota <pyra@duck.com> * fix python ignoring py version from requirements Signed-off-by: pyranota <pyra@duck.com> * optimize php Signed-off-by: pyranota <pyra@duck.com> * require admin to alter Signed-off-by: pyranota <pyra@duck.com> * fix(cli): flow generateLocks raw deps Signed-off-by: pyranota <pyra@duck.com> * progress in checklist Signed-off-by: pyranota <pyra@duck.com> * fix agent workers Signed-off-by: pyranota <pyra@duck.com> * nits Signed-off-by: pyranota <pyra@duck.com> * nits Signed-off-by: pyranota <pyra@duck.com> * nit: remove default features Signed-off-by: pyranota <pyra@duck.com> * oh-wow Signed-off-by: pyranota <pyra@duck.com> * remove dbg! Signed-off-by: pyranota <pyra@duck.com> * nits Signed-off-by: pyranota <pyra@duck.com> * add indexes Signed-off-by: pyranota <pyra@duck.com> * cleanup Signed-off-by: pyranota <pyra@duck.com> * nits Signed-off-by: pyranota <pyra@duck.com> * remove todos Signed-off-by: pyranota <pyra@duck.com> * fix cli Signed-off-by: pyranota <pyra@duck.com> * add debug flag Signed-off-by: pyranota <pyra@duck.com> * cli: remove noise Signed-off-by: pyranota <pyra@duck.com> * fix cli Signed-off-by: pyranota <pyra@duck.com> * remove todos Signed-off-by: pyranota <pyra@duck.com> * trigger deps correctly Signed-off-by: pyranota <pyra@duck.com> * fix frontend Signed-off-by: pyranota <pyra@duck.com> * fix frontend again Signed-off-by: pyranota <pyra@duck.com> * finally fix frontend Signed-off-by: pyranota <pyra@duck.com> * ee repo ref Signed-off-by: pyranota <pyra@duck.com> * fix all Signed-off-by: pyranota <pyra@duck.com> * more fixes... Signed-off-by: pyranota <pyra@duck.com> * remove test Signed-off-by: pyranota <pyra@duck.com> * Update backend-test.yml * comment out legacy test Signed-off-by: pyranota <pyra@duck.com> * fix ci Signed-off-by: pyranota <pyra@duck.com> * fix ci? Signed-off-by: pyranota <pyra@duck.com> * comment out thing Signed-off-by: pyranota <pyra@duck.com> * ignore test Signed-off-by: pyranota <pyra@duck.com> * ci Signed-off-by: pyranota <pyra@duck.com> * base fixture Signed-off-by: pyranota <pyra@duck.com> * fix regression Signed-off-by: pyranota <pyra@duck.com> * fix docs links Signed-off-by: pyranota <pyra@duck.com> * update min version Signed-off-by: pyranota <pyra@duck.com> * simplify * implement cache for get_latest Signed-off-by: pyranota <pyra@duck.com> * move to workspace settings Signed-off-by: pyranota <pyra@duck.com> * sqlx + migration Signed-off-by: pyranota <pyra@duck.com> * more migrations Signed-off-by: pyranota <pyra@duck.com> * use box pin Signed-off-by: pyranota <pyra@duck.com> * nit Signed-off-by: pyranota <pyra@duck.com> --------- Signed-off-by: pyranota <pyra@duck.com> Co-authored-by: Ruben Fiszel <ruben@windmill.dev>	2025-11-28 16:00:11 +00:00
wendrulandRuben Fiszel	32fae7a10c	feat: ansible playbook execution git repo mode (repo viewer + UI utils) (#6831) ... * Improve minio on flake.nix * Add first asset parsing logic for ansible * Correct html gt sign * Decouple s3 file picker from drawer * Factor duplicate code into snippet * Update S3FilePickerInner to be compatible * Fix pane shrinking issue * Git repo viewer * Change GitRepoViewer * Endpoints for git repo visualizer * Move git repo viewer to its own component * Add button to populate git repo viewer * Update parser yaml for new ansisble features (repo viewer) * Reflect parser changes for ansible * Add button to add the git repo mode of declaration for ansible * Factor function * Playbook + inventories into the drawer * Add button to add inventories from s3 * Move tests to lib.rs * Inventory loading from s3 * Move get github app token logic to be reused by ansible * Update parser and ansible executor * Use the correct path for inventories * Add nushell to flake for wasm builds * Add published parser * Update hubPaths with clone and upload to s3 * Update ee-repo to the branch ref * Fix npm run check * Update cargo.lock * Change labels on buttons * Remove debug log * Update ee-repo-ref * Fix ee issues * Update ee-repo ref * Fix typo * Fix ee * Update ee-repo-ref * Fix missing imports * Unused var * Fix typo * Layout improvents * Fix typos * Remove unused function and log --------- Co-authored-by: Ruben Fiszel <ruben@windmill.dev>	2025-10-16 09:08:18 +00:00
centdix	7b9e2c2d68	internal: fix flake and cli dev usage (#6761) ... * fix flake * fix cli build * fix deno_ffi	2025-10-07 08:07:47 +00:00
3 people	3dadcbe865	feat: forkables workspaces v0 (#6479) ... * Add create_ephemeral workspace endpoint * Add cli devShell * List ephemeral workspaces + improve endpoint * Add postgres function to clone a workspace (to be revisited) * Clone workspace using the postgres function * Add first iteration of ephemeral workspaces command * Update display of forked workspaces * Remove SQLX_OFFLINE * Add UI to create ephemeral workspace * Add option to exclude repository from being inherited to forks * WIP: reworking cloning logic * Fix cloning * Fix redirect after creating fork * Clean up cloning behaviour * Rename ephemeral to fork * emove ephemeral_workspaces table in favour of columns in workspaces * Fix display of forked workspaces * Fix skip inherit git sync repo setting * Fix fork invite display + creating fork as user * Fix SideMenu bug * Fix alignment * Simplify migrations * Update deletion of workspaces * Delete forked workspace from cli * Deleting fork workspaces from the UI as non-admin * Update cli sync and fork creation to adapt to branches and forks * Update fork prefix * Remove skip tracking toggle * Fix npm check warnings * Fix last npm check * fix: force stdin to Stdio::null for all user code execution (#6575) Set stdin to Stdio::null for all Commands that execute user code across all supported languages to prevent unwanted input consumption. This affects Python, Deno, Bash, PowerShell, Go, Rust, PHP, Ruby, Java, C#, Ansible, Nu, and Bun executors. The dedicated worker handler was intentionally left unchanged as it requires stdin for inter-process communication. Co-authored-by: claude[bot] <209825114+claude[bot]@users.noreply.github.com> Co-authored-by: windmill-internal-app[bot] <windmill-internal-app[bot]@users.noreply.github.com> * Update ee-repo ref * Update SQLx metadata * Fix typos --------- Co-authored-by: claude[bot] <209825114+claude[bot]@users.noreply.github.com> Co-authored-by: windmill-internal-app[bot] <windmill-internal-app[bot]@users.noreply.github.com>	2025-09-10 17:42:45 +00:00
pyranota	6775191f09	fix(go): fix go client (#6561) ... * add jq to flake Signed-off-by: pyranota <pyra@duck.com> * make build.sh compatible with non-FHS complient systems Signed-off-by: pyranota <pyra@duck.com> * fix client Signed-off-by: pyranota <pyra@duck.com> --------- Signed-off-by: pyranota <pyra@duck.com>	2025-09-09 14:19:54 +00:00
centdix	2050872e58	fix typo in flake (#6559)	2025-09-09 12:45:51 +00:00
3 people	eaf4054bd3	feat: native k8s autoscaling integration (EE) (#6405) ... * stage progress * stage progress (dirty) * feat: native k8s autoscaling integration (EE) Signed-off-by: pyranota <pyra@duck.com> * fix frontend Signed-off-by: pyranota <pyra@duck.com> * update ee ref Signed-off-by: pyranota <pyra@duck.com> * update ee ref Signed-off-by: pyranota <pyra@duck.com> * Update backend/windmill-api/src/configs.rs Co-authored-by: ellipsis-dev[bot] <65095814+ellipsis-dev[bot]@users.noreply.github.com> * Update ee-repo-ref.txt --------- Signed-off-by: pyranota <pyra@duck.com> Co-authored-by: ellipsis-dev[bot] <65095814+ellipsis-dev[bot]@users.noreply.github.com> Co-authored-by: Ruben Fiszel <ruben@windmill.dev>	2025-08-19 02:20:14 +00:00
pyranotaandellipsis-dev[bot]	11dd4118ce	feat: add ruby support (#5939) ... * implement single line pin * make panic-safe * use pin even if multiple modules imported withing single statement * add repins and make imports respect pins * keep all pins * Allow multiple pins * add comments + handle stuff more safely * fix fully qualified imports * remove ignore * sort nested * apply unique to output requirements list * fix typo * remove mut * update sqlx * sort imports * sort imports * fix formatter and format * feat: add ruby support * fix * update sqlx * add ruby to flake.nix Signed-off-by: pyranota <pyra@duck.com> * upgrade par_install_language_dependencies helper rename fields + allow passing custom payload Signed-off-by: pyranota <pyra@duck.com> * change migrations Signed-off-by: pyranota <pyra@duck.com> * make full pipeline work! Signed-off-by: pyranota <pyra@duck.com> * do a bit fixing * stage Signed-off-by: pyranota <pyra@duck.com> * add ruby in dockerfiles Signed-off-by: pyranota <pyra@duck.com> * implement auto-require + 'windmill/inline' Signed-off-by: pyranota <pyra@duck.com> * create 'windmill/inline' only if does not exist Signed-off-by: pyranota <pyra@duck.com> * add nsjail Signed-off-by: pyranota <pyra@duck.com> * update global settings Signed-off-by: pyranota <pyra@duck.com> * add ruby icon * generate lockfile on deploy (but broken) Signed-off-by: pyranota <pyra@duck.com> * fix(frontend): display deployed script lockfile and other info Signed-off-by: pyranota <pyra@duck.com> * feat: add mini-wmill client Methods: - get_resource - get_variable Additionally add shortcuts on frontend Signed-off-by: pyranota <pyra@duck.com> * add nit warning Signed-off-by: pyranota <pyra@duck.com> * support private repos credentials Signed-off-by: pyranota <pyra@duck.com> * fix compilation error Signed-off-by: pyranota <pyra@duck.com> * found rustc bug + refactor universal installer Signed-off-by: pyranota <pyra@duck.com> * more refactor + fixes Signed-off-by: pyranota <pyra@duck.com> * bug fixing + polishing Signed-off-by: pyranota <pyra@duck.com> * cleanup Signed-off-by: pyranota <pyra@duck.com> * change 'windmill/inline' Signed-off-by: pyranota <pyra@duck.com> * delegate env_clear to universal installer Signed-off-by: pyranota <pyra@duck.com> * update init script Signed-off-by: pyranota <pyra@duck.com> * add cli support Signed-off-by: pyranota <pyra@duck.com> * unhardcode frontend Signed-off-by: pyranota <pyra@duck.com> * fix cli Signed-off-by: pyranota <pyra@duck.com> * fix init script Signed-off-by: pyranota <pyra@duck.com> * make it work on ms windows Signed-off-by: pyranota <pyra@duck.com> * do not clean env in the end Signed-off-by: pyranota <pyra@duck.com> * fix windows (again) Signed-off-by: pyranota <pyra@duck.com> * include PROXY_ENVS (untested) Signed-off-by: pyranota <pyra@duck.com> * feat: self-signed certs + proxy support (tested) Signed-off-by: pyranota <pyra@duck.com> * flake: pin ruby to 3.4 * merge follow-ups Signed-off-by: pyranota <pyra@duck.com> * remove obsolete libc crate Signed-off-by: pyranota <pyra@duck.com> * cleanup parser Signed-off-by: pyranota <pyra@duck.com> * fix java ci Signed-off-by: pyranota <pyra@duck.com> * cleanup Signed-off-by: pyranota <pyra@duck.com> * fix nits Signed-off-by: pyranota <pyra@duck.com> * Update backend/windmill-worker/src/universal_pkg_installer.rs Co-authored-by: ellipsis-dev[bot] <65095814+ellipsis-dev[bot]@users.noreply.github.com> * Update backend/windmill-worker/src/ruby_executor.rs Co-authored-by: ellipsis-dev[bot] <65095814+ellipsis-dev[bot]@users.noreply.github.com> * add nsjail for lock stage Signed-off-by: pyranota <pyra@duck.com> * fix merge Signed-off-by: pyranota <pyra@duck.com> * Update common.rs * regen windmill-api-client Signed-off-by: pyranota <pyra@duck.com> * fix tests Signed-off-by: pyranota <pyra@duck.com> * fix tests again Signed-off-by: pyranota <pyra@duck.com> * reference published ruby parser Signed-off-by: pyranota <pyra@duck.com> --------- Signed-off-by: pyranota <pyra@duck.com> Co-authored-by: ellipsis-dev[bot] <65095814+ellipsis-dev[bot]@users.noreply.github.com>	2025-08-15 13:03:32 +00:00
pyranota	83aa0d0267	fix(rust-sdk): revert openapi-generator-cli to 7.10.0 (#6325) ... In > 7.10.0 regression was introduced breaking our build process. This commit reverts the change and fixes tests Signed-off-by: pyranota <pyra@duck.com>	2025-08-05 11:08:16 +00:00
wendrul	205674a72a	Add dependencies to run sqlx prepare to nix flake (#6309)	2025-07-31 13:23:52 +00:00
3 people	8bc2a5733e	fixt: Add runScriptByPath and runScriptByHash methods to SDK clients (#6252) ... * feat: Add runScriptByPath and runScriptByHash methods to SDK clients - Add runScriptByPath and runScriptByHash methods to TypeScript client - Add run_script_by_path and run_script_by_hash methods to Python client - Split functionality from existing methods that took both path and hash parameters - Add deprecation warnings to existing run_script methods - Maintain backward compatibility while encouraging use of focused methods Closes #6251 Co-authored-by: Ruben Fiszel <rubenfiszel@users.noreply.github.com> * Refactor SDK methods to eliminate code duplication - Extract common logic into internal helper methods - Python client: _run_script_async_internal() and _run_script_internal() - TypeScript client: _runScriptAsyncInternal() and _runScriptInternal() - Eliminate duplicated parameter processing and HTTP setup - Maintain exact same public API surface and functionality - Reduce lines of code while preserving all existing behaviors Co-authored-by: Ruben Fiszel <rubenfiszel@users.noreply.github.com> * fix ts and dev.nu for python * trade warnings.warn for logging.warning Signed-off-by: pyranota <pyra@duck.com> --------- Signed-off-by: pyranota <pyra@duck.com> Co-authored-by: claude[bot] <209825114+claude[bot]@users.noreply.github.com> Co-authored-by: Ruben Fiszel <rubenfiszel@users.noreply.github.com> Co-authored-by: pyranota <pyra@duck.com>	2025-07-22 12:21:21 +00:00
pyranotaandellipsis-dev[bot]	71d6bbbdc3	feat(cli): make flow generate-locks respect raw requirements (#6105) ... * build: add claude-code to nix shell Signed-off-by: pyranota <pyra@duck.com> * stage progress * stage files * make it work * stage work Signed-off-by: pyranota <pyra@duck.com> * make it work + refactor logic a bit Signed-off-by: pyranota <pyra@duck.com> * small cleanup Signed-off-by: pyranota <pyra@duck.com> * rename `local_lockfiles` to `rawReqs` Signed-off-by: pyranota <pyra@duck.com> * cleanup Signed-off-by: pyranota <pyra@duck.com> * fix transpilation to deno error Signed-off-by: pyranota <pyra@duck.com> * update sqlx Signed-off-by: pyranota <pyra@duck.com> * cleanup features.default Signed-off-by: pyranota <pyra@duck.com> * follow up rename Signed-off-by: pyranota <pyra@duck.com> * nits Signed-off-by: pyranota <pyra@duck.com> * overwrite flow.yaml on `flow generate-locks` Signed-off-by: pyranota <pyra@duck.com> * nits Signed-off-by: pyranota <pyra@duck.com> * remove invalid code generated by claude Signed-off-by: pyranota <pyra@duck.com> * nits Signed-off-by: pyranota <pyra@duck.com> * fix typo Co-authored-by: ellipsis-dev[bot] <65095814+ellipsis-dev[bot]@users.noreply.github.com> * remove double quotes Signed-off-by: pyranota <pyra@duck.com> * use async to write file Signed-off-by: pyranota <pyra@duck.com> --------- Signed-off-by: pyranota <pyra@duck.com> Co-authored-by: ellipsis-dev[bot] <65095814+ellipsis-dev[bot]@users.noreply.github.com>	2025-07-17 14:01:28 +00:00
pyranota	626372954b	build: add claude-code to nix shell (#6101) ... Signed-off-by: pyranota <pyra@duck.com>	2025-07-01 14:11:30 +02:00
pyranota	74de2397ce	docs: add Nix development guide (#5962) ... * update readme * change formatting * link docker/dev.nu Signed-off-by: pyranota <pyra@duck.com> * move stuff a bit * remove unrelated docs * remove duplicates Signed-off-by: pyranota <pyra@duck.com> * add php to path Signed-off-by: pyranota <pyra@duck.com> --------- Signed-off-by: pyranota <pyra@duck.com>	2025-06-17 18:15:30 +02:00
pyranota	6623972ee9	build: correctly build WASM parsers with nix shell (#5943)	2025-06-14 09:03:45 +02:00
pyranota	697f660532	chore: fix python client build (#5856) ... * chore: fix python client build * nit	2025-06-03 00:05:12 +02:00
pyranotaandellipsis-dev[bot]	ed61d97700	feat(rust): shared build directory (#5610) ... * feat: rust incremental compilation * update dockerfiles * fix compilation error * add windows flags * init * polishing * update * return mount-point * Update backend/windmill-worker/src/rust_executor.rs Co-authored-by: ellipsis-dev[bot] <65095814+ellipsis-dev[bot]@users.noreply.github.com> * share worker dir if nsjail off * final refactor --------- Co-authored-by: ellipsis-dev[bot] <65095814+ellipsis-dev[bot]@users.noreply.github.com>	2025-05-29 22:11:35 +02:00
pyranota	75fa9e4730	chore: improve openapi.yaml (#5841) ... * fix schema * update openpi-deref.yaml * update openapi-deref.json * add openapi-generator-cli in flake.nix * add GH action * fix HubScriptKind * fix errors	2025-05-29 21:58:31 +02:00
pyranota	b0b3ab595a	build: make flake.nix build with all features (#5773)	2025-05-20 09:43:44 +02:00
3 people	4fd0561239	feat: critical alert if disk near full (#5549) ... * feat: critical alert if disk near full * update logic to cover edge-case * update logic * windows support * Update README.md Co-authored-by: ellipsis-dev[bot] <65095814+ellipsis-dev[bot]@users.noreply.github.com> * make it periodically * add extra safety * respect killpills * do not check every 3 secs * move to monitor.rs * rework * alter readme * extending functionality * fix worker_mode * fix compilation * fix typo * make use of AI suggestion * update ee-repo-ref * update ee ref * logs are CE * update ee repo ref * remove systemstat from worker crate * fix comp error + sync cargo.lock * more comptime fixing * Update ee-repo-ref.txt * fix compilation error --------- Co-authored-by: ellipsis-dev[bot] <65095814+ellipsis-dev[bot]@users.noreply.github.com> Co-authored-by: Ruben Fiszel <ruben@windmill.dev>	2025-05-05 08:54:48 +02:00
wendrulandRuben Fiszel	fdd1642ce1	feat: Ansible improvements (vault, roles and git repos) (#5655) ... * Ansible vault + roles * Clone arbitrary repos * Fix cloning logic after merge * Make function for cloning without history any commit * Cloning repos and lockfile on the commit * Improve error messages * Create lockfile for roles and collections * Simplify ansible ssh identity interface * Ansible vault password: pass just a variable instead of 2 step approach * Lock lockfiles for roles and collections * fix typo * Change git ssh identity section name * Rename variable * Update init script for ansible * Suppress error when no roles --------- Co-authored-by: Ruben Fiszel <ruben@windmill.dev>	2025-04-24 16:04:28 +02:00
pyranota	f3ed108e43	nit: replace KJQXZ with more meaningful notation (#5547) ... * nit: replace `KJQXZ` with more meaningful notation Originally this string is located in all places, where modification is needed in order to add new language support * relative -> related * revert shebang in substitue.sh * remove '}'	2025-04-02 21:03:04 +00:00
pyranota	59740c0478	feat: add java support (#5458) ... * feat: add nu (nushell) support * add worker tests * deactivate tables and non-any types below top-level full support will come in V1 for V0 it's better to keep things minimal and simple * add syntax highlighting used python's grammar, since nushell isn't supported by monaco nor svelte-highlights for V1 nu will get it`s own grammar * add logo * partially implement plugin support * change logo + ability to deploy + nsjail draft * static variables + get_resource + get_variable * lsp/dev.nu + initial nu lsp (not working yet) * make it work with nsjail * nullguard * Much more flexible signature parsing and better error-messages * add init script * rename nulsp to nu * install nu to dockerfile * fix merge * implement Default for MainArgSignature * stage NU_CACHE_DIR * improve dockerfiles * dev.nu for parser-wasm + flake.nix * update code for windows * add nushell to flake * upload Cargo.lock * make build.sh work on nixos * build wasm cli parsers * add docs to README_DEV.md * add helper script docker/dev.nu * improve docker/dev.nu * fix windows * commit frontend/package(lock).json * update cargo.lock * correctly update cargo.lock * remove lsp * update flake.nix to include svelte server and nushell * Revert base.sql to main * remove PLUGIN_USE_RE * make CARGO_PATH private * add nu to cli * Change flags to build wasm-nu-parser * remove flake.nix from parser-wasm * update wasm-build target * remove unused import * add cli support for nu * update github workflows * wasm-build 0.17 -> 0.19 * update build script * update cargo.lock * Fix typographical error * start working on java * do java boilerplate * implement parser for java * update Cargo.lock * update ENV_SETTINGS * use published nu parser * update package.lock * java is S3 + Caching enabled * install nsjail backup * commit v0 * fix nsjail * v0.1 * rewrite parser in tree-sitter * implement parser from scratch * polishing * change init script to match new parser * fix imports * fix cli build * fix cli build * refactor install phase * implement .valid.windmill atomic verification * implement java init functionality * remove quick-xml * fix windows not recognizing 'mvn' * create empty settings.xml if there is no config provided * clean up * change default settings.xml * change classpath format for windows * docs to helper * java copy bin cache instead of symlink * remove comments * merge * fix package.json * fix package.json 2 * minor fixing * migrate to Coursier * update misc * Http(s) Proxy + CA certs * remove unused .wasm * make requirements insensitive to spaces * update handle_child refs * rework save_cache for directories * fix s3 bug * compile .wasm for cli * remove uuid import * fix compilation * use reference * fix zero-dep failure * removing unsafe stuff * remove unneeded imports * revert: we still need winapi * remove nix store from nsjail * do not create cache_nomount * add java to dnt * remove duplicated dependency in init script * fix typos * fix CI * use published parser	2025-03-27 15:12:55 +01:00
pyranota	a3faea16e7	feat: add nu-lang support (#5217) ... * feat: add nu (nushell) support * add worker tests * deactivate tables and non-any types below top-level full support will come in V1 for V0 it's better to keep things minimal and simple * add syntax highlighting used python's grammar, since nushell isn't supported by monaco nor svelte-highlights for V1 nu will get it`s own grammar * add logo * partially implement plugin support * change logo + ability to deploy + nsjail draft * static variables + get_resource + get_variable * lsp/dev.nu + initial nu lsp (not working yet) * make it work with nsjail * nullguard * Much more flexible signature parsing and better error-messages * add init script * rename nulsp to nu * install nu to dockerfile * fix merge * implement Default for MainArgSignature * stage NU_CACHE_DIR * improve dockerfiles * dev.nu for parser-wasm + flake.nix * update code for windows * add nushell to flake * upload Cargo.lock * make build.sh work on nixos * build wasm cli parsers * add docs to README_DEV.md * add helper script docker/dev.nu * improve docker/dev.nu * fix windows * commit frontend/package(lock).json * update cargo.lock * correctly update cargo.lock * remove lsp * update flake.nix to include svelte server and nushell * Revert base.sql to main * remove PLUGIN_USE_RE * make CARGO_PATH private * add nu to cli * Change flags to build wasm-nu-parser * remove flake.nix from parser-wasm * update wasm-build target * remove unused import * add cli support for nu * update github workflows * wasm-build 0.17 -> 0.19 * update build script * update cargo.lock * Fix typographical error * update Cargo.lock * update ENV_SETTINGS * use published nu parser * update package.lock * rewrite parser in tree-sitter * implement parser from scratch * polishing * change init script to match new parser * fix imports * fix cli build * fix cli build * merge * update wasm * use MiniPulledJob * update cli * change cli wasm schema * change cli * update deno.json * make wasm modules load lazily * regenerate parsers * remove leftover * update cargo.lock * clean up dnt.ts * add docs to cli/test.nu * add schema validation option * add Nu to try_validate_schema * reference frontend to new parser version	2025-03-20 16:45:21 +01:00
Ruben Fiszel	39c0dd3736	fix(python): PYTHON_PATH overrides python from uv	2025-02-20 02:54:51 +01:00
wendrul	5df05d5f0f	nix: Update flake for C# and oracle + add flake for building parsers (#5167)	2025-01-29 18:06:53 +01:00
Lucas Abel	bc92cc0d15	build: update nix flake lock (#5083)	2025-01-17 15:18:16 +01:00
Lucas Abel	0ee7c9ea78	build: update nix flake (#4998) ... - update lock file - improve npm client package using `pkgs.buildNpmPackage` - improve windmill package	2025-01-03 16:27:20 +01:00
Lucas Abel	71d527c29e	nix: rust support (#4957)	2024-12-20 11:12:44 +01:00
Lucas Abel	343cd02ad5	nix: update flake (#4895)	2024-12-11 08:57:14 +01:00
Lucas Abel	9163060c90	nix: nit fixes + bench script (#4703)	2024-11-14 09:08:11 +01:00
Lucas Abel	9022d4e1c8	add nix documentation + flake cleanups (#4662)	2024-11-07 17:18:09 +01:00
Lucas Abel	f767d92ec7	add flake.nix and package windmill backend (#4660)	2024-11-07 12:23:19 +01:00