Commit Graph
1914 Commits
Author SHA1 Message Date
Ruben FiszelandClaude Opus 5.5 0250a90849 chore: bump tokio to 1.53 lts (#11663)
* chore: bump tokio to 1.51 lts

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* chore: move tokio pin to 1.53 lts

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-10 12:50:56 +02:00
Ruben FiszelandClaude Opus 5.5 93e81ec189 [ee] chore: upgrade native runtime to current deno crates and v8 150 (#11657)
* chore: upgrade the native runtime to the current deno crates and v8 150

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* fix: keep data and blob fetches working under no_network

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* fix: pin the simdutf v8 archive and keep telemetry on http exporters

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* fix: force the http otlp exporter for native telemetry and pin v8

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-09 19:54:57 +00:00
Ruben Fiszelandrubenfiszel 0a19d26d5f chore(main): release 1.830.0 (#11650)
* chore(main): release 1.830.0

* Apply automatic changes

---------

Co-authored-by: rubenfiszel <275584+rubenfiszel@users.noreply.github.com>
2026-10-09 19:12:12 +00:00
Ruben Fiszelandrubenfiszel 92af345d05 chore(main): release 1.829.0 (#11621)
* chore(main): release 1.829.0

* Apply automatic changes

---------

Co-authored-by: rubenfiszel <275584+rubenfiszel@users.noreply.github.com>
2026-10-09 16:35:04 +02:00
Ruben Fiszelandrubenfiszel d505916609 chore(main): release 1.828.0 (#11611)
* chore(main): release 1.828.0

* Apply automatic changes

---------

Co-authored-by: rubenfiszel <275584+rubenfiszel@users.noreply.github.com>
2026-10-08 09:44:54 +02:00
Ruben Fiszelandrubenfiszel f40eea2a5f chore(main): release 1.827.0 (#11589)
* chore(main): release 1.827.0

* Apply automatic changes

---------

Co-authored-by: rubenfiszel <275584+rubenfiszel@users.noreply.github.com>
2026-10-07 21:52:12 +02:00
Ruben Fiszelandrubenfiszel f6fbf75916 chore(main): release 1.826.0 (#11565)
* chore(main): release 1.826.0

* Apply automatic changes

---------

Co-authored-by: rubenfiszel <275584+rubenfiszel@users.noreply.github.com>
2026-10-07 12:28:33 +02:00
Ruben Fiszelandrubenfiszel cc37c45c3d chore(main): release 1.825.0 (#11548)
* chore(main): release 1.825.0

* Apply automatic changes

---------

Co-authored-by: rubenfiszel <275584+rubenfiszel@users.noreply.github.com>
2026-10-06 14:56:07 +02:00
Ruben Fiszelandrubenfiszel 0abc0ac0c4 chore(main): release 1.824.1 (#11543)
* chore(main): release 1.824.1

* Apply automatic changes

---------

Co-authored-by: rubenfiszel <275584+rubenfiszel@users.noreply.github.com>
2026-10-06 00:24:43 +02:00
Ruben Fiszelandrubenfiszel af42c471ef chore(main): release 1.824.0 (#11520)
* chore(main): release 1.824.0

* Apply automatic changes

---------

Co-authored-by: rubenfiszel <275584+rubenfiszel@users.noreply.github.com>
2026-10-05 15:27:00 +00:00
Ruben Fiszelandrubenfiszel 4096c3da74 chore(main): release 1.823.1 (#11516)
* chore(main): release 1.823.1

* Apply automatic changes

---------

Co-authored-by: rubenfiszel <275584+rubenfiszel@users.noreply.github.com>
2026-10-05 08:24:51 +02:00
Ruben Fiszelandrubenfiszel 14d2995d76 chore(main): release 1.823.0 (#11503)
* chore(main): release 1.823.0

* Apply automatic changes

---------

Co-authored-by: rubenfiszel <275584+rubenfiszel@users.noreply.github.com>
2026-10-04 14:11:48 +02:00
Ruben Fiszelandrubenfiszel 3c30c82fa2 chore(main): release 1.822.0 (#11480)
* chore(main): release 1.822.0

* Apply automatic changes

---------

Co-authored-by: rubenfiszel <275584+rubenfiszel@users.noreply.github.com>
2026-10-03 09:42:45 +02:00
Ruben FiszelandClaude Opus 5.5 b7f74cd562 fix: never reuse a pg connection a script left inside a transaction (#11497)
* fix: never reuse a pg connection a script left inside a transaction

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01KHjj2iG8kKPCaFuK6bqXvi

* fix: close a pg connection whose job failed instead of caching it

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01KHjj2iG8kKPCaFuK6bqXvi

* docs: describe every caller of PgConnectionLease::discard

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01KHjj2iG8kKPCaFuK6bqXvi

---------

Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-03 09:03:44 +02:00
4beb1f9420 feat: add trigger, tag, run-as and digest claims to job OIDC tokens (#11481)
* feat: add trigger, tag, run-as and digest claims to job OIDC tokens

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

* fix: parse digest floats exactly and derive the codebase digest like push

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

* fix: give flows that reference flow nodes no digest and keep --json clean

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

* fix: list flow step digests from module positions only

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

* test: skip the pulled module-script digest check on windows

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

* chore: update ee-repo-ref to 71b8c1042fd8188c2d2882476f12b671cb5ba421

This commit updates the EE repository reference after PR #840 was merged in windmill-ee-private.

Previous ee-repo-ref: ba1870536789a43c5b6ec18522a93edb9cb07f3d

New ee-repo-ref: 71b8c1042fd8188c2d2882476f12b671cb5ba421

Automated by sync-ee-ref workflow.

---------

Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Co-authored-by: windmill-internal-app[bot] <windmill-internal-app[bot]@users.noreply.github.com>
2026-10-02 19:02:17 +02:00
Ruben FiszelandClaude Opus 5.5 14801fdd68 fix: unstick postgres jobs on large results with custom-typed columns (#11475)
* fix: unstick postgres jobs on large results with custom-typed columns

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

* fix: carry the parked-rows delivery fix in the postgres fork

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

* fix: describe postgres queries before streaming instead of parking rows

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

* perf: skip the postgres describe for statements that return no rows

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

* fix: look for returning in the whole postgres statement

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

* fix: read the leading postgres keyword past nested block comments

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

* fix: end leading postgres line comments at CR too

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-02 17:21:20 +02:00
Ruben Fiszelandrubenfiszel fee401f01e chore(main): release 1.821.0 (#11432)
* chore(main): release 1.821.0

* Apply automatic changes

---------

Co-authored-by: rubenfiszel <275584+rubenfiszel@users.noreply.github.com>
2026-10-01 18:35:57 +02:00
Ruben FiszelandClaude Opus 5.5 9028432e8a fix: embed a VERSIONINFO resource in the Windows binary (#11433)
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-09-30 09:20:58 +02:00
Ruben Fiszelandrubenfiszel f890355211 chore(main): release 1.820.0 (#11404)
* chore(main): release 1.820.0

* Apply automatic changes

---------

Co-authored-by: rubenfiszel <275584+rubenfiszel@users.noreply.github.com>
2026-09-29 20:26:55 +02:00
Ruben FiszelandClaude Opus 5.5 054109d855 fix: create workspace forks in the background, with progress (#11406)
* fix: create a fork in the background so a proxy timeout cannot cut it

create_fork copies the whole workspace inside the request, which can run
past the route timeout of an ingress in front of Windmill (Envoy's 15s
default), and the UI then reports a failure for a fork still being made.

create_fork?background=true now returns once the request is validated and
records the copy in workspace_fork_creation, which the new
fork_creation_status endpoint reads. The wizard and AI-session forks use it.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

* fix: drain background forks on shutdown and fork in the background from the CLI

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

* fix: settle fork polls by the fork's existence, adopt in-flight creations

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

* feat: show which part of the copy a fork being created is in

The background copy reports its phase (data tables, settings, resources,
scripts, flows, apps, drafts, triggers) through a watch channel. The heartbeat
task records it on the fork's creation row as soon as it changes, and
fork_creation_status returns it. The fork wizard shows it on its button, and
the CLI logs each step.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

* fix: join a retried fork creation server-side, settle status by the fork's existence

A retry from the same user and parent joins the creation in flight instead
of being refused, so clients no longer match the refusal's wording, and
another requester can never adopt it. The status route reports a fork that
exists under its parent as completed, whatever its run's record says. Clients
give up on failing polls after a time window rather than a count, which a
rolling deploy can exhaust.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

* fix: drop fork-creation joins and existence probes

A second request for a fork being created is refused again; only the
AI-session fork, whose request never varies, waits for its own earlier one.
Clients recognise a server without background forks by its synchronous
answer instead of probing for a workspace by id, which could name another
parent's fork.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

* fix: poll a background fork by the id of its own attempt

create_fork?background=true answers with a creation id, and the status
route reads that attempt only, for the user who started it. A retry that
reuses the fork id is a new attempt, so a poller never reads another
attempt's outcome. The AI-session fork no longer adopts a creation in flight,
which it could not tell apart from someone else's.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

* fix: record a fork's completion in its own commit, resume a session's fork after reload

The attempt is marked complete in the transaction that creates the fork, so
the status never infers completion from a workspace that may belong to
another request. An AI session keeps the creation id on its pending fork and,
after a reload mid-copy, waits for that attempt instead of requesting the
fork again.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-09-29 15:26:41 +02:00
8741843d2e feat: external instance cluster for data tables and Ducklake catalogs (#11197)
* fix pg_dump stuck on version 17 on nix

* fix(datatables): refuse a malformed role annotation instead of ignoring it

`-- Role operator`, `-- role operator;` and `-- role operator -- why` all failed
the annotation parser's exact-match rule, so the query fell through to the data
table's default role and ran, silently, under a login the author did not choose.
Naming a role exists precisely to not do that.

A leading comment whose first word is `role` is now an annotation attempt: the
keyword matches case-insensitively, one trailing `;` is tolerated, and anything
else is an error naming the line. Only callers that already know the target is a
`datatable://` reference ever run this, so ordinary SQL keeps its comments.

Also bumps the dev shell's postgres client to 18 — it trailed the server the dev
database runs, which takes out every data table export, clone and fork-with-data.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012ti5HyeTikPMYyW8YSdiHR

* fix(datatables): refuse a malformed role query string instead of ignoring it

`?Role=analytics`, `?role=` and `?x=1&role=…` all fell through the reference
parser's exact-match rule, so the connection resolved to the data table's default
role and ran under a login the caller never asked for — the URI half of the same
trap as a malformed `-- role` annotation.

The key now matches case-insensitively, and anything else in the query string is
an error naming it; `role` is the only parameter a reference takes. Callers that
only need the entry keep a lenient `datatable_ref_name`, since they never act on
the role. The DuckDB `ATTACH` parser propagates it rather than attaching under
the default.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012ti5HyeTikPMYyW8YSdiHR

* fix(datatables): carry the role annotation into the row_to_json retry

The retry rebuilds its SQL from `pruneComments(code)`, so the leading comment
block never reached the second attempt — and with it the `-- role <name>` line
that decides which login the query runs as. The retry connected as the data
table's default role instead, so a query the first attempt was denied could
succeed on the second, reported as "recovered with the row_to_json fix".

Carry the leading comment block over. The retry itself is unchanged.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BjfMkJyKzodxkobqGZ6Lqb

* chore(datatables): don't mount the roles UI until the ACL editor lands

Enforcement ships first. The permissions drawer is what turns roles on, and the
catalog section is what creates them — both are only useful once there is a way
to grant a role the privileges it needs, which arrives with the ACL editor. Left
mounted they would offer a feature whose other half does not exist.

The two components are complete and reviewed; only their call sites here are
commented out, with a note pointing the follow-up PRs at them.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BjfMkJyKzodxkobqGZ6Lqb

* fix(datatables): honour `-- role: x`, and fix the DuckDB attach test

Two review findings, both real.

`attach_datatable_parses_name_and_role` never compiled: `parse_attach_datatable`
returns `Result<Option<_>>` now and one call site kept a single `unwrap`. Its
`?Role=analytics` case also asserted a refusal, contradicting the parser in the
same commit, which matches the key case-insensitively. Replaced with the cases
that are genuinely malformed, and a positive one for the cased key.

`-- role: analytics` fell through to the default role — the silent fallback the
strict parser exists to remove, for the spelling most likely to be typed. The
keyword now accepts an optional colon, attached or spaced, while a word that
merely starts with it (`rolebased`) is still not an attempt.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BjfMkJyKzodxkobqGZ6Lqb

* fix(datatables): clone a fork's pointer instead of failing after the copy

Forking a fork with cloning left an orphan database. The preflight resolves the
pointer and sees the governing entry, so both endpoints ran and filled the new
database; `apply_forked_datatable` then refused the inherited pointer and rolled
the fork back, stranding a registered `wm_fork_*` that no entry names and whose
name blocks the retry.

Refusing earlier would have been the smaller change, but forking a fork and
cloning worked before pointers existed, so it would trade an orphan for a
regression. Resolve what the pointer names and write the terminal entry the
clone needs: the whole `database` object rather than a patch of its
`resource_path`, since a pointer has none, and `reference` removed with it.

Also accepts `-- role=x` and `-- Role = x`, two more spellings that fell through
to the default role.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BjfMkJyKzodxkobqGZ6Lqb

* fix(datatables): refuse to roll back the catalog while roles exist

The down migration dropped the table and left every role behind: live Postgres
logins whose passwords only that table carried, so after a revert Windmill could
neither use, disable nor delete them, and re-applying could not recreate them
because the names were taken. Cleaning up here is not possible either — dropping
a role means reassigning what it owns in every instance database, and a
migration runs in one — so it now refuses while the catalog is non-empty and
says to delete the roles through instance settings, which does the cluster work.

Also enforces the instance-only invariant the resolved-pointer clone relies on
rather than only asserting it in a comment.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BjfMkJyKzodxkobqGZ6Lqb

* refactor(datatables): settle clonability in one place, before anything is created

A clone is three stages a workspace apart — `create_pg_database`, then
`import_pg_database`, then `apply_forked_datatable` inside the fork transaction.
Only the third can roll back, and `CREATE DATABASE` is not transactional, so any
refusal that lives there strands a registered `wm_fork_*` that no entry names
and whose name blocks the retry.

That orphan has now been fixed three times, most recently reintroduced by a
guard added one commit ago. Patching each new refusal into the first endpoint is
not the fix; having two places that can refuse is. `ensure_datatable_is_clonable`
now answers every reason a copy can be refused and returns what it resolved, and
the stage that writes the entry only does the work.

Also takes an ACCESS EXCLUSIVE lock before the rollback guard counts, so a role
created concurrently cannot slip between the check and the drop.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BjfMkJyKzodxkobqGZ6Lqb

* fix(datatables): let a retried clone reclaim its own leftover database

A clone creates its target database one request before it copies into it, and
the fork that would name it is written a request after that. Any failure in
between — a pg_dump error, a bad restore, a dropped connection, the source's
roles changing mid-flow — left a registered `wm_fork_*` that no entry names,
and every retry then failed on its name. This predates data table roles.

`create_pg_database` now reclaims such a leftover before creating: only a
`wm_fork_*` database Windmill registered as a data table database and that no
data table or ducklake entry names, in any workspace, archived ones included.
The drop never terminates connections, so a clone still copying into it makes
the reclaim fail instead of being cut off. It is limited to callers who
administer the source — reaching it is not enough, since on a data table
without roles every member reaches it — and anyone else gets the refusal an
existing database always got.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* Revert "fix(datatables): let a retried clone reclaim its own leftover database"

This reverts commit 7dd3275a10.

The reclaim tied the caller to the source they administer, but not to the
database it dropped. Between another workspace's import and its final fork
request, that workspace's target is full, registered, unnamed and has no open
connection, so an admin of any instance data table could name it and have it
dropped and recreated empty. The victim's fork would then commit pointing at
the empty copy. Safe reclaim needs durable clone ownership and serialization
with the request that names the database; until then the leftover stays, as it
did before this PR.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* docs(datatables): record the stale clone database as a known limitation

A clone is three requests and `CREATE DATABASE` is not transactional, so a
failure after the first leaves a registered `wm_fork_*` behind, as it did
before data table roles. Accepted for this PR: it is harmless to data and goes
away once the clone is a single server-side operation.

The comment also records why the obvious fix is wrong: reclaiming the leftover
on retry, without durable clone ownership, can drop another workspace's fully
copied database between its import and its final fork request.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* fix(datatables): bounce the streams reading a data table when it is deleted

Deleting a governing data table, or the workspace that holds it, only collected
the fork pointers it stranded, for the warning. A Postgres trigger or capture
already streaming through one of those pointers kept the replication connection
it opened while the pointer still resolved, so it went on dispatching the
governing database's rows after the fork lost access — until its connection
happened to restart. The governing workspace's own streams on a deleted entry
did the same.

Both deletion paths now bounce the affected listeners inside their own
transaction, through the helper a permission change already uses, so a
listener that reconnects re-resolves the entry and finds it gone. The helper is
split so a caller can pass the (workspace, local name) pairs it already holds.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* fix(datatables): keep the fork schema baseline, and bounce streams on every removal

Three fixes from review.

`edit_datatable_config` took `forked_from` wholesale from the stored entry, so
the fork schema diff's save of an advanced baseline was silently discarded and
an applied change was offered again. Whether an entry carries a clone stamp is
still carried from the store, since that is what marks its database droppable,
but the baseline inside it is now taken from the request.

The stranded-pointer warning and the stream bounce ran over the optional
`deleted_datatables` hint, which the settings-sync CLI never sends, so removing
a governing data table through `wmill` bounced nothing. Removals are now derived
from the stored configuration against the saved one.

`delete_workspace` read the pointers to bounce before its transaction, so a fork
committing a pointer during the deletion was missed. The read now happens inside
the transaction, after the workspace row is deleted: a fork's insert key-share
locks that row through its parent foreign key, so it is either seen or fails on
the missing parent.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* refactor(datatables): keep Postgres triggers and data table roles apart

A replication stream reads every row of every table whatever the data table's
roles grant, and its listener checks access only when it connects. Rather than
chase every way access can change and bounce the streams each one affects, a
data table now carries one or the other:

- a Postgres trigger or capture cannot be created on, or connect to, a data
  table under roles;
- roles cannot be turned on while an enabled trigger or a live capture reads
  the data table, its own or a fork's through its pointer. The refusal names
  each one to disable.

This removes the stream bounces on roles edits and on data table and workspace
deletion, and the trigger gate that admitted admins. The fork schema baseline
fix from the same review round is kept.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* fix(datatables): refuse a Postgres trigger on a data table under roles when it is saved

Creating or editing a trigger that points at a data table under roles was
accepted, and its listener then retried the refused connection every 30
seconds forever. The save is now refused, and a trigger that reaches such a
data table anyway (re-enabled, or cloned into a fork) is disabled by its
listener with the reason, as a missing replication slot is.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* fix(datatables): disable a data table role before deleting it

Deleting a role reassigns and drops what it owns in each registered
database on its own connection, and each of those passes commits as it
goes. A database failing part-way left the role enabled in the catalog and
able to log in, but already stripped in the databases reached before it.
The role is now disabled in its own commit first, so a failed delete
leaves a disabled role to retry.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* fix(datatables): serialize roles going on with a stream starting

Turning roles on looked for enabled triggers and live captures once,
without a lock anything starting a stream also took. A trigger enabled in
that window could have its listener connect before roles committed, and a
healthy listener never checks again. Both transitions now serialize on one
advisory lock: roles going on hold it exclusive while they look, and
trigger create, edit and enable, and capture setup and ping hold it shared
while they commit. Either the look sees the stream, or the listener
connects after roles are committed and refuses.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* fix(datatables): wait out live listeners, and resolve stored names containing `?`

Turning roles on counted a trigger as gone once disabled, and a capture
once its client stopped pinging, but the listener keeps its replication
connection until its next heartbeat notices. A trigger or capture whose
listener pinged in the last 15 seconds, the window a server holds a
listener for, now still counts as streaming.

Data table names could contain `?` before they were restricted, and such
entries are still stored. Splitting `?role=` off a reference misread them:
`a?b` became `a` with an unknown parameter, and the clone checks looked at
a different entry than the one copied. An entry stored under the whole
reference is now looked up first, in the Postgres executor, DuckDB ATTACH
and the clone checks. Agent workers cannot read the workspace and keep
the strict parse, which refuses such a name rather than misreading it.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* fix(datatables): warn when a settings sync strands fork pointers

A settings save reported the fork pointers left resolving to nothing only
for the names in `deleted_datatables`, which `wmill sync push` never sends.
The save now works out what it removed from the locked entries, and the
CLI prints the stranded pointers it returns.

Also correct the replication helper's contract: no role or admin check
makes a replication connection safe, so a data table under roles is
refused outright rather than gated as an admin operation.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BjfMkJyKzodxkobqGZ6Lqb

* fix(datatables): refuse a save that drops a data table's roles through an undeclared rename

A data table's roles follow its entry only through a declared rename. A
settings sync sends the whole map and never declares one, so renaming a
data table under roles there read as a delete and a new entry on the same
database: the new entry carried no roles, and every caller connected as
admin. Such a save is now refused, naming both entries.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BjfMkJyKzodxkobqGZ6Lqb

* fix(datatables): no entry without roles may newly reach a database under roles

The previous guard only caught a new name replacing an entry under roles.
A whole-map save could also repoint an existing entry without roles at
that database, or another workspace could point one there, and every
caller of that entry would connect as admin. The rule is now stated on
the saved entries: one that carries no roles and newly points at an
instance database any entry under roles uses, in this workspace or
another, is refused. A declared rename carries its roles and passes.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BjfMkJyKzodxkobqGZ6Lqb

* feat(datatables): move data table role catalog and resolution to the enterprise edition

Roles are an Enterprise Edition feature. The catalog, the Postgres logins,
CONNECT convergence, tenant evaluation and the role half of connection
resolution move to windmill-ee-private. Every public function keeps its path
and signature and forwards through datatable_roles_oss, which re-exports the
enterprise implementation or, without it, refuses.

Without the enterprise edition a data table under roles, or a caller naming a
role, is refused a connection rather than resolved as admin, and the reach and
admin-access checks refuse one under roles. A data table not under roles
resolves as before in every edition, and an instance database keeps the
CONNECT grants it was created with. The catalog lock, the stream lock, the
tenant cascades and the permissions stripping stay in OSS: they only restrict.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BjfMkJyKzodxkobqGZ6Lqb

* feat(datatables): move the data table permissions endpoints to the enterprise edition

The permissions read, save and usable-roles handlers move to
windmill-ee-private; the routes stay registered and, without the enterprise
edition, answer that data table roles are an Enterprise Edition feature.
ensure_governs_datatable and ensure_reaches_datatable keep their paths: the
first refuses, the second passes a data table not under roles.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BjfMkJyKzodxkobqGZ6Lqb

* feat(datatables): move the data table role catalog endpoints to the enterprise edition

The superadmin list, create, update and delete handlers move to
windmill-ee-private. The routes stay registered and, without the enterprise
edition, refuse after authentication.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BjfMkJyKzodxkobqGZ6Lqb

* test(datatables): run the roles tests on the enterprise edition, refusals without it

Each test that exercises roles runs with private and enterprise. Two tests run
without them: every roles route answers the Enterprise refusal, and a data
table saved under roles, or a named role, is refused a connection while one
not under roles resolves as before.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BjfMkJyKzodxkobqGZ6Lqb

* feat(datatables): gate the roles UI mount sites on an enterprise license

Both mount sites are still commented out; the gate travels with them.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BjfMkJyKzodxkobqGZ6Lqb

* test(datatables): run the tenant matcher test on the enterprise edition

The matcher it covers is enterprise code now, so without the enterprise
edition the test hit the stub and failed the default windmill-common run. It
runs with private and enterprise, and a counterpart without them asserts that
no tenant list covers anyone, the wildcard and a workspace admin included.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BjfMkJyKzodxkobqGZ6Lqb

* chore: update ee-repo-ref to a1873dbb67f2302b85ff5362f8387b48eccdb607

This commit updates the EE repository reference after PR #783 was merged in windmill-ee-private.

Previous ee-repo-ref: 5c853e2c20eca6b748415fc0d6862a6ebfb5fec4

New ee-repo-ref: a1873dbb67f2302b85ff5362f8387b48eccdb607

Automated by sync-ee-ref workflow.

* fix(datatables): refuse roles while a same-workspace alias reaches the database

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* feat(datatables): add an ACL editor for data table roles

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* feat(datatables): data table roles in the DB manager and raw apps

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* fix: never add a role to the reference of a data table whose name contains '?'

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* fix: read the roles of a data table whose name contains '?'

The generated client leaves a '?' in a path param unencoded, so the lookup
404'd and the raw-app picker blocked Start on such a data table.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* fix: take every pooled connection before the ACL apply locks

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BRoYE5ZeAVvrDYdfhDAYXb

* fix: refresh grant options only after the ACL apply validates its plan

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BRoYE5ZeAVvrDYdfhDAYXb

* fix(datatables): refuse a reference naming both a legacy data table and a role

When a workspace stores both `sales` and a legacy `sales?role=analytics`, the
reference resolved to the legacy entry without a role, so browsing `sales` as
`analytics` reached another data table.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* fix: declare the default role in migrations written for a data table whose name contains '?'

Such a data table connects as its default role without naming it, so the
migrations the manager wrote for it declared no role and ran as admin.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* fix(datatables): let CE migrations connect as an explicitly named admin

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* fix: add only missing grant options before an ACL apply, never default privileges

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BRoYE5ZeAVvrDYdfhDAYXb

* fix(datatables): serialize roles going on with aliases saved from other workspaces

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* docs(datatables): note that legacy names with ? cannot be migrated

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* fix: run one data table ACL apply at a time per server before it connects

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BRoYE5ZeAVvrDYdfhDAYXb

* feat(datatables): set up an external instance cluster for data tables

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* fix(datatables): send external cluster passwords as SCRAM verifiers

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* fix(datatables): scope external cluster credential readers to the crate

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* [ee] feat(datatables): external_instance data tables on the external cluster

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* fix: hold the ACL connection to the database that was authorized

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BRoYE5ZeAVvrDYdfhDAYXb

* fix(datatables): compare the external cluster settings under a row lock before storing setup

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* fix(datatables): only drop external databases Windmill marked, and check use under the lock

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* fix: build the ACL connection from the authorized data table entry

Resolving the settings again could land on a resource with the same
database name on another server, which the later entry checks never see.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BRoYE5ZeAVvrDYdfhDAYXb

* fix: check ACL read reach against the entry it connects from

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BRoYE5ZeAVvrDYdfhDAYXb

* feat(datatables): put a data table's connection under Postgres roles

A data table backed by the instance database resolved to exactly one Postgres connection,
`custom_instance_user`, for everyone who could reach it at all. There was no way to say
this job reads, that one writes, this one never sees the salaries table.

A data table role is now a real Postgres login on the cluster, defined once for the
instance by a superadmin and named exactly as they named it. A script that declares
`-- role analytics` connects as `analytics`, and Postgres decides what it may touch —
grants are ordinary SQL. Windmill answers only "may this caller ask for this role", from
the tenant lists on the data table entry: `u/alice`, `g/analysts`, `f/finance` or `*`.
A data table with no `permissions` block behaves exactly as before.

Everything that opens a connection on someone's behalf goes through one chokepoint,
`get_datatable_resource_from_db`, which takes the identity explicitly and fails closed when
there is none. The role logs in as itself — never `SET ROLE`, which a script could
`RESET ROLE` its way out of.

A fork's data table entry becomes a pointer at the workspace that governs it rather than a
copy of it. The settings clone used to hand a fork a byte-identical entry naming the
parent's database, which a fork admin could edit to grant themselves `admin` there; a
pointer has nothing local to edit, and its tenants are evaluated as a member of the
governing workspace, by email. `permissions` is stripped from the workspace export and
ignored on import: tenants name principals of one workspace, and a settings push is not
where an access decision should be made.

Operations that see the whole database whatever the roles grant stay with the governing
workspace's admins: editing the roles, a migration that declares none, and opening a
replication stream for a Postgres trigger or capture.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012ti5HyeTikPMYyW8YSdiHR

* fix(datatables): gate the paths that reach a whole database as admin

Auditing what still resolved through the unchecked resolver turned up three that act for a
caller and hand back the admin connection: `resolve_pg_source_checked` (behind schema
export, the full-schema read, database creation, import and the forked-database drop), the
connection test, and the schema snapshot a fork clone takes of its parent. On a data table
under roles each let any workspace member — or a fork admin who is nobody in the governing
workspace — read or copy the whole database whatever its roles grant.

All three now require admin reach on the governing workspace. A dump taken under a
restricted role would be a silently truncated copy rather than an error, so refusing is the
only right answer for the copy paths.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012ti5HyeTikPMYyW8YSdiHR

* fix(datatables): confine roles to the instance database, and stop a fork reaching the parent's bookkeeping

A data table role is a login on Windmill's own Postgres. Nothing stopped a workspace admin
putting a *resource-backed* data table under roles, at which point the executor dialled the
host that resource names — one the admin chose — with the role's real cluster password, and
`CONNECT` is granted to every registered instance database. Both ends now refuse: the
permissions endpoint rejects the save, and the chokepoint refuses to substitute credentials
on a non-instance entry rather than trusting the record it read.

Two more places reached the governing database without answering to it. The initial-migration
generator returned a `pg_dump` of the whole schema to any member. And the migration
rename/delete cascade followed a fork's pointer into the parent, so a fork admin renaming or
removing their own local entry relabelled or wiped the parent's `_wm_migrations` — after
which the parent re-runs every migration from zero. The remote half is now skipped when the
entry resolves into another workspace, which is also just correct: a fork renaming what it
calls a data table changes nothing about the data table.

Also: revoking a tenant now bounces the replication streams of every workspace holding an
entry that resolves here, not only the governing one, so a fork's trigger stops rather than
living on inside its open connection; the instance role catalog and the governing workspace's
tenant lists are no longer returned to someone who cannot edit them; and the tenant rename
dedup collapses non-adjacent duplicates, per role rather than once any role changed.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012ti5HyeTikPMYyW8YSdiHR

* fix(datatables): fail loudly where a role or a pointer can be left half-recorded

Three ways the feature could end up in a state nobody could see or undo.

Creating a role writes the cluster first and the catalog second, but the catalog write was an
`UPDATE` that matched nothing when the instance Postgres settings row was absent — leaving a
live login with a password nobody recorded: invisible to the catalog, un-recreatable because
the name is taken, and un-deletable because there is no entry to delete. It now errors, so
the operation is retryable once the row is restored.

Deleting a workspace only nulls the fork lineage; the data table entries pointing at it are
left resolving to nothing. Sweeping them is not an option — turning a pointer back into a copy
would hand each fork the database outright — so the delete now names the data tables it
stranded, and resolving one says which workspace is missing rather than reporting a data table
this workspace never had.

`InstanceDatatableRole` derived `Debug` while holding a Postgres password; it is now
hand-written so `{:?}` on the catalog cannot put a live credential in a log line.

Adds the two branches the reviews found unpinned: a caller who is not a member of the
governing workspace at all, and `NoIdentity` — the compatibility path for an agent worker that
predates this and sends no job id.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012ti5HyeTikPMYyW8YSdiHR

* fix(datatables): unbreak two operator messages and two comments that described other code

The two strings this branch added for states an operator hits once — the catalog write that
matched nothing, and the delete that stranded a pointer — were collapsed from their multi-line
form with the indentation left in, so both rendered with a fourteen-space gap mid-sentence.

`list_datatables` claimed to report a chain it cannot follow and then dropped it; it does drop
it, and the comment now says why that is the right place to stay quiet. The non-superadmin
check in `edit_datatable_config` was introduced as also covering references, which it does not
and need not: `reference` is overwritten from the stored entry for every caller before the
check runs.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012ti5HyeTikPMYyW8YSdiHR

* fix(datatables): serialize role catalog mutations, and state each helper's authorization contract

The catalog is one JSON document, so create, rename, enable and delete are all
read-modify-write. Two concurrent creates read the same snapshot, both succeed in the
cluster, and the second write drops the first — leaving a live Postgres login with a password
nobody recorded, which is the exact state the delete path exists to prevent. Every mutation
now runs in one transaction holding an advisory lock across the read, the cluster DDL and the
write, so a lost update cannot happen and a failure rolls the whole thing back. The DDL
helpers take that transaction rather than the pool, which is what makes the lock cover them.

Their statements moved off `sqlx::raw_sql`: the simple protocol is only needed for genuinely
multi-statement SQL, and its future is not `Send`, which an axum handler holding the
transaction requires. Each of these is one statement anyway.

The new cross-crate surface now says what callers must do. `read_role_catalog` returns
plaintext credentials; `create`/`rename`/`set_login`/`drop_instance_role` and
`converge_connect_grants` mutate cluster-wide state; `read_datatable_entry` reads a workspace's
raw config. All of them are superadmin-gated by their current handlers, but nothing said so at
the definition, which is where the next caller looks.

Also: the roles table reloads after a failed login toggle instead of leaving it claiming a flip
that did not land; the rename affordance is the design-system `Button`, not a raw one; and
`resolve_datatable_pg_as_caller` drops a `role` parameter no caller ever filled — browsing
resolves as the data table's default until the database manager grows a picker.

Why role passwords stay a plain `String` while the instance user's password beside them is a
`StringOrSecretRef`, asked three times across reviews: that one is a secret ref because an
operator supplies it and may want it from their own backend, while these are minted here and
never entered by anyone, so there is nothing for a ref to point at. Encrypting generated
secrets at rest is a separate change that would take the replication password with it. Now
said at the field.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012ti5HyeTikPMYyW8YSdiHR

* fix(datatables): give the role catalog its own row, out of reach of the config machinery

Putting it inside `custom_instance_pg_databases` was the wrong call, and it cost two ways.
The catalog serializes a generated Postgres password per role, and that row is the
operator-facing instance config, so the passwords reached `get_instance_config` and its YAML
editor — a live cluster credential in a response body, a UI field and any log of either.
Worse in the other direction: `to_settings_map` strips the catalog, so a full-row upsert of
that key writes the row back without it and the catalog is gone, while the cluster keeps every
login it described.

`custom_instance_replication_pwd` is the precedent and says exactly why — a generated secret,
written only by the server, never operator-authored, hidden so the config machinery cannot
read, rewrite or drop it. The catalog is the same thing, so it now has the same shape:
`datatable_roles`, in `HIDDEN_SETTINGS`, `PROTECTED_SETTINGS` and the agent-worker denylist.
No redaction to keep in step with three code paths, and no way for a neighbouring write to
take it out.

Two races on the same shared documents. `edit_datatable_config` read the stored data tables
outside its transaction and then wrote the whole `datatable` document, so a permissions save
committing in between was silently rolled back; it now reads under `FOR UPDATE`. And
`set_datatable_permissions` validated role ids against the catalog before opening its
transaction, so a deletion in between let it write a deleted role back — including as the
default, which every later job then fails on; it now holds the catalog lock and the settings
row across validation and write.

Completes the authorization contracts the previous commit claimed but did not finish:
`read_datatable_entry` (which it named and missed), `resolve_governing_datatable`, whose whole
job is to answer for a workspace the caller may not belong to, and
`converge_connect_grants_with`, which had not inherited its wrapper's.

Also the generic Python SDK reference: `_format_py_params` learned the bare `*` last time, but
`extract_py_functions` is a second formatter and still rendered `datatable(name, role)`, so
code written from that page passed a keyword-only argument positionally.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012ti5HyeTikPMYyW8YSdiHR

* fix(datatables): make the concurrency test pin the handlers, and the contracts describe what is enforced

The concurrency test reimplemented the read-modify-write inline, so deleting the lock from all
three handlers left it green — it pinned Postgres, not the code it was written for. It now
drives `create_datatable_role` twice concurrently and asserts the catalog kept both names.
Checked the way the last one should have been: removing the lock from the handler makes it
fail with "wmtest_a_… is a live cluster login the catalog forgot".

The contracts added last commit were stricter than this PR's own callers, which is worse than
none — the next reader sees a rule already broken and learns to ignore it.
`read_role_catalog` said superadmin-only while two of its four callers are open to any
workspace member, and `converge_connect_grants` said superadmin while
`set_datatable_permissions` reaches it as a workspace admin. Both were fine on substance: the
rule that actually holds is about the credential never reaching a response, log, audit record
or export, not about who may call. They now say that. `read_datatable_entry` gets the same
treatment rather than the one the earlier message claimed for it: it is the primitive every
resolution goes through, so it is deliberately open, and what must not escape is `permissions`
— it names the governing workspace's users, groups and folders.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012ti5HyeTikPMYyW8YSdiHR

* fix(datatables): close the last ways a role or a pointer can be left pointing at nothing

The raw settings readers hand back whatever is in the row, so moving the catalog into its own
`global_settings` key protected the config machinery and left `GET /settings/global/datatable_roles`
and the settings listing returning every live password. Both now filter that one key. The
neighbouring `custom_instance_replication_pwd` has the same shape and is not touched here: it
predates this and widening the fix to it is a decision about an operator workflow, not a
consequence of this change.

Three ways a save could leave something resolving to nothing:

A permissioned data table could be moved to a PostgreSQL resource. The block was carried across
as a server-owned field, the runtime refuses roles on a resource-backed table, so the save
succeeded and every job afterwards failed. Refused instead — turning roles off first is one step,
and it keeps discarding an access decision something somebody chose.

Renaming a governing data table left every fork pointing at the old name: the data table
disappears from their pickers and their jobs stop, with nothing in the renaming workspace to
suggest why. The rename now follows into the pointers in the same transaction.

Deleting one cannot be followed the same way, so it is reported instead — the response names what
it stranded, the way deleting a workspace does, and the fork's own error already says which
workspace is gone.

Also: `ensure_instance_db_grant_options_unchecked` claimed superadmin while the permissions
handler reaches it as a workspace admin (the same class fixed last commit, one instance missed);
the role entry kept an `instance_config_schema` derive it no longer needs; `write_role_catalog`
was the one writer of that table not stamping `updated_at`; and the concurrency test dropped its
roles only on success — a failing run is exactly the one that creates them without recording them.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012ti5HyeTikPMYyW8YSdiHR

* refactor(datatables): put the role catalog in its own table, not in global_settings

Five findings across three rounds were all the same choice. A set of live Postgres credentials
was living in `global_settings`, which has generic read, list, write, config-export and CLI
round-trip paths that know nothing about what they carry: the passwords reached the instance
config and its YAML editor, a full-row upsert of a neighbouring key erased the catalog,
`GET /settings/global/{key}` and the settings listing returned them raw, and this round the
redaction that fixed the last two turned `wmill instance push` into something that wipes every
password — a fix breaking the assumption the previous fix made. `POST /settings/global/datatable_roles`
could also empty it outside the lock.

The approved plan offered a table or `global_settings`, so this is the other option it already
allowed rather than a new design. `datatable_role` is a table: no generic settings path can read
it, list it, export it, write it or round-trip it, so none of the five needs a guard. The
redaction, the hidden/protected/agent-denylist entries and the JSON document all go with it.

One row per role also removes the read-modify-write the concurrency work was about: two
concurrent creates are two inserts, and the unique index on `name` is what settles a collision.
The advisory lock stays for the one window rows do not cover — `CREATE ROLE` is invisible to
another transaction until commit, so without it both creates pass their `pg_roles` check.

Also from this round: rename mappings are checked against the configuration they claim to
describe, since fork pointers are rewritten from them — a caller could otherwise submit
`main -> missing` against an unchanged config and repoint every fork of `main` at a name nothing
has, and `A -> B` plus `B -> C` moved what pointed at `A` all the way to `C`. And the warning
naming forks a delete stranded reached the response but not the screen: both the data table
settings save and the workspace delete now show it.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012ti5HyeTikPMYyW8YSdiHR

* fix(datatables): validate a rename against the save it describes, and re-check under the locks

Three from the round, all about deciding on state that could already have moved.

A permission save resolved the data table and checked it was instance-backed before taking any
lock, then wrote under one. A config save committing in between could move the table onto a
PostgreSQL resource — recreating exactly what the transition guard refuses — or rename it, in
which case the write targeted a key that no longer existed and reported success having changed
nothing. It now re-resolves and re-checks on the locked state.

Rename validation checked that the source existed before and the target existed after, which
still accepts `main -> decoy` against a save that keeps both: every fork of `main` then follows
onto a different data table, silently, because it keeps resolving. The rule is now the actual
old-to-new key transition — a source may only survive if another rename took its name, and a
target may only pre-exist if another rename freed it. That also stops two sources sharing one
target, and it admits a swap, which the previous guard refused: `datatables` is keyed by name, so
a swap cannot be done one save at a time, and refusing it was a regression against main. The
pointer cascade now runs in two passes through a temporary name, the way the migration cascade
one layer down already handles the same shape, so `A -> B` with `B -> C` moves each pointer once
from what it named before the save.

The tenant mutators say what they are for: they write an access decision for any workspace named,
with an arbitrary mutation, and exist for the transaction that frees or renames a principal.
Editing a decision on purpose belongs in the permissions endpoint.

Carried in the same change: the stranded-fork list is a field rather than a phrase to grep out of
a success string; the pointer cascade matches with `EXISTS` instead of a `LIKE` over the whole
document, so a workspace whose pointers name something else is not rewritten to a byte-identical
value under an exclusive lock; and `InstanceDatatableRole` drops the serde derives left over from
the JSON document, one of which would emit `pwd`.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012ti5HyeTikPMYyW8YSdiHR

* fix(datatables): cascade on the leave route that is used, gate migrations before the admin connection, and drop a role atomically

The tenant cascade on leaving went onto `/users/leave`. The UI and the generated client call
`/workspaces/leave` — a different handler in a different crate with the same name — which
deleted the membership and left `u/<username>` in the tenant lists. Leaving and rejoining
therefore restored the access the leave was supposed to end, and a later account taking the
username would have inherited it. The regression test drives the route the client actually
calls; without the fix it fails with "leaving kept the tenant".

The migration endpoints authorized too late. `run_datatable_migrations` opened the data table's
admin connection, created `_wm_migrations` and read it before reaching the per-migration role
check — so with nothing pending, nothing was checked at all. Rollback returned before its check
when nothing was applied, and the status endpoint had none. All three now ask, before any
connection is opened, whether the caller can reach the data table as any role at all; which role
a given migration runs as is still decided per migration, and by the executor after that.

Deleting a role committed the cluster drop and the catalog row, then swept the tenant lists in
separate transactions. A sweep failing part-way left workspaces naming a role nothing can connect
as, while the retry answered `NotFound` because the catalog entry was already gone. The sweep now
runs in the same transaction, so the drop, the row and every tenant list commit together.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012ti5HyeTikPMYyW8YSdiHR

* fix(datatables): refuse to copy a data table that is under roles

pg_dump carries no roles and the import runs with --no-privileges, so a copied
data table arrives owned by the admin connection with no GRANT for any role.
The settings clone brings `permissions` across, so the fork's tenants pass
Windmill's check, connect as the role they were given, and are denied by
Postgres on everything: an entry that reads as configured and answers nothing.

Refuse the copy — in the import endpoint before any data moves, and in the fork
path the CLI takes. Replaying the source's owners and ACLs into the clone is
what lifts this, and is a change of its own. Dropping `permissions` from the
copy instead would be the unsafe half, since the copy holds the parent's rows.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012ti5HyeTikPMYyW8YSdiHR

* fix(datatables): refuse the clone's database too, not only its data

A clone is two endpoints: `create_pg_database` then `import_pg_database`. Only
the second refused a data table under roles, so a fork asking to clone one
created and registered an empty `wm_fork_…` instance database and then failed —
and nothing collects it, since `drop_forked_datatable_databases` only drops
entries carrying `forked_from` and no entry names this one.

Refuse in both, so the clone stops before a database exists.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012ti5HyeTikPMYyW8YSdiHR

* nit worker error msg

* fix pg_dump stuck on version 17 on nix

* fix(datatables): refuse a malformed role annotation instead of ignoring it

`-- Role operator`, `-- role operator;` and `-- role operator -- why` all failed
the annotation parser's exact-match rule, so the query fell through to the data
table's default role and ran, silently, under a login the author did not choose.
Naming a role exists precisely to not do that.

A leading comment whose first word is `role` is now an annotation attempt: the
keyword matches case-insensitively, one trailing `;` is tolerated, and anything
else is an error naming the line. Only callers that already know the target is a
`datatable://` reference ever run this, so ordinary SQL keeps its comments.

Also bumps the dev shell's postgres client to 18 — it trailed the server the dev
database runs, which takes out every data table export, clone and fork-with-data.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012ti5HyeTikPMYyW8YSdiHR

* fix(datatables): refuse a malformed role query string instead of ignoring it

`?Role=analytics`, `?role=` and `?x=1&role=…` all fell through the reference
parser's exact-match rule, so the connection resolved to the data table's default
role and ran under a login the caller never asked for — the URI half of the same
trap as a malformed `-- role` annotation.

The key now matches case-insensitively, and anything else in the query string is
an error naming it; `role` is the only parameter a reference takes. Callers that
only need the entry keep a lenient `datatable_ref_name`, since they never act on
the role. The DuckDB `ATTACH` parser propagates it rather than attaching under
the default.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012ti5HyeTikPMYyW8YSdiHR

* fix(datatables): carry the role annotation into the row_to_json retry

The retry rebuilds its SQL from `pruneComments(code)`, so the leading comment
block never reached the second attempt — and with it the `-- role <name>` line
that decides which login the query runs as. The retry connected as the data
table's default role instead, so a query the first attempt was denied could
succeed on the second, reported as "recovered with the row_to_json fix".

Carry the leading comment block over. The retry itself is unchanged.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BjfMkJyKzodxkobqGZ6Lqb

* chore(datatables): don't mount the roles UI until the ACL editor lands

Enforcement ships first. The permissions drawer is what turns roles on, and the
catalog section is what creates them — both are only useful once there is a way
to grant a role the privileges it needs, which arrives with the ACL editor. Left
mounted they would offer a feature whose other half does not exist.

The two components are complete and reviewed; only their call sites here are
commented out, with a note pointing the follow-up PRs at them.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BjfMkJyKzodxkobqGZ6Lqb

* fix(datatables): honour `-- role: x`, and fix the DuckDB attach test

Two review findings, both real.

`attach_datatable_parses_name_and_role` never compiled: `parse_attach_datatable`
returns `Result<Option<_>>` now and one call site kept a single `unwrap`. Its
`?Role=analytics` case also asserted a refusal, contradicting the parser in the
same commit, which matches the key case-insensitively. Replaced with the cases
that are genuinely malformed, and a positive one for the cased key.

`-- role: analytics` fell through to the default role — the silent fallback the
strict parser exists to remove, for the spelling most likely to be typed. The
keyword now accepts an optional colon, attached or spaced, while a word that
merely starts with it (`rolebased`) is still not an attempt.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BjfMkJyKzodxkobqGZ6Lqb

* fix(datatables): clone a fork's pointer instead of failing after the copy

Forking a fork with cloning left an orphan database. The preflight resolves the
pointer and sees the governing entry, so both endpoints ran and filled the new
database; `apply_forked_datatable` then refused the inherited pointer and rolled
the fork back, stranding a registered `wm_fork_*` that no entry names and whose
name blocks the retry.

Refusing earlier would have been the smaller change, but forking a fork and
cloning worked before pointers existed, so it would trade an orphan for a
regression. Resolve what the pointer names and write the terminal entry the
clone needs: the whole `database` object rather than a patch of its
`resource_path`, since a pointer has none, and `reference` removed with it.

Also accepts `-- role=x` and `-- Role = x`, two more spellings that fell through
to the default role.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BjfMkJyKzodxkobqGZ6Lqb

* fix(datatables): refuse to roll back the catalog while roles exist

The down migration dropped the table and left every role behind: live Postgres
logins whose passwords only that table carried, so after a revert Windmill could
neither use, disable nor delete them, and re-applying could not recreate them
because the names were taken. Cleaning up here is not possible either — dropping
a role means reassigning what it owns in every instance database, and a
migration runs in one — so it now refuses while the catalog is non-empty and
says to delete the roles through instance settings, which does the cluster work.

Also enforces the instance-only invariant the resolved-pointer clone relies on
rather than only asserting it in a comment.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BjfMkJyKzodxkobqGZ6Lqb

* refactor(datatables): settle clonability in one place, before anything is created

A clone is three stages a workspace apart — `create_pg_database`, then
`import_pg_database`, then `apply_forked_datatable` inside the fork transaction.
Only the third can roll back, and `CREATE DATABASE` is not transactional, so any
refusal that lives there strands a registered `wm_fork_*` that no entry names
and whose name blocks the retry.

That orphan has now been fixed three times, most recently reintroduced by a
guard added one commit ago. Patching each new refusal into the first endpoint is
not the fix; having two places that can refuse is. `ensure_datatable_is_clonable`
now answers every reason a copy can be refused and returns what it resolved, and
the stage that writes the entry only does the work.

Also takes an ACCESS EXCLUSIVE lock before the rollback guard counts, so a role
created concurrently cannot slip between the check and the drop.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BjfMkJyKzodxkobqGZ6Lqb

* fix(datatables): let a retried clone reclaim its own leftover database

A clone creates its target database one request before it copies into it, and
the fork that would name it is written a request after that. Any failure in
between — a pg_dump error, a bad restore, a dropped connection, the source's
roles changing mid-flow — left a registered `wm_fork_*` that no entry names,
and every retry then failed on its name. This predates data table roles.

`create_pg_database` now reclaims such a leftover before creating: only a
`wm_fork_*` database Windmill registered as a data table database and that no
data table or ducklake entry names, in any workspace, archived ones included.
The drop never terminates connections, so a clone still copying into it makes
the reclaim fail instead of being cut off. It is limited to callers who
administer the source — reaching it is not enough, since on a data table
without roles every member reaches it — and anyone else gets the refusal an
existing database always got.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* Revert "fix(datatables): let a retried clone reclaim its own leftover database"

This reverts commit 7dd3275a10.

The reclaim tied the caller to the source they administer, but not to the
database it dropped. Between another workspace's import and its final fork
request, that workspace's target is full, registered, unnamed and has no open
connection, so an admin of any instance data table could name it and have it
dropped and recreated empty. The victim's fork would then commit pointing at
the empty copy. Safe reclaim needs durable clone ownership and serialization
with the request that names the database; until then the leftover stays, as it
did before this PR.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* docs(datatables): record the stale clone database as a known limitation

A clone is three requests and `CREATE DATABASE` is not transactional, so a
failure after the first leaves a registered `wm_fork_*` behind, as it did
before data table roles. Accepted for this PR: it is harmless to data and goes
away once the clone is a single server-side operation.

The comment also records why the obvious fix is wrong: reclaiming the leftover
on retry, without durable clone ownership, can drop another workspace's fully
copied database between its import and its final fork request.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* fix(datatables): bounce the streams reading a data table when it is deleted

Deleting a governing data table, or the workspace that holds it, only collected
the fork pointers it stranded, for the warning. A Postgres trigger or capture
already streaming through one of those pointers kept the replication connection
it opened while the pointer still resolved, so it went on dispatching the
governing database's rows after the fork lost access — until its connection
happened to restart. The governing workspace's own streams on a deleted entry
did the same.

Both deletion paths now bounce the affected listeners inside their own
transaction, through the helper a permission change already uses, so a
listener that reconnects re-resolves the entry and finds it gone. The helper is
split so a caller can pass the (workspace, local name) pairs it already holds.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* fix(datatables): keep the fork schema baseline, and bounce streams on every removal

Three fixes from review.

`edit_datatable_config` took `forked_from` wholesale from the stored entry, so
the fork schema diff's save of an advanced baseline was silently discarded and
an applied change was offered again. Whether an entry carries a clone stamp is
still carried from the store, since that is what marks its database droppable,
but the baseline inside it is now taken from the request.

The stranded-pointer warning and the stream bounce ran over the optional
`deleted_datatables` hint, which the settings-sync CLI never sends, so removing
a governing data table through `wmill` bounced nothing. Removals are now derived
from the stored configuration against the saved one.

`delete_workspace` read the pointers to bounce before its transaction, so a fork
committing a pointer during the deletion was missed. The read now happens inside
the transaction, after the workspace row is deleted: a fork's insert key-share
locks that row through its parent foreign key, so it is either seen or fails on
the missing parent.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* refactor(datatables): keep Postgres triggers and data table roles apart

A replication stream reads every row of every table whatever the data table's
roles grant, and its listener checks access only when it connects. Rather than
chase every way access can change and bounce the streams each one affects, a
data table now carries one or the other:

- a Postgres trigger or capture cannot be created on, or connect to, a data
  table under roles;
- roles cannot be turned on while an enabled trigger or a live capture reads
  the data table, its own or a fork's through its pointer. The refusal names
  each one to disable.

This removes the stream bounces on roles edits and on data table and workspace
deletion, and the trigger gate that admitted admins. The fork schema baseline
fix from the same review round is kept.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* fix(datatables): refuse a Postgres trigger on a data table under roles when it is saved

Creating or editing a trigger that points at a data table under roles was
accepted, and its listener then retried the refused connection every 30
seconds forever. The save is now refused, and a trigger that reaches such a
data table anyway (re-enabled, or cloned into a fork) is disabled by its
listener with the reason, as a missing replication slot is.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* fix(datatables): disable a data table role before deleting it

Deleting a role reassigns and drops what it owns in each registered
database on its own connection, and each of those passes commits as it
goes. A database failing part-way left the role enabled in the catalog and
able to log in, but already stripped in the databases reached before it.
The role is now disabled in its own commit first, so a failed delete
leaves a disabled role to retry.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* fix(datatables): serialize roles going on with a stream starting

Turning roles on looked for enabled triggers and live captures once,
without a lock anything starting a stream also took. A trigger enabled in
that window could have its listener connect before roles committed, and a
healthy listener never checks again. Both transitions now serialize on one
advisory lock: roles going on hold it exclusive while they look, and
trigger create, edit and enable, and capture setup and ping hold it shared
while they commit. Either the look sees the stream, or the listener
connects after roles are committed and refuses.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* fix(datatables): wait out live listeners, and resolve stored names containing `?`

Turning roles on counted a trigger as gone once disabled, and a capture
once its client stopped pinging, but the listener keeps its replication
connection until its next heartbeat notices. A trigger or capture whose
listener pinged in the last 15 seconds, the window a server holds a
listener for, now still counts as streaming.

Data table names could contain `?` before they were restricted, and such
entries are still stored. Splitting `?role=` off a reference misread them:
`a?b` became `a` with an unknown parameter, and the clone checks looked at
a different entry than the one copied. An entry stored under the whole
reference is now looked up first, in the Postgres executor, DuckDB ATTACH
and the clone checks. Agent workers cannot read the workspace and keep
the strict parse, which refuses such a name rather than misreading it.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* fix(datatables): warn when a settings sync strands fork pointers

A settings save reported the fork pointers left resolving to nothing only
for the names in `deleted_datatables`, which `wmill sync push` never sends.
The save now works out what it removed from the locked entries, and the
CLI prints the stranded pointers it returns.

Also correct the replication helper's contract: no role or admin check
makes a replication connection safe, so a data table under roles is
refused outright rather than gated as an admin operation.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BjfMkJyKzodxkobqGZ6Lqb

* fix(datatables): refuse a save that drops a data table's roles through an undeclared rename

A data table's roles follow its entry only through a declared rename. A
settings sync sends the whole map and never declares one, so renaming a
data table under roles there read as a delete and a new entry on the same
database: the new entry carried no roles, and every caller connected as
admin. Such a save is now refused, naming both entries.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BjfMkJyKzodxkobqGZ6Lqb

* fix(datatables): no entry without roles may newly reach a database under roles

The previous guard only caught a new name replacing an entry under roles.
A whole-map save could also repoint an existing entry without roles at
that database, or another workspace could point one there, and every
caller of that entry would connect as admin. The rule is now stated on
the saved entries: one that carries no roles and newly points at an
instance database any entry under roles uses, in this workspace or
another, is refused. A declared rename carries its roles and passes.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BjfMkJyKzodxkobqGZ6Lqb

* feat(datatables): move data table role catalog and resolution to the enterprise edition

Roles are an Enterprise Edition feature. The catalog, the Postgres logins,
CONNECT convergence, tenant evaluation and the role half of connection
resolution move to windmill-ee-private. Every public function keeps its path
and signature and forwards through datatable_roles_oss, which re-exports the
enterprise implementation or, without it, refuses.

Without the enterprise edition a data table under roles, or a caller naming a
role, is refused a connection rather than resolved as admin, and the reach and
admin-access checks refuse one under roles. A data table not under roles
resolves as before in every edition, and an instance database keeps the
CONNECT grants it was created with. The catalog lock, the stream lock, the
tenant cascades and the permissions stripping stay in OSS: they only restrict.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BjfMkJyKzodxkobqGZ6Lqb

* feat(datatables): move the data table permissions endpoints to the enterprise edition

The permissions read, save and usable-roles handlers move to
windmill-ee-private; the routes stay registered and, without the enterprise
edition, answer that data table roles are an Enterprise Edition feature.
ensure_governs_datatable and ensure_reaches_datatable keep their paths: the
first refuses, the second passes a data table not under roles.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BjfMkJyKzodxkobqGZ6Lqb

* feat(datatables): move the data table role catalog endpoints to the enterprise edition

The superadmin list, create, update and delete handlers move to
windmill-ee-private. The routes stay registered and, without the enterprise
edition, refuse after authentication.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BjfMkJyKzodxkobqGZ6Lqb

* test(datatables): run the roles tests on the enterprise edition, refusals without it

Each test that exercises roles runs with private and enterprise. Two tests run
without them: every roles route answers the Enterprise refusal, and a data
table saved under roles, or a named role, is refused a connection while one
not under roles resolves as before.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BjfMkJyKzodxkobqGZ6Lqb

* feat(datatables): gate the roles UI mount sites on an enterprise license

Both mount sites are still commented out; the gate travels with them.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BjfMkJyKzodxkobqGZ6Lqb

* test(datatables): run the tenant matcher test on the enterprise edition

The matcher it covers is enterprise code now, so without the enterprise
edition the test hit the stub and failed the default windmill-common run. It
runs with private and enterprise, and a counterpart without them asserts that
no tenant list covers anyone, the wildcard and a workspace admin included.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BjfMkJyKzodxkobqGZ6Lqb

* chore: update ee-repo-ref to a1873dbb67f2302b85ff5362f8387b48eccdb607

This commit updates the EE repository reference after PR #783 was merged in windmill-ee-private.

Previous ee-repo-ref: 5c853e2c20eca6b748415fc0d6862a6ebfb5fec4

New ee-repo-ref: a1873dbb67f2302b85ff5362f8387b48eccdb607

Automated by sync-ee-ref workflow.

* fix(datatables): refuse roles while a same-workspace alias reaches the database

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* fix(datatables): let CE migrations connect as an explicitly named admin

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* fix(datatables): serialize roles going on with aliases saved from other workspaces

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* docs(datatables): note that legacy names with ? cannot be migrated

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* feat(datatables): add an ACL editor for data table roles

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* fix: take every pooled connection before the ACL apply locks

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BRoYE5ZeAVvrDYdfhDAYXb

* fix: refresh grant options only after the ACL apply validates its plan

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BRoYE5ZeAVvrDYdfhDAYXb

* fix: add only missing grant options before an ACL apply, never default privileges

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BRoYE5ZeAVvrDYdfhDAYXb

* fix: run one data table ACL apply at a time per server before it connects

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BRoYE5ZeAVvrDYdfhDAYXb

* fix: hold the ACL connection to the database that was authorized

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BRoYE5ZeAVvrDYdfhDAYXb

* fix: build the ACL connection from the authorized data table entry

Resolving the settings again could land on a resource with the same
database name on another server, which the later entry checks never see.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BRoYE5ZeAVvrDYdfhDAYXb

* fix: check ACL read reach against the entry it connects from

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BRoYE5ZeAVvrDYdfhDAYXb

* feat(datatables): Ducklake catalogs on the external instance cluster

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* fix(datatables): never grant CREATEROLE to custom_instance_user on the external cluster

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* docs(datatables): state the authorization contract of external database usage lookups

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* fix(datatables): drop a DuckDB data table secret once its ATTACH has used it

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* perf(datatables): resolve a workspace's data tables per pointer hop, not per entry

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* fix(datatables): hold the parent's settings while a fork points at its data tables

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* fix(datatables): refuse repointing the external cluster while it is in use, and keep verify-ca working for pg_dump

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* fix(datatables): protect external databases pending fork cleanup, and describe Ducklake usage in the API

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* feat(datatables): per-cluster data table role catalogs, with roles on the external instance cluster

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* fix(datatables): write the external cluster setting under the lifecycle lock, and check fork targets are registered

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* fix(datatables): register external fork catalogs under the lifecycle lock, and keep certificate verification in DuckDB attaches

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* fix(datatables): keep certificate verification when DuckDB attaches an external data table

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* fix(datatables): keep certificate verification when DuckDB attaches an external data table

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* fix(datatables): refuse fork cleanup of an external database another workspace uses

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* fix(datatables): refuse rolling back while external data tables are under roles, and type external_instance in the CLI

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* fix(datatables): stop counting storage-only fork cleanup rows as uses of an external database

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* fix(datatables): bind fork database copies to their workspace, and count every use before dropping one

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* fix(datatables): authenticate instance database setup before writing its status, and keep a fork reservation across it

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* fix(datatables): create external databases only on a cluster setup succeeded on, and document the registry reader

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* fix(datatables): keep only the most recently used DuckDB root certificate files

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* fix(datatables): migrate fork reservations on workspace rename, and lock the parent's data tables for the whole fork

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* fix(datatables): take the fork data table lock once, before the external cluster's

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* docs(datatables): describe the external instance cluster and how to run one locally

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* fix(datatables): keep fork reservations private, drop a cleaned-up entry with its database, and serialize cleanup with settings saves

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* fix(datatables): hold the fork lock across a fork import, and carry the reservation inside the setup write

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* fix(datatables): check the external cluster setting on its own transaction, and gate the registry probe

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* feat(datatables): add an ACL editor for data table roles

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* fix: take every pooled connection before the ACL apply locks

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BRoYE5ZeAVvrDYdfhDAYXb

* fix: refresh grant options only after the ACL apply validates its plan

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BRoYE5ZeAVvrDYdfhDAYXb

* fix: add only missing grant options before an ACL apply, never default privileges

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BRoYE5ZeAVvrDYdfhDAYXb

* fix: run one data table ACL apply at a time per server before it connects

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BRoYE5ZeAVvrDYdfhDAYXb

* fix: hold the ACL connection to the database that was authorized

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BRoYE5ZeAVvrDYdfhDAYXb

* fix: build the ACL connection from the authorized data table entry

Resolving the settings again could land on a resource with the same
database name on another server, which the later entry checks never see.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BRoYE5ZeAVvrDYdfhDAYXb

* fix: check ACL read reach against the entry it connects from

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BRoYE5ZeAVvrDYdfhDAYXb

* chore: update ee-repo-ref to 7e338e4dabf91689bfd7fb0333c6534040b17b59

This commit updates the EE repository reference after PR #787 was merged in windmill-ee-private.

Previous ee-repo-ref: 0edd40979cf36bfba59323f3f6a0811ae1369cf5

New ee-repo-ref: 7e338e4dabf91689bfd7fb0333c6534040b17b59

Automated by sync-ee-ref workflow.

* fix(datatables): keep DuckDB root certificate files in the job directory

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* chore: drop the unused json import from the settings crate

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* fix(datatables): drop the serde_json::json import left unused by the fork setup write

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* fix(datatables): accept external_instance data tables in the settings form type

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* fix(datatables): serialize fork database reservations, cleanup, setup and Ducklake saves on the database

- A fork copy is created and registered under its workspace's fork lock, and refused once the
  workspace is archived; a rename re-migrates reservations under that lock after archiving.
- Ducklake saves lock every instance database they newly name, as data table saves do.
- Instance database setup holds the database's lock until its entry is written.
- Fork import also holds the database's lock across the restore.
- Cleanup re-reads the entry under its locks before dropping anything.
- Non-superadmins no longer see fork copies reserved for workspaces they are not in.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* fix(datatables): state the authorization contract of the external cluster status and write check

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* fix(datatables): lock the fork copy at finalization, and order cleanup's registry write after its settings row

- Fork finalization holds the copy's database lock from its availability check to the commit.
- Cleanup removes the registry entry in its own transaction, in a task of its own, instead of on a
  second connection; a rename migrates reservations after its settings rewrites, so both take the
  settings rows before the registry.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* fix(datatables): hold the fork lock across a rename's settings copy

Fork cleanup of the old id could otherwise drop a copy the renamed workspace goes on using. Also
states the authorization contract of the instance database drop helpers.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* fix(datatables): refuse moving the external cluster to another host or port while it holds databases

Every settings writer goes through the same check as removal: the login, TLS and maintenance
database may still change, the cluster may not.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* feat(datatables): clone a data table under roles with its owners and grants (#11120)

Claude-Session: https://claude.ai/code/session_01UbrtwiYNfayrmqouBJHwGV

Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>

* fix: open the raw app data table drawer when the workspace has none

Selecting the first data table of an empty list passed undefined to the name
check, which threw instead of opening the drawer on no data table.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* fix(datatables): offer cloning a data table under roles where its grants can be replayed

The server clones such a data table and replays the source's owners and grants,
which only the Enterprise Edition does, so the fork wizard hid both clone
options everywhere instead of on a build that cannot replay them.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* docs: name the placeholder the empty raw app data drawer renders

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* fix(datatables): replace a job directory file at the DuckDB root certificate path instead of trusting it

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* test: pin the enterprise refusal the role pickers read as 'not under roles'

The server's sentence and the frontend's copy of it were coupled by nothing,
so rewording either one turned every role picker on a community build into a
failed lookup.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* fix(datatables): type Ducklake catalogs on the external instance cluster in the settings form

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* fix(datatables): read a roles answer only for the workspace it was asked in

A fork and its parent each have their own roles on a data table of the same
name, so an answer stamped with the name alone settled the role from the
workspace the editor was acting on before.

Also derive the AI table creation flag from the data replaced into the editor:
data naming no data table left the flag on from before.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* fix(datatables): keep an instance database a settings save is waiting to name

Cleanup for a database whose setup failed took the lock first, read no user,
and dropped it while a save blocked on that same lock was about to commit a
reference to it.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* fix(datatables): check the workspace stamp in the default database selector too

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* fix(datatables): read what a save racing instance-database cleanup committed

A transaction blocked on the lock may still roll back, so keeping the database
for it stranded one whose name then blocks every retry: it is let through and
its outcome read instead. The waiter query also matches this database's locks
only, since pg_locks spans the cluster.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* fix(datatables): tell a waiting request apart from the workspaces using a database

Both callers render what cleanup returns as the workspaces that keep the
database, so a waiting request's pid read as one of them. Each now words that
case itself, and the give-up comment names where the kept name actually goes.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* fix(datatables): take the cleanup lock on a connection the pool cannot reclaim

A session lock outlives the future holding it, so a cancellation between
taking it and releasing it handed a locked session back to the pool, where
every later settings save waits on it. Detached, the connection closes when it
is dropped and the server releases the lock with the session.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* fix(datatables): close the cleanup connection on drop instead of detaching it

Detaching released the pool permit while the session stayed alive, so
concurrent cleanups waiting on their locks could open as many connections as
they liked. Closing on drop covers the same cancellation and keeps them
counted.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* chore: update ee-repo-ref to 179d2454a063ee818a9387a1eafdd354a217a16c

This commit updates the EE repository reference after PR #802 was merged in windmill-ee-private.

Previous ee-repo-ref: c2e43f5b5ff753d339b70e232fd17b6ffecb054d

New ee-repo-ref: 179d2454a063ee818a9387a1eafdd354a217a16c

Automated by sync-ee-ref workflow.

* chore: update ee-repo-ref to fd5b8af748f2c985b13e18d9ea30894f3bd7e9a3

This commit updates the EE repository reference after PR #798 was merged in windmill-ee-private.

Previous ee-repo-ref: 3145e422d61d580f0a82804f075285c112879da0

New ee-repo-ref: fd5b8af748f2c985b13e18d9ea30894f3bd7e9a3

Automated by sync-ee-ref workflow.

* fix(datatables): classify a fork import's target from the resolution that built its connection

A second read could see the entry flipped to a resource and skip the reservation check while the
connection already built still reached the instance cluster.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* fix(datatables): run external database lifecycle on the caller's transaction, one lock order for fork cleanup

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* fix(datatables): check a fork copy's reservation on the locked transaction's connection

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* fix(datatables): make and drop fork copies of external data tables on the external cluster

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* fix(datatables): refuse a fork whose external copy something already names

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* fix(datatables): check an external fork copy's uses before removing its entry, so child fork pointers still count

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* fix(datatables): migrate external cluster fork reservations on workspace rename

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* feat(datatables): configure the external instance cluster and pick its databases from the UI

Instance settings gets an External Postgres tab: the cluster's connection, the setup run
with its report, and the databases Windmill created there. Data table and Ducklake settings
offer the external_instance kind, with a picker over those databases.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* fix(datatables): keep the external cluster form out of the instance settings the page bulk-saves

The form seeded its key into the settings store after the page snapshotted them, so merely
opening the tab made the page send a setting the validator refuses, failing an admin's
unrelated save. The form is local state now, and only the setup writes the key.

The database picker also treats a superadmin-only listing as authoritative: a workspace
admin, who cannot list, no longer sees a saved database as missing.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* fix(tests): app paths are validated, so a guest app cannot be renamed to one with a space

Path validation on apps landed after this test, which still expected a space to be accepted
and created its fixture on a ':' path. The scopable-path guard it exists for is kept by
planting that path on the row, which is now the only way an app can hold one.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* feat(datatables): one Managed Postgres tab, and a switch for Windmill's own database

The instance settings tab covers both substrates Windmill administers: the external cluster,
which can now be disabled once nothing sits on it, and Windmill's own database, which an
operator can turn off so the cluster is the only one a workspace may newly name. A save that
names it then refuses, as the create endpoint does.

Workspace settings name them the way an admin meets them: Postgres Resource, and Managed
instance, qualified as Internal or External only while both are on offer.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* feat(datatables): offer the external cluster in the add-data-table wizard, and give Ducklake its room back

The wizard gets the external cluster as a fourth substrate: pick a database it already holds
or name a new one, which the run creates before writing the entry, so Try again does not trip
on a database the last attempt made.

Ducklake's maintenance column moves into the settings popover next to the extra args, which
the wider catalog select had squeezed the name box out of.

A setting a component saves itself now also moves the page's baseline, or Discard would
restore what it replaced and a later save would send that stale value back.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* fix(datatables): only a database this wizard run made makes its retry idempotent

A retry skipped the create whenever the name was registered, which let a run adopt a database
another superadmin had made and share their data without the warning the existing-database
branch shows. The run tracks what it created instead, and reports it so Discard can say the
database is still there. Creating one now refreshes the registry the next run's default name
and its validation read, and that default counts the external cluster's databases rather than
the instance's.

Row ownership carries the substrate: "instance" and "external_instance" can hold the same name,
and a row repointed between them while a run probes must not read as that run's own.

Also fixes the merge leftovers in the DuckDB executor's tests, which cargo check never compiles:
the new PgDatabase field and the attach helper's job directory.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* fix(datatables): shorten the picked managed kind, and say what an empty database list means

The closed select has room for the qualifier, not the whole name, so a picked managed kind
reads "Managed (Internal)". An external entry gets the tooltip its internal counterpart has,
and a database picker with nothing in it says to type a name rather than "No items found".

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* fix(datatables): keep the wizard's external-database bookkeeping optional and parked

The run deps gained a required field, which every existing caller of runSetup -- the sibling
test suite included -- does not pass. It defaults to empty instead, the parked payload carries
it across a Supabase redirect as it claims to, and a resumed run counts it as something left
behind. Two regressions pin the create: made on a first attempt, skipped only for a database
this run made.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* fix(datatables): review an external-cluster database as one, not as a Postgres resource

The review step fell through to the resource branch for the new provider, so it announced a
connection already in the workspace for a database that does not exist yet.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* fix(datatables): show the server's refusal instead of an object in the toast

Saving data table settings passed the whole error to the toast, which rendered the request
and response as JSON, and the instance-database wizard replaced it with "check console".
Both surface the message the API sent, through the helper that exists for it.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* chore: point at the EE commit dropping the unused import

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* fix(datatables): define a role on the cluster its data table sits on

The permissions drawer created and listed roles without naming a cluster, so a role defined
from an external data table became a login on Windmill's own, and the refresh then replaced
the correct catalog with the internal one. The drawer passes the data table's cluster to both.

The wizard also read a database it had just created as a name collision, which blocked
returning to the failed attempt and finishing under the same name.

And the merge had left the CONNECT-grant pass and its log in both the caller and the callee.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* feat(datatables): manage the external cluster's role catalog, and name the cluster in the copy

The roles drawer offered one catalog, so the external cluster's roles could only be reached
through a data table sitting on it — and became unreachable once the last one was gone, while
the cluster could not be unset until they were dropped. Opened from the page it now offers the
clusters the instance has; a data table's own drawer still pins its cluster, and its copy names
that cluster rather than "the instance".

A DuckDB attach also decides whether to verify certificates the way every other Postgres
connection does, so a resource carrying a root certificate is no longer downgraded to require
here alone.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* fix(datatables): bind the roles list to the cluster it was read for

Switching catalogs left the previous cluster's rows on screen and applied whichever response
landed last, so a slower read could seat one cluster's logins under the other's heading while
every control acted on the wrong id — invisible where a name exists on both. The switch clears
the rows, a token discards a response the selection has moved past, and the controls stay inert
while a catalog is being read.

The drop confirmation also names the cluster whose databases it reaches.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* chore: update ee-repo-ref to dfe7b8b9204d21e0264fbea1c6f6eedf9e738d56

This commit updates the EE repository reference after PR #810 was merged in windmill-ee-private.

Previous ee-repo-ref: 30db1b33bda0446f5fc5dfe353fbb226d57a26d4

New ee-repo-ref: dfe7b8b9204d21e0264fbea1c6f6eedf9e738d56

Automated by sync-ee-ref workflow.

---------

Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
Co-authored-by: windmill-internal-app[bot] <windmill-internal-app[bot]@users.noreply.github.com>
Co-authored-by: Ruben Fiszel <ruben@windmill.dev>
2026-09-29 14:45:47 +02:00
Ruben Fiszelandrubenfiszel 651a6b01f5 chore(main): release 1.819.0 (#11364)
* chore(main): release 1.819.0

* Apply automatic changes

---------

Co-authored-by: rubenfiszel <275584+rubenfiszel@users.noreply.github.com>
2026-09-29 08:33:39 +02:00
Ruben FiszelandClaude Opus 5.5 ec6ec1b06f fix: judge IPv4 embedded in IPv6 and pin the object storage test connect (#11389)
* fix: judge IPv4 embedded in IPv6 and pin the object storage test connect

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

* fix: let the public-only object store client reach the egress proxy

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

* fix: refuse private IP literals and the proxy host in the public-only store client

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

* fix: refuse the egress proxy as a target whether named or an IP literal

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-09-28 17:43:15 +02:00
Ruben Fiszelandrubenfiszel 3974bbeac6 chore(main): release 1.818.0 (#11294)
* chore(main): release 1.818.0

* Apply automatic changes

---------

Co-authored-by: rubenfiszel <275584+rubenfiszel@users.noreply.github.com>
2026-09-25 18:42:49 +00:00
9375c93fd8 fix: trust the system CA store for SMTP TLS (#11328)
* fix: trust the system CA store for SMTP TLS

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

* ci: run the smtp-gated backend tests

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

* chore: depend on webpki-roots 1 directly

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

* chore: update ee-repo-ref to 48193da8cb30bc4ae82a50f944caef85d8a20b48

This commit updates the EE repository reference after PR #826 was merged in windmill-ee-private.

Previous ee-repo-ref: cd3447143b25d9f3301975feca4b202755f2508f

New ee-repo-ref: 48193da8cb30bc4ae82a50f944caef85d8a20b48

Automated by sync-ee-ref workflow.

---------

Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Co-authored-by: windmill-internal-app[bot] <windmill-internal-app[bot]@users.noreply.github.com>
2026-09-24 15:10:28 +02:00
Ruben Fiszelandrubenfiszel 25c8bfaab4 chore(main): release 1.817.0 (#11268)
* chore(main): release 1.817.0

* Apply automatic changes

---------

Co-authored-by: rubenfiszel <275584+rubenfiszel@users.noreply.github.com>
2026-09-22 12:13:32 +00:00
Alexander PetricandClaude Fable 5.1 9e58f9a20f chore(security): resolve Dependabot alerts in backend/Cargo.lock and windmill-parser-wasm/Cargo.lock (#11182)
Targeted dependency bumps only; no behavior change intended.

Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
2026-09-22 08:57:21 +02:00
Ruben Fiszelandrubenfiszel 965e8b0c23 chore(main): release 1.816.0 (#11237)
* chore(main): release 1.816.0

* Apply automatic changes

---------

Co-authored-by: rubenfiszel <275584+rubenfiszel@users.noreply.github.com>
2026-09-21 16:55:24 +00:00
5095456bb6 chore(security): bump jsonwebtoken to 10 (#11194)
* refactor(jwt): route JWK algorithm lookup through one helper

Add `windmill_common::jwt::jwk_algorithm` as the single place that reads a
JWK's `alg`, and make `guest_jwt::jwk_algorithms` use it. EE code will call
the same helper, so the upcoming jsonwebtoken bump (which renames the field
to `key_algorithm: Option<KeyAlgorithm>`) only has to touch the helper's body.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

* chore(security): bump jsonwebtoken to 10

Resolves dependency alerts 193 and 344 (jsonwebtoken) and, for the
windmill-parser-wasm lock, 341 (ring 0.16).

- jsonwebtoken 8.3.0 -> 10.4.0 with the `rust_crypto` backend (10.x has no
  default crypto backend). `jwk_algorithm` now reads `key_algorithm` (a
  `KeyAlgorithm`, signing and encryption algorithms alike) and maps only the
  signing subset onto `Algorithm`; `guest_jwt::jwk_algorithms` refuses a key
  naming a non-signing `alg` (RSA-OAEP, ...) instead of treating it as
  alg-less. `decode_without_verify` moves off the removed
  `insecure_disable_signature_validation` onto `dangerous::insecure_decode`.
- Workspace lock: only jsonwebtoken and `pem 1.1.1` (removed) change; the
  rust_crypto tree was already present. `ring 0.16.20` stays in
  backend/Cargo.lock because `gcp_auth 0.9.0` holds it.
- windmill-parser-wasm lock: jsonwebtoken 8.3.0 -> 10.4.0 drops `ring 0.16.20`
  (jsonwebtoken was its only holder there), `spin 0.5.2`, `untrusted 0.7.1`,
  `base64 0.13.1`; `pem 1.1.1 -> 3.0.6` and `serde_json 1.0.143 -> 1.0.151`
  were forced by the new jsonwebtoken.
- ee-repo-ref.txt -> 514eb5f5 (windmill-ee-private chore/ee-dep-bumps, on top
  of the previous ref d252afcc), which reads the JWK algorithm through
  `windmill_common::jwt::jwk_algorithm` and builds proto `KeyValue`s with
  `..Default::default()`.

The opentelemetry 0.32 bump is deferred: tracing-opentelemetry 0.33 removed
`OtelData`/`PreSampledTracer`, which the EE `otel_ee.rs` log bridge uses, and
`otel` ships in every EE image (`ee_core`); that bridge needs a
`Dispatch`-based rewrite first.

Checks (SQLX_OFFLINE, EE files from 514eb5f5): cargo check --features
all_sqlx_features; --features all_sqlx_features,private; --features
enterprise,private,otel; cargo test -p windmill-common --lib jwt (22 passed).

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

* fix(jwt): keep audience-bearing tokens and 8192-bit RSA keys working on jsonwebtoken 10

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

* chore: update ee-repo-ref to 6f3161192ab0eba36a8630b92e6765a36a3326fc

This commit updates the EE repository reference after PR #809 was merged in windmill-ee-private.

Previous ee-repo-ref: 7dcc5741c2fa9f9997b3da085f1f18ffc1e446d6

New ee-repo-ref: 6f3161192ab0eba36a8630b92e6765a36a3326fc

Automated by sync-ee-ref workflow.

---------

Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
Co-authored-by: windmill-internal-app[bot] <windmill-internal-app[bot]@users.noreply.github.com>
Co-authored-by: Ruben Fiszel <ruben@windmill.dev>
2026-09-21 14:36:03 +00:00
Ruben Fiszelandrubenfiszel 01b38320ea chore(main): release 1.815.0 (#11212)
* chore(main): release 1.815.0

* Apply automatic changes

---------

Co-authored-by: rubenfiszel <275584+rubenfiszel@users.noreply.github.com>
2026-09-18 15:28:25 +02:00
Ruben Fiszelandrubenfiszel d440ebb520 chore(main): release 1.814.0 (#11171)
* chore(main): release 1.814.0

* Apply automatic changes

---------

Co-authored-by: rubenfiszel <275584+rubenfiszel@users.noreply.github.com>
2026-09-17 20:10:04 +02:00
Ruben Fiszelandrubenfiszel c3e11bd223 chore(main): release 1.813.0 (#11141)
* chore(main): release 1.813.0

* Apply automatic changes

---------

Co-authored-by: rubenfiszel <275584+rubenfiszel@users.noreply.github.com>
2026-09-16 12:12:22 +02:00
Ruben Fiszelandrubenfiszel 9fe493311d chore(main): release 1.812.0 (#11111)
* chore(main): release 1.812.0

* Apply automatic changes

---------

Co-authored-by: rubenfiszel <275584+rubenfiszel@users.noreply.github.com>
2026-09-15 11:40:01 +02:00
Ruben Fiszelandrubenfiszel 7b89e37322 chore(main): release 1.811.1 (#11107)
* chore(main): release 1.811.1

* Apply automatic changes

---------

Co-authored-by: rubenfiszel <275584+rubenfiszel@users.noreply.github.com>
2026-09-13 18:40:23 +00:00
Ruben Fiszelandrubenfiszel 62d4632fad chore(main): release 1.811.0 (#11098)
* chore(main): release 1.811.0

* Apply automatic changes

---------

Co-authored-by: rubenfiszel <275584+rubenfiszel@users.noreply.github.com>
2026-09-12 10:06:30 +02:00
Ruben Fiszelandrubenfiszel 4a293cf77a chore(main): release 1.810.0 (#11079)
* chore(main): release 1.810.0

* Apply automatic changes

---------

Co-authored-by: rubenfiszel <275584+rubenfiszel@users.noreply.github.com>
2026-09-11 19:02:12 +02:00
Ruben FiszelandClaude Opus 5 b156778da2 fix: support gzip and zstd compression for OTLP export over gRPC (#11077)
Claude-Session: https://claude.ai/code/session_01RbLzVfFDZ9pjBGHSzCSrNZ

Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-11 08:01:54 +02:00
Ruben Fiszelandrubenfiszel 7915540f66 chore(main): release 1.809.0 (#11045)
* chore(main): release 1.809.0

* Apply automatic changes

---------

Co-authored-by: rubenfiszel <275584+rubenfiszel@users.noreply.github.com>
2026-09-10 22:57:04 +02:00
Ruben Fiszelandrubenfiszel 133e21080a chore(main): release 1.808.0 (#11043)
* chore(main): release 1.808.0

* Apply automatic changes

---------

Co-authored-by: rubenfiszel <275584+rubenfiszel@users.noreply.github.com>
2026-09-09 12:35:50 +00:00
Ruben Fiszelandrubenfiszel 22c1a106cf chore(main): release 1.807.0 (#11034)
* chore(main): release 1.807.0

* Apply automatic changes

---------

Co-authored-by: rubenfiszel <275584+rubenfiszel@users.noreply.github.com>
2026-09-09 10:04:17 +00:00
Ruben Fiszelandrubenfiszel 63c40f046b chore(main): release 1.806.0 (#11012)
* chore(main): release 1.806.0

* Apply automatic changes

---------

Co-authored-by: rubenfiszel <275584+rubenfiszel@users.noreply.github.com>
2026-09-08 18:04:06 +02:00
Ruben Fiszel 2ae8509b14 fix: reduce php parser stack use in debug workers (#11025)
* fix: reduce php parser stack use in debug workers

* test: document php stack regression expression depth
2026-09-08 14:33:47 +02:00
Ruben FiszelandClaude Opus 5 0139467b01 feat: ingest dbt column lineage and real column schemas from the engine's parquet index (#10977)
* feat: column-level lineage for dbt from the engine's parquet index

`manifest.json` carries no column-to-column edges, which is why decision 14
recorded column lineage as unavailable. The edges live in a different artifact:
`dbt compile --static-analysis strict --write-index` writes `target/index/`,
whose `dbt.column_lineage.parquet` holds them and whose
`dbt.node_columns.parquet` holds every column of every node, typed and ordered
rather than only the ones an author documented.

Strict analysis rejects SQL the default accepts, so this is a separate compile
with its own `--target-path`, opt-in per project via `column_lineage: true`, and
best-effort throughout: a project it cannot analyze keeps exactly the graph it
had, with the engine's own diagnostics in the job log.

Storage mirrors `dbt_edge`: `dbt_column_edge` keyed by (path, version, job) with
the same composite FK to `script` and the same sweeps. The typed column list
lands in `dbt_node.column_schema`, beside `columns` rather than merged into it,
so `columns` stays what the author declared.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PRtsPQ3Ck69Fu9DNr7bMcJ

* fix: address review findings on the dbt column-lineage pass

- The workspace fork copied every other dbt sidecar table and not this one, so
  a fork lost its column lineage silently and could not recover it: the cloned
  digest covers the column edges, so a dynamic run in the fork matched it and
  stored nothing.
- The parquet was collected whole before the edge cap applied, which is exactly
  the input the cap exists for — a project whose `scan` lineage is quadratic in
  its widest model could take the worker process down. Decoded a row at a time
  with the bound enforced during the decode.
- The pass swallowed every error from the runner, including the job poller's
  cancellation and deadline, so a run that blew its timeout inside an optional
  annotation could still publish a graph and report success. `run_captured`
  now carries the exit status in its value, so only a failed COMPILE is
  downgraded, and the pass may spend at most half the remaining wall clock so
  it cannot starve the build that follows it.
- `scan` edges are stored but no longer served: they are most of a project's
  lineage, nothing renders them, and the graph endpoint is polled by the run
  page. They are also the first thing the storage cap gives up now, rather than
  evicting the direct edges the trace draws.
- `column_schema` and the column edges take the same gate as the model's SQL. A
  column-level view is the shape of what the author wrote, one level finer than
  the `ref()` graph, which is ungated only because it draws relations the
  caller already sees.
- `graph_digest` hashes the new section only when it has edges, so a project
  that never asked for the pass keeps the digest it has instead of
  re-snapshotting on every dynamic run until it is redeployed.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PRtsPQ3Ck69Fu9DNr7bMcJ

* fix: the editor buffer's column lineage, and three bounds that were wrong

Round-2 review found four defects, all of them introduced by the round-1 fixes.

- The `script_visible` gate on the column edges was copied from the node query
  without its `script_hash IS NULL` arm. `= NULL` is never true, so every
  version-less row was filtered out and an editor buffer's parse rendered its
  typed columns and none of their lineage — the one place the feature is meant
  to be used. Pinned by an assertion in `dbt_pinned_graph.rs`, which is where
  this class of bug already had a home.
- The phase budget was handed to the poller, whose expiry is an `Err`
  indistinguishable from a cancellation or the job's own deadline, so a slow
  but valid analysis aborted the build it exists to annotate. The runner gets
  the full deadline again — those two must still fail the job — and the budget
  is a race around the whole pass, where expiring is this budget and nothing
  else.
- The decode cap counted parquet ROWS, so `scan` and out-of-graph rows could
  spend it before a single drawn edge was read. It now counts what is kept,
  takes direct kinds in a first pass, and is handed the graph's own nodes so
  the budget cannot go on rows that could never be stored.
- Hashing the new digest section conditionally did not preserve old digests,
  because an absent `column_schema` still serialized as `null` inside the
  nodes. It is skipped when absent instead.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PRtsPQ3Ck69Fu9DNr7bMcJ

* refactor: split the lineage pass by error contract, and read it in one query

Round 3's findings were all consequences of round 1 and 2's fixes, clustered in
the same two files, so this reshapes those two seams rather than patching again.

The worker pass was one function being three things at once — a subprocess
runner with job-lifecycle error semantics, a bounded decoder, and a best-effort
degrader — which is why each fix to one perturbed another. It is now
`compile_index`, which owns the JOB's semantics (only a cancellation or the
job's deadline can `Err`; a non-zero exit, the output ceiling and the phase
budget are outcomes), and `read_index`, which owns the ARTIFACT's and knows
nothing about the job. The budget wraps the compile alone, so a decode can no
longer outlive the timeout that reported the build would get the rest. The
output ceiling likewise becomes a value rather than a job error, for the caller
that can carry on without the tail of a compile's stdout.

The column edges were read by a fourth hand-written copy of the `live`/`chosen`
CTEs and the version/editor-buffer join conditions, and copying them is what
dropped the `script_hash IS NULL` arm and hid every buffer parse's lineage. Both
kinds of edge now come from ONE statement over a `UNION ALL`'d edge source, so
those conditions exist once. The union is at the source rather than a join
because column lineage can name a node pair `dbt_edge` has no row for: a model
reading `{{ this }}` gets edges from itself to itself, and `parent_map` has no
self-loop.

The cap on the column half now sits after the scope filter, the visibility
check and the graph joins — the scope moved into SQL via the existing
`ScopePathFilter` — so a row the caller may not read can no longer spend it and
leave an allowed project's trace short.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PRtsPQ3Ck69Fu9DNr7bMcJ

* refactor: serve dbt column lineage from its own endpoint

The column edges rode on the folder-wide asset graph, which a run page polls,
while the trace is drawn for one selected relation. That needed a cap, and a cap
has to be applied after every filter that can drop a row.

Keyed to the asset there is no cap: `assets/column_lineage` answers for one
relation, and the caller's `scripts:read` scope and the project's visibility are
decided once, for the script that owns it. Pinning to a run's snapshot or the
editor's parse of its buffer costs the job-read gate, so that form is
`jobs/dbt_column_lineage/{id}` — the same shape `jobs/dbt_graph/{id}` has.

The worker's decode now bounds work and memory separately, and a compile stopped
by the output ceiling reports as truncated rather than complete.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* fix: resolve the owning dbt version the way the graph does

The unpinned arm picked the newest live version at the path without narrowing to
dbt, so a path since redeployed in another language answered with no lineage
while the graph beside it still drew that project's stale nodes.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* fix: gate pinned column lineage on reading the project, and answer the component

Four things round 5 found, three of them in code this branch rewrote:

- The pinned arm resolved the version from the job and stopped there, so a
  share-link viewer entitled to a run got the project's column names and edges
  while the graph beside it still redacted `raw_code` and `column_schema`.
  Resolving WHICH version answers is not deciding whether the caller may read
  it; the version-less editor buffer keeps its exemption, having no `script` row
  to ask.
- The answer was the whole owning project's edges. The canvas lays out the
  connected component of the selected relation's columns, so the rest was
  unrenderable weight; a recursive walk over both directions returns exactly
  what is drawn, and the project key travels with it so a `unique_id` two
  projects share cannot walk from one graph into the other.
- The decode had no exit but the 4M-row backstop once its buckets were full,
  spending wall clock the build below does not get.
- An unreadable index was reported as a missing one, sending the reader to look
  at their engine rather than at the file.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* fix: stitch the two column graphs, and walk the component in Rust

Round 6's two findings, both regressions this branch introduced:

- The decode returned `Continue` on the edge that FILLED the direct-edge
  budget, so a `scan`-only tail after it decoded to the 4M-row backstop with
  nowhere to put anything. The read now ends on that edge.
- Seam 3 made the pipeline page choose between the dbt graph and the producer
  one. They share node ids — `// column total <- dbt://wh/analytics/orders.amount`
  mints the same `(dbt, path, column)` node dbt's own lineage does — so choosing
  ended a trace at the boundary in both directions. They are merged again, and
  a ducklake selection asks about the dbt relation its producers name so the
  chain continues past it. The dbt editor gets the same merge.

Also: the component is walked in Rust rather than by a recursive CTE. A CTE has
no index, so the recursive term rescanned the doubled edge set once per level —
1243ms against 59ms for the query alone on a 3000-model project, 11.7M rows in
the plan. Same answers, same tests; end to end 1.48s to 0.73s there and 1.60s to
0.26s on a 1000-deep chain. The client stops re-asking for a component it
already holds, which is most clicks within one project.

The four doc sites that described a whole-project answer are rewritten around
what it now is, rather than edited where they disagreed.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* fix: expand every dbt boundary a selection reaches, and only skip what was asked

Round 7's findings, all in the frontend seam this branch added:

- A ducklake selection seeded the dbt fetch from the FIRST boundary relation it
  found, so a table derived from two unconnected dbt relations expanded one and
  left the other a leaf — the same "stops at the boundary" symptom the round-6
  fix removed, one hop further along. Every distinct boundary is fetched now and
  the components merged.
- The component cache skipped a relation merely PRESENT in the graph in hand.
  A relation two projects describe has an owner row in each, and a component
  fetched for one carries it as an endpoint without the other's half, so that
  skipped the request that would have resolved the second owner. Only a relation
  actually asked about under this pin is skipped.
- A comment still called the producer graph gated to ducklake selections after
  it was widened to dbt.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* refactor: land dbt column lineage as storage and ingest only

The API surface that draws a column trace moves to a follow-up PR, on
`dbt-column-lineage-surface`. It kept generating findings — a client cache
whose premise was wrong for a two-owner relation, then staleness and a lost
retry from tightening it, and a seed walk that stopped at the first boundary —
and the fix for the last of them is a transitive owner expansion, which has to
re-apply the caller's gate to every newly discovered project. That is the same
shape as the leak four reviewers caught in the pinned arm, and it wants its own
review rather than being the fourth fix at the end of this one.

What lands here stands on its own: the analysis pass, `dbt_column_edge`,
`dbt_node.column_schema`, the engine gating and the error-contract split — plus
the one user-visible half, the typed and ordered column list, which rides the
asset graph the details pane already fetches and replaces a panel that could
only show the columns an author had documented.

Also fixes a real bug in the pass, found in review: it compiled without the
build's `--full-refresh`. `is_incremental()` branches on that flag, so an
incremental model reading `{{ this }}` compiles its self-join — and any `ref()`
inside that branch — only when the flag is absent, and the pass was storing
lineage for SQL a full-refresh run never executed. The flag now comes from one
place shared with the build, and a run that overrides it gets its own graph
rather than standing as the version's.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* docs: say why direct kinds get the budget without naming a view

The bucketing comments explained the priority by what a trace draws, which is
a forward reference now that the surface moved out. The reason stands on its
own: `copy`/`mod` say the value travelled, `scan` says the column was read to
produce the row and so reaches every output column of its model.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* fix: round-9 findings on the descoped PR

- The `full_refresh` helper was inserted between `selection_is_overridden` and
  its doc comment, so thirteen lines about `select`/`exclude` echoes documented
  the wrong function and the one they were written for had none. Moved below it.
- The parse path ran the analysis compile and the parquet decode BEFORE the
  guard that returns when there is no warehouse identity, paying for both and
  dropping the result. Moved after it.
- Three sites still described a `/column_lineage` endpoint this branch no longer
  has, and two user-facing strings promised a column trace it no longer renders:
  the panel's hint and the descriptor template now say what the flag actually
  buys, which is the typed column schema.
- Dropped test scaffolding the removed suite left behind: a `raw_orders` node
  and `dbt_edge` whose only assertion re-tested pre-existing graph behaviour,
  and a second editor-buffer node nothing asserts on.

Documented rather than fixed: an incremental model has two shapes, and which one
the index holds depends on whether the target existed when the pass ran.
`is_incremental()` is false with no target as well as under `--full-refresh`, and
dbt has no mode that emits both — so a version's graph describes the compile that
produced it, and only a re-ingesting run describes its own run.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* fix: keep lineage_kind in the edge key, and one answer for --full-refresh

- Both unique indexes omitted `lineage_kind`, so a column that is projected AND
  used as a predicate for the same output column — an ordinary shape — had its
  `copy` and `scan` edges collapse under `ON CONFLICT DO NOTHING`, while the
  digest counted both. The kind is part of the fact, so it is part of the key.
  Edited in the migration rather than added as a second one: it has not landed.
- `full_refresh` was shared between the build and the analysis pass without the
  `command != "test"` condition that sat at the build's call site, so the two
  disagreed for exactly the runs that build nothing. The condition moved inside
  the function, which is the point of sharing it, and the command is threaded to
  the pass.
- The "what a trace draws" rewrite missed the copy in `dbt_manifest.rs`.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* fix: drop the unreachable full_refresh threading, test the uniqueness key

`DBT_COMMANDS` is `["build", "retry", "show", "parse"]` and `default_command`
returns `build` in every arm, so `command == "test"` cannot happen — the guard
the last commit moved into `full_refresh` was already inert where it came from.
Threading the command through five signatures to preserve it bought nothing, so
it is gone; the build and the pass call one function of the descriptor and the
invocation, which is what the sharing was for.

The uniqueness-key fix now has a test: a column projected AND used as a
predicate for the same output column stores both its `copy` and its `scan` row.
Verified against the old key, where it returns 1 instead of 2.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* fix: restore the dbt test --full-refresh guard I removed on a wrong premise

The previous commit removed it after reading `DBT_COMMANDS` and concluding
`"test"` was unreachable. That is only true of the command a CALLER can name:
`run_dbt` is invoked with `"test"` directly for the `after_all` test phase, so
an `after_all` project with `full_refresh: true` reached it — and dbt rejects
`--full-refresh` on `test`, failing the phase. Both reviewers caught it.

The guard is back inside the shared function, where the build and the pass get
one answer, and its doc now records why reading the allowlist alone is
misleading. The test covering the `test` case is restored with it.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* fix: notice a job that ended during the decode, and name truncation as the cause

- The parquet decode runs on a blocking thread with no poller watching it, so a
  cancellation or an expired deadline during it was invisible: `dbt_dep` went on
  to publish the graph and the job returned success. The job's state is checked
  once the decode returns, before the caller publishes anything, and an ended
  job `Err`s — which this module may always do for the job's own semantics.
- A compile stopped by the output ceiling could leave no artifact, and the log
  then blamed the engine's capability, sending the reader to check their adapter
  rather than the ceiling. Truncation now names itself in the missing and
  unreadable branches too.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* fix: read cancellation from the DB after the decode, not from a poller's field

`ctx.canceled_by` is only ever written by a poller, and no poller runs during
the blocking decode — which is the exact window the check was added for. So the
guard caught only a cancellation already observed before it, and the comment
beside it claimed more than it did. It now queries `v2_job_queue` directly, the
same probe `worker_lockfiles` uses before it overwrites a flow.

A failed probe answers "still running": this decides whether to discard work
already done, so an unreachable database must not be the reason a healthy deploy
loses its graph.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* refactor: reuse job_is_canceled rather than a second copy of it

The probe added last round was `job_is_canceled` from the same file, retyped —
same query, same `Connection::Http` behaviour. Reused instead.

Its doc said a non-database connection was "a failed probe", which reads as an
error path. It is not: it is the agent worker, and on one there is no database
to ask, so only the deadline answers and a cancel issued during the decode is
not observable. The retry path avoids that by refusing to run on an agent worker
at all — which an optional annotation has no business doing — so the gap is
recorded at both ends instead.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* fix: close the agent-worker cancellation gap instead of documenting it

The previous commit said a cancel issued during the decode is not observable on
an agent worker. It is: `ping_job_status` returns `canceled_by` over both
connection kinds, and is how the poller itself notices one there. So the check
asks through the ping rather than querying `v2_job_queue` directly, and holds on
an agent worker, where a direct query reaches no database at all.

`job_is_canceled` goes back to private and its doc to what it said before — the
retry that calls it still refuses to run on an agent worker for its own reasons.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* fix: decode the index under the job poller instead of checking after it

Two findings with one cause: the decode was the only phase of this pass with no
subprocess behind it, so nothing heartbeated while it ran. A large index left
the worker silent for as long as it took, which the zombie sweep reads as a dead
job and restarts — and the cancellation check bolted on afterwards could only
ever report what had already happened, while dropping the ping's
`already_completed`, so a force-cancelled deploy still published its graph.

Running it under `run_future_with_polling_update_job_poller` answers all of it:
the poller pings throughout, and ends the phase with an `Err` on cancellation,
`AlreadyCompleted` or the phase timeout. The bespoke probe is gone with it.

Verified on a live deploy: 32 edges and 4 typed schemas ingested through the
polled decode.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* fix: stop a cancelled decode, and say what the read phase can now do

Putting the decode under the poller heartbeats it and ends the phase when the
job does, but dropping a `JoinHandle` detaches a blocking task rather than
cancelling it — so a cancelled job left a thread decoding up to four million
rows for a job that was over. The row loop reads an abandonment flag that a drop
guard on the awaiting future sets, so the decode stops at its next row.

That same change made the read phase able to `Err`, and three places still said
it could not — decision 14 in as many words. The distinction that holds is
narrower: nothing the ARTIFACT does or fails to do can fail a job, so absent,
unreadable and partial are all values; the JOB can still end the phase the read
runs in. Stated that way in the module doc, the `Artifact` doc, `MAX_INDEX_ROWS`
and the decision.

Verified on a live deploy: 32 edges and 4 typed schemas.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* refactor: share AbortOnDrop, and stop citing a hazard that is now handled

`Abandon` was `ansible_executor`'s `AbortOnDrop` retyped — same struct, same
reason, same `spawn_blocking` shape. Moved to `common` and used from both.

The paragraph explaining why the phase budget wraps the compile alone gave as
its reason "a decode still running on a blocking thread", which is exactly what
the abandonment flag now prevents. The reason that survives is the one that was
always the point: the budget exists to leave the build its share of the clock,
and only the compile can spend that share unboundedly. The decode's end is the
job's, through the poller it runs under.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* fix: put both doc comments back on the items they describe

Moving AbortOnDrop orphaned a doc at each end: it landed between
`raw_to_string`'s doc and `raw_to_string`, and the doc of the struct it replaced
stayed behind to prefix `fetch_repo_archive`.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* docs: name the binding the row loop actually reads

`Abandoned` was neither the type nor the binding; the flag is `abandoned`.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-08 09:58:38 +02:00
Ruben Fiszelandrubenfiszel a9d42b489f chore(main): release 1.805.0 (#10995)
* chore(main): release 1.805.0

* Apply automatic changes

---------

Co-authored-by: rubenfiszel <275584+rubenfiszel@users.noreply.github.com>
2026-09-07 18:20:28 +00:00
Ruben Fiszelandrubenfiszel c37f59e22a chore(main): release 1.804.0 (#10963)
* chore(main): release 1.804.0

* Apply automatic changes

---------

Co-authored-by: rubenfiszel <275584+rubenfiszel@users.noreply.github.com>
2026-09-05 11:11:54 +00:00
8aab5034a6 feat: guest JWT entry for embedded apps (#10954)
* feat: guest JWT entry for embedded apps (jwt_guest_)

A second way in for a guest, alongside the signed-in guest session: a JWT the
embedding customer's backend mints and signs, verified per request against a
per-workspace key (a PEM public key or a JWKS URL), resolving to the same
seatless guest identity confined to the one app its app_path claim names.
Bearer prefix jwt_guest_, stateless (no token row). See PR #10954.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

* feat: surface guest JWT as the embed method in the app deploy drawer

The deploy drawer explained the secret-URL embed but not the guest JWT path, so
the primary way to embed an app for a customer's own authenticated users was
undiscoverable. For a guest-mode app with guests enabled, show how to mint a
`jwt_guest_` token and append `guest.<jwt>` to the app URL, with a copyable
iframe template pre-filled with this app's workspace_id and app_path, and a note
that new guest emails are refused past the instance's free allowance (the live
count is shown just above).

Also log a guest JWT allowance refusal at warn, not info: the caller gets a bare
401 (the reason must not leak to an unauthenticated caller), so the log is the
admin's signal that the instance hit its guest cap.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

* fix: correct the guest JWT minting instructions in the embed block

The block said "sign it with the workspace's guest JWT key", but that setting
holds the public verification key. Clarify the keypair relationship (configure
the public key or a JWKS URL in the workspace; sign with the matching private
key), name the accepted algorithms (RS/PS/ES; HS* refused), and keep the
required claims, so an embedder knows how to actually mint the token.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

* feat: fall back to the instance JWT issuer for guest verification (off on cloud)

A workspace with no guest key of its own now verifies guest JWTs against the
instance issuer (JWT_EXT_JWKS_URL, already used by jwt_ext_), so an operator
running one issuer configures it once. Verification and the guest grant are CE;
granting a full login from that issuer stays EE (jwt_ext_, unchanged). Disabled
under CLOUD_HOSTED, where one instance issuer must not be trusted to mint guests
in every tenant's workspace — there the per-workspace key is the only source,
which also stays the override everywhere. The workspace settings note (hidden on
cloud) explains the fallback.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

* fix: embed instructions cover both the workspace key and instance issuer

The embed block said to set the workspace's guest JWT key; now it says Windmill
verifies against the workspace key or, off cloud, the instance issuer
(JWT_EXT_JWKS_URL) when no workspace key is set. The instance clause is hidden
under isCloudHosted().

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

* fix: show the guest JWT embed block only when Embed is toggled

It belongs with the iframe snippet, not the plain-URL view, so gate it on
embedMode alongside the guest-mode / guests-enabled checks.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

* fix: trust the instance issuer in the guest fallback; refresh stale docs

P1 (CI review): the fallback wrapped JWT_EXT_JWKS_URL as a workspace JwksUrl, so
it hit validate_guest_jwks_url and was refused for http/private issuers unless
ALLOW_PRIVATE_GUEST_JWKS_URLS was also set — a self-hosted internal issuer that
works for jwt_ext_ failed for guests, though the UI says setting the env var is
enough. fetch_jwks now fetches the instance issuer without the https/private
restriction (matching the jwt_ext_ loader; it stays operator-trusted), while a
workspace-admin URL is validated and pinned as before. All the size/key/URL
bounds still apply to both.

P2 (CI review): refresh the stale docs that said a missing workspace key always
refuses a guest JWT — the module, bearer, key-source, and EditGuestJwtKey field
docs now describe the workspace key with the off-cloud instance-issuer fallback.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

* fix: fetch the trusted instance issuer like the jwt_ext_ loader

P1 (CI review): the instance-issuer fetch skipped SSRF validation but still
disabled redirects and default cert validation, so an instance issuer that works
for jwt_ext_ through a redirect or an operator-approved self-signed cert failed
the guest fallback. Fetch it with HTTP_CLIENT_PERMISSIVE (follows redirects,
honors ACCEPT_INVALID_CERTS) — the same behavior jwt_ext_ has — while a
workspace-admin URL stays validated, DNS-pinned and redirect-free. The body size
cap still bounds both.

P2 (CI review): the WorkspaceSettings field doc still said None/None means no JWT
guests; it now names the off-cloud instance-issuer fallback.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

* docs: schema summary + OpenAPI cover the guest JWT columns and fallback

P2 (CI review): summarized_schema.txt was missing guest_activity.jwt_entry and
the two workspace_settings guest-JWT key columns (required by docs/validation.md
after a schema change). The edit_guest_jwt_key OpenAPI description now notes that
clearing the workspace key falls back to the instance issuer (JWT_EXT_JWKS_URL)
off cloud rather than necessarily stopping guest JWTs.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

* fix: keep JWKS single-flight locks in a self-cleaning map, not a bounded cache

P1 (CI review): JWKS_FETCH_LOCKS was a 200-entry quick_cache. Past 200 cold URLs
it can evict a lock whose fetch is still in flight; the next request for that URL
then mints a fresh lock and starts a second fetch, so cycling configured
workspaces defeats single-flight and can storm the issuers. Replace it with a
plain map guarded by a JwksFetchLock RAII handle that removes each entry once its
last holder drops, so the map only ever holds the fetches in flight and never
evicts an in-flight lock. Add a unit test pinning the shared-lock and
self-cleaning invariants.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

* chore: update ee-repo-ref to c2270eb5fe2d9f0968253e6b460c33186363f4e7

This commit updates the EE repository reference after PR #773 was merged in windmill-ee-private.

Previous ee-repo-ref: 5a1d9dee34159512c0823fddcd3d096490edbcce

New ee-repo-ref: c2270eb5fe2d9f0968253e6b460c33186363f4e7

Automated by sync-ee-ref workflow.

---------

Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
Co-authored-by: windmill-internal-app[bot] <windmill-internal-app[bot]@users.noreply.github.com>
2026-09-05 10:23:37 +00:00
fce635d3c4 feat: guest app execution mode, a role that takes no seat (#10929)
* feat: guest app execution mode, a fourth role that takes no seat

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BayTppRCstWX6qTf3LMco5

* fix: make the guest grant a server-minted label, not a declarable scope

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BayTppRCstWX6qTf3LMco5

* chore: pin ee-repo-ref to the guest session companion branch

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BayTppRCstWX6qTf3LMco5

* fix: close the relabel hole, guest embed tokens, read-path switch, custom-path entry

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BayTppRCstWX6qTf3LMco5

* fix: guest tokens are not rescopable and guest embed tokens keep the sentinel

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BayTppRCstWX6qTf3LMco5

* fix: guest-derived tokens share one constraint set; gate sign-in on guest discovery

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BayTppRCstWX6qTf3LMco5

* fix: the label alone governs a guest; refuse guests with accounts; unserialize discovery

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BayTppRCstWX6qTf3LMco5

* fix: guest discovery fails closed; SAML aborts if the guest cookie write fails

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BayTppRCstWX6qTf3LMco5

* refactor: enforce the guest switch once at the auth door; sign-in for a guest of another app

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BayTppRCstWX6qTf3LMco5

* fix: guest app-mode decided once at the on-behalf resolver; clear a stale guest session before offering another app's sign-in

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BayTppRCstWX6qTf3LMco5

* fix: a guest may use anonymous apps; await the stale-session logout; trim comments

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BayTppRCstWX6qTf3LMco5

* fix: a guest's path confinement waits for the app's mode, so anonymous apps stay open to it

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BayTppRCstWX6qTf3LMco5

* fix: guest target survives http (Lax cookie), rides SAML RelayState; tell account holders on arrival

* fix: a guest uses an anonymous app as itself; S3 uploads confined by app mode

* fix: a guest upload needs an app policy; a missing app does not skip the confinement

* fix: guests are gated on the Enterprise plan server-side; pin ee-repo-ref

* fix: the guest plan gate fails closed on non-enterprise builds; settings report the effective switch

* fix: guest controls read the plan, not the key; gate the guest tests on the features they need

* docs: tighten the guest session invariant comments

* feat: 100 free guests per 30 days, then a quarter seat each on Enterprise and a hard cap elsewhere; superadmin guest list; refusals reach the page

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BayTppRCstWX6qTf3LMco5

* fix: the cap is exact, an account ends a guest session at the door, popups close, and guest mode survives the CLI round trip

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BayTppRCstWX6qTf3LMco5

* feat: a superadmin switch over guests for the whole instance; the pre-existing-user flag keeps its meaning

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BayTppRCstWX6qTf3LMco5

* fix: drop the dead guest-access helper, name the instance setting once, guests tab states, CE save order

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BayTppRCstWX6qTf3LMco5

* fix: a guest app path is refused at the mint if it could widen the scope; the instance toggle waits for its reload

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BayTppRCstWX6qTf3LMco5

* fix: guests stop at the launched-by-me job grant; canonical app paths at the mint and discovery; the toggle ends on the stored value

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BayTppRCstWX6qTf3LMco5

* fix: only the scope grammar's own characters bar an app path from guests, refused at deploy as well as at the mint

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BayTppRCstWX6qTf3LMco5

* fix: the deploy-time guest path guard checks the destination of a rename and refuses a leading slash

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BayTppRCstWX6qTf3LMco5

* fix: a workspace rename keeps the guest switch; the rename guard reads the deployed mode under the row lock

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BayTppRCstWX6qTf3LMco5

* fix: guest_activity follows a workspace rename and goes with a workspace delete

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BayTppRCstWX6qTf3LMco5

* chore: pin ee-repo-ref to the state-bound guest target

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BayTppRCstWX6qTf3LMco5

* chore: pin ee-repo-ref; the guest cookie is never cleared by a callback

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BayTppRCstWX6qTf3LMco5

* docs: the workspace-scoped guest_activity delete moves an instance-wide count; assert the mint records the guest

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BayTppRCstWX6qTf3LMco5

* test: the seeded allowance is a day old, so only the mint can write today's guest_activity row

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BayTppRCstWX6qTf3LMco5

* chore: update ee-repo-ref to 1a10132e4f3cb442c7d0c2cf6e5d92d150bf6e07

This commit updates the EE repository reference after PR #769 was merged in windmill-ee-private.

Previous ee-repo-ref: 32841072aa396bff91d30bd91854fa348cb3c439

New ee-repo-ref: 1a10132e4f3cb442c7d0c2cf6e5d92d150bf6e07

Automated by sync-ee-ref workflow.

---------

Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
Co-authored-by: windmill-internal-app[bot] <windmill-internal-app[bot]@users.noreply.github.com>
2026-09-04 22:47:28 +02:00
Ruben Fiszelandrubenfiszel 38fc0d3a12 chore(main): release 1.803.0 (#10952)
* chore(main): release 1.803.0

* Apply automatic changes

---------

Co-authored-by: rubenfiszel <275584+rubenfiszel@users.noreply.github.com>
2026-09-03 13:08:33 +02:00
Ruben Fiszelandrubenfiszel 74c1813f98 chore(main): release 1.801.0 (#10921)
* chore(main): release 1.801.0

* Apply automatic changes

---------

Co-authored-by: rubenfiszel <275584+rubenfiszel@users.noreply.github.com>
2026-09-02 01:01:01 +02:00
Ruben Fiszelandrubenfiszel 870f67121d chore(main): release 1.800.1 (#10910)
* chore(main): release 1.800.1

* Apply automatic changes

---------

Co-authored-by: rubenfiszel <275584+rubenfiszel@users.noreply.github.com>
2026-09-01 11:34:32 +02:00