Commit Graph
70 Commits
Author SHA1 Message Date
8741843d2e feat: external instance cluster for data tables and Ducklake catalogs (#11197)
* fix pg_dump stuck on version 17 on nix

* fix(datatables): refuse a malformed role annotation instead of ignoring it

`-- Role operator`, `-- role operator;` and `-- role operator -- why` all failed
the annotation parser's exact-match rule, so the query fell through to the data
table's default role and ran, silently, under a login the author did not choose.
Naming a role exists precisely to not do that.

A leading comment whose first word is `role` is now an annotation attempt: the
keyword matches case-insensitively, one trailing `;` is tolerated, and anything
else is an error naming the line. Only callers that already know the target is a
`datatable://` reference ever run this, so ordinary SQL keeps its comments.

Also bumps the dev shell's postgres client to 18 — it trailed the server the dev
database runs, which takes out every data table export, clone and fork-with-data.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012ti5HyeTikPMYyW8YSdiHR

* fix(datatables): refuse a malformed role query string instead of ignoring it

`?Role=analytics`, `?role=` and `?x=1&role=…` all fell through the reference
parser's exact-match rule, so the connection resolved to the data table's default
role and ran under a login the caller never asked for — the URI half of the same
trap as a malformed `-- role` annotation.

The key now matches case-insensitively, and anything else in the query string is
an error naming it; `role` is the only parameter a reference takes. Callers that
only need the entry keep a lenient `datatable_ref_name`, since they never act on
the role. The DuckDB `ATTACH` parser propagates it rather than attaching under
the default.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012ti5HyeTikPMYyW8YSdiHR

* fix(datatables): carry the role annotation into the row_to_json retry

The retry rebuilds its SQL from `pruneComments(code)`, so the leading comment
block never reached the second attempt — and with it the `-- role <name>` line
that decides which login the query runs as. The retry connected as the data
table's default role instead, so a query the first attempt was denied could
succeed on the second, reported as "recovered with the row_to_json fix".

Carry the leading comment block over. The retry itself is unchanged.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BjfMkJyKzodxkobqGZ6Lqb

* chore(datatables): don't mount the roles UI until the ACL editor lands

Enforcement ships first. The permissions drawer is what turns roles on, and the
catalog section is what creates them — both are only useful once there is a way
to grant a role the privileges it needs, which arrives with the ACL editor. Left
mounted they would offer a feature whose other half does not exist.

The two components are complete and reviewed; only their call sites here are
commented out, with a note pointing the follow-up PRs at them.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BjfMkJyKzodxkobqGZ6Lqb

* fix(datatables): honour `-- role: x`, and fix the DuckDB attach test

Two review findings, both real.

`attach_datatable_parses_name_and_role` never compiled: `parse_attach_datatable`
returns `Result<Option<_>>` now and one call site kept a single `unwrap`. Its
`?Role=analytics` case also asserted a refusal, contradicting the parser in the
same commit, which matches the key case-insensitively. Replaced with the cases
that are genuinely malformed, and a positive one for the cased key.

`-- role: analytics` fell through to the default role — the silent fallback the
strict parser exists to remove, for the spelling most likely to be typed. The
keyword now accepts an optional colon, attached or spaced, while a word that
merely starts with it (`rolebased`) is still not an attempt.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BjfMkJyKzodxkobqGZ6Lqb

* fix(datatables): clone a fork's pointer instead of failing after the copy

Forking a fork with cloning left an orphan database. The preflight resolves the
pointer and sees the governing entry, so both endpoints ran and filled the new
database; `apply_forked_datatable` then refused the inherited pointer and rolled
the fork back, stranding a registered `wm_fork_*` that no entry names and whose
name blocks the retry.

Refusing earlier would have been the smaller change, but forking a fork and
cloning worked before pointers existed, so it would trade an orphan for a
regression. Resolve what the pointer names and write the terminal entry the
clone needs: the whole `database` object rather than a patch of its
`resource_path`, since a pointer has none, and `reference` removed with it.

Also accepts `-- role=x` and `-- Role = x`, two more spellings that fell through
to the default role.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BjfMkJyKzodxkobqGZ6Lqb

* fix(datatables): refuse to roll back the catalog while roles exist

The down migration dropped the table and left every role behind: live Postgres
logins whose passwords only that table carried, so after a revert Windmill could
neither use, disable nor delete them, and re-applying could not recreate them
because the names were taken. Cleaning up here is not possible either — dropping
a role means reassigning what it owns in every instance database, and a
migration runs in one — so it now refuses while the catalog is non-empty and
says to delete the roles through instance settings, which does the cluster work.

Also enforces the instance-only invariant the resolved-pointer clone relies on
rather than only asserting it in a comment.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BjfMkJyKzodxkobqGZ6Lqb

* refactor(datatables): settle clonability in one place, before anything is created

A clone is three stages a workspace apart — `create_pg_database`, then
`import_pg_database`, then `apply_forked_datatable` inside the fork transaction.
Only the third can roll back, and `CREATE DATABASE` is not transactional, so any
refusal that lives there strands a registered `wm_fork_*` that no entry names
and whose name blocks the retry.

That orphan has now been fixed three times, most recently reintroduced by a
guard added one commit ago. Patching each new refusal into the first endpoint is
not the fix; having two places that can refuse is. `ensure_datatable_is_clonable`
now answers every reason a copy can be refused and returns what it resolved, and
the stage that writes the entry only does the work.

Also takes an ACCESS EXCLUSIVE lock before the rollback guard counts, so a role
created concurrently cannot slip between the check and the drop.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BjfMkJyKzodxkobqGZ6Lqb

* fix(datatables): let a retried clone reclaim its own leftover database

A clone creates its target database one request before it copies into it, and
the fork that would name it is written a request after that. Any failure in
between — a pg_dump error, a bad restore, a dropped connection, the source's
roles changing mid-flow — left a registered `wm_fork_*` that no entry names,
and every retry then failed on its name. This predates data table roles.

`create_pg_database` now reclaims such a leftover before creating: only a
`wm_fork_*` database Windmill registered as a data table database and that no
data table or ducklake entry names, in any workspace, archived ones included.
The drop never terminates connections, so a clone still copying into it makes
the reclaim fail instead of being cut off. It is limited to callers who
administer the source — reaching it is not enough, since on a data table
without roles every member reaches it — and anyone else gets the refusal an
existing database always got.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* Revert "fix(datatables): let a retried clone reclaim its own leftover database"

This reverts commit 7dd3275a10.

The reclaim tied the caller to the source they administer, but not to the
database it dropped. Between another workspace's import and its final fork
request, that workspace's target is full, registered, unnamed and has no open
connection, so an admin of any instance data table could name it and have it
dropped and recreated empty. The victim's fork would then commit pointing at
the empty copy. Safe reclaim needs durable clone ownership and serialization
with the request that names the database; until then the leftover stays, as it
did before this PR.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* docs(datatables): record the stale clone database as a known limitation

A clone is three requests and `CREATE DATABASE` is not transactional, so a
failure after the first leaves a registered `wm_fork_*` behind, as it did
before data table roles. Accepted for this PR: it is harmless to data and goes
away once the clone is a single server-side operation.

The comment also records why the obvious fix is wrong: reclaiming the leftover
on retry, without durable clone ownership, can drop another workspace's fully
copied database between its import and its final fork request.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* fix(datatables): bounce the streams reading a data table when it is deleted

Deleting a governing data table, or the workspace that holds it, only collected
the fork pointers it stranded, for the warning. A Postgres trigger or capture
already streaming through one of those pointers kept the replication connection
it opened while the pointer still resolved, so it went on dispatching the
governing database's rows after the fork lost access — until its connection
happened to restart. The governing workspace's own streams on a deleted entry
did the same.

Both deletion paths now bounce the affected listeners inside their own
transaction, through the helper a permission change already uses, so a
listener that reconnects re-resolves the entry and finds it gone. The helper is
split so a caller can pass the (workspace, local name) pairs it already holds.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* fix(datatables): keep the fork schema baseline, and bounce streams on every removal

Three fixes from review.

`edit_datatable_config` took `forked_from` wholesale from the stored entry, so
the fork schema diff's save of an advanced baseline was silently discarded and
an applied change was offered again. Whether an entry carries a clone stamp is
still carried from the store, since that is what marks its database droppable,
but the baseline inside it is now taken from the request.

The stranded-pointer warning and the stream bounce ran over the optional
`deleted_datatables` hint, which the settings-sync CLI never sends, so removing
a governing data table through `wmill` bounced nothing. Removals are now derived
from the stored configuration against the saved one.

`delete_workspace` read the pointers to bounce before its transaction, so a fork
committing a pointer during the deletion was missed. The read now happens inside
the transaction, after the workspace row is deleted: a fork's insert key-share
locks that row through its parent foreign key, so it is either seen or fails on
the missing parent.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* refactor(datatables): keep Postgres triggers and data table roles apart

A replication stream reads every row of every table whatever the data table's
roles grant, and its listener checks access only when it connects. Rather than
chase every way access can change and bounce the streams each one affects, a
data table now carries one or the other:

- a Postgres trigger or capture cannot be created on, or connect to, a data
  table under roles;
- roles cannot be turned on while an enabled trigger or a live capture reads
  the data table, its own or a fork's through its pointer. The refusal names
  each one to disable.

This removes the stream bounces on roles edits and on data table and workspace
deletion, and the trigger gate that admitted admins. The fork schema baseline
fix from the same review round is kept.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* fix(datatables): refuse a Postgres trigger on a data table under roles when it is saved

Creating or editing a trigger that points at a data table under roles was
accepted, and its listener then retried the refused connection every 30
seconds forever. The save is now refused, and a trigger that reaches such a
data table anyway (re-enabled, or cloned into a fork) is disabled by its
listener with the reason, as a missing replication slot is.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* fix(datatables): disable a data table role before deleting it

Deleting a role reassigns and drops what it owns in each registered
database on its own connection, and each of those passes commits as it
goes. A database failing part-way left the role enabled in the catalog and
able to log in, but already stripped in the databases reached before it.
The role is now disabled in its own commit first, so a failed delete
leaves a disabled role to retry.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* fix(datatables): serialize roles going on with a stream starting

Turning roles on looked for enabled triggers and live captures once,
without a lock anything starting a stream also took. A trigger enabled in
that window could have its listener connect before roles committed, and a
healthy listener never checks again. Both transitions now serialize on one
advisory lock: roles going on hold it exclusive while they look, and
trigger create, edit and enable, and capture setup and ping hold it shared
while they commit. Either the look sees the stream, or the listener
connects after roles are committed and refuses.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* fix(datatables): wait out live listeners, and resolve stored names containing `?`

Turning roles on counted a trigger as gone once disabled, and a capture
once its client stopped pinging, but the listener keeps its replication
connection until its next heartbeat notices. A trigger or capture whose
listener pinged in the last 15 seconds, the window a server holds a
listener for, now still counts as streaming.

Data table names could contain `?` before they were restricted, and such
entries are still stored. Splitting `?role=` off a reference misread them:
`a?b` became `a` with an unknown parameter, and the clone checks looked at
a different entry than the one copied. An entry stored under the whole
reference is now looked up first, in the Postgres executor, DuckDB ATTACH
and the clone checks. Agent workers cannot read the workspace and keep
the strict parse, which refuses such a name rather than misreading it.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* fix(datatables): warn when a settings sync strands fork pointers

A settings save reported the fork pointers left resolving to nothing only
for the names in `deleted_datatables`, which `wmill sync push` never sends.
The save now works out what it removed from the locked entries, and the
CLI prints the stranded pointers it returns.

Also correct the replication helper's contract: no role or admin check
makes a replication connection safe, so a data table under roles is
refused outright rather than gated as an admin operation.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BjfMkJyKzodxkobqGZ6Lqb

* fix(datatables): refuse a save that drops a data table's roles through an undeclared rename

A data table's roles follow its entry only through a declared rename. A
settings sync sends the whole map and never declares one, so renaming a
data table under roles there read as a delete and a new entry on the same
database: the new entry carried no roles, and every caller connected as
admin. Such a save is now refused, naming both entries.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BjfMkJyKzodxkobqGZ6Lqb

* fix(datatables): no entry without roles may newly reach a database under roles

The previous guard only caught a new name replacing an entry under roles.
A whole-map save could also repoint an existing entry without roles at
that database, or another workspace could point one there, and every
caller of that entry would connect as admin. The rule is now stated on
the saved entries: one that carries no roles and newly points at an
instance database any entry under roles uses, in this workspace or
another, is refused. A declared rename carries its roles and passes.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BjfMkJyKzodxkobqGZ6Lqb

* feat(datatables): move data table role catalog and resolution to the enterprise edition

Roles are an Enterprise Edition feature. The catalog, the Postgres logins,
CONNECT convergence, tenant evaluation and the role half of connection
resolution move to windmill-ee-private. Every public function keeps its path
and signature and forwards through datatable_roles_oss, which re-exports the
enterprise implementation or, without it, refuses.

Without the enterprise edition a data table under roles, or a caller naming a
role, is refused a connection rather than resolved as admin, and the reach and
admin-access checks refuse one under roles. A data table not under roles
resolves as before in every edition, and an instance database keeps the
CONNECT grants it was created with. The catalog lock, the stream lock, the
tenant cascades and the permissions stripping stay in OSS: they only restrict.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BjfMkJyKzodxkobqGZ6Lqb

* feat(datatables): move the data table permissions endpoints to the enterprise edition

The permissions read, save and usable-roles handlers move to
windmill-ee-private; the routes stay registered and, without the enterprise
edition, answer that data table roles are an Enterprise Edition feature.
ensure_governs_datatable and ensure_reaches_datatable keep their paths: the
first refuses, the second passes a data table not under roles.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BjfMkJyKzodxkobqGZ6Lqb

* feat(datatables): move the data table role catalog endpoints to the enterprise edition

The superadmin list, create, update and delete handlers move to
windmill-ee-private. The routes stay registered and, without the enterprise
edition, refuse after authentication.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BjfMkJyKzodxkobqGZ6Lqb

* test(datatables): run the roles tests on the enterprise edition, refusals without it

Each test that exercises roles runs with private and enterprise. Two tests run
without them: every roles route answers the Enterprise refusal, and a data
table saved under roles, or a named role, is refused a connection while one
not under roles resolves as before.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BjfMkJyKzodxkobqGZ6Lqb

* feat(datatables): gate the roles UI mount sites on an enterprise license

Both mount sites are still commented out; the gate travels with them.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BjfMkJyKzodxkobqGZ6Lqb

* test(datatables): run the tenant matcher test on the enterprise edition

The matcher it covers is enterprise code now, so without the enterprise
edition the test hit the stub and failed the default windmill-common run. It
runs with private and enterprise, and a counterpart without them asserts that
no tenant list covers anyone, the wildcard and a workspace admin included.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BjfMkJyKzodxkobqGZ6Lqb

* chore: update ee-repo-ref to a1873dbb67f2302b85ff5362f8387b48eccdb607

This commit updates the EE repository reference after PR #783 was merged in windmill-ee-private.

Previous ee-repo-ref: 5c853e2c20eca6b748415fc0d6862a6ebfb5fec4

New ee-repo-ref: a1873dbb67f2302b85ff5362f8387b48eccdb607

Automated by sync-ee-ref workflow.

* fix(datatables): refuse roles while a same-workspace alias reaches the database

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* feat(datatables): add an ACL editor for data table roles

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* feat(datatables): data table roles in the DB manager and raw apps

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* fix: never add a role to the reference of a data table whose name contains '?'

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* fix: read the roles of a data table whose name contains '?'

The generated client leaves a '?' in a path param unencoded, so the lookup
404'd and the raw-app picker blocked Start on such a data table.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* fix: take every pooled connection before the ACL apply locks

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BRoYE5ZeAVvrDYdfhDAYXb

* fix: refresh grant options only after the ACL apply validates its plan

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BRoYE5ZeAVvrDYdfhDAYXb

* fix(datatables): refuse a reference naming both a legacy data table and a role

When a workspace stores both `sales` and a legacy `sales?role=analytics`, the
reference resolved to the legacy entry without a role, so browsing `sales` as
`analytics` reached another data table.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* fix: declare the default role in migrations written for a data table whose name contains '?'

Such a data table connects as its default role without naming it, so the
migrations the manager wrote for it declared no role and ran as admin.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* fix(datatables): let CE migrations connect as an explicitly named admin

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* fix: add only missing grant options before an ACL apply, never default privileges

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BRoYE5ZeAVvrDYdfhDAYXb

* fix(datatables): serialize roles going on with aliases saved from other workspaces

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* docs(datatables): note that legacy names with ? cannot be migrated

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* fix: run one data table ACL apply at a time per server before it connects

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BRoYE5ZeAVvrDYdfhDAYXb

* feat(datatables): set up an external instance cluster for data tables

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* fix(datatables): send external cluster passwords as SCRAM verifiers

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* fix(datatables): scope external cluster credential readers to the crate

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* [ee] feat(datatables): external_instance data tables on the external cluster

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* fix: hold the ACL connection to the database that was authorized

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BRoYE5ZeAVvrDYdfhDAYXb

* fix(datatables): compare the external cluster settings under a row lock before storing setup

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* fix(datatables): only drop external databases Windmill marked, and check use under the lock

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* fix: build the ACL connection from the authorized data table entry

Resolving the settings again could land on a resource with the same
database name on another server, which the later entry checks never see.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BRoYE5ZeAVvrDYdfhDAYXb

* fix: check ACL read reach against the entry it connects from

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BRoYE5ZeAVvrDYdfhDAYXb

* feat(datatables): put a data table's connection under Postgres roles

A data table backed by the instance database resolved to exactly one Postgres connection,
`custom_instance_user`, for everyone who could reach it at all. There was no way to say
this job reads, that one writes, this one never sees the salaries table.

A data table role is now a real Postgres login on the cluster, defined once for the
instance by a superadmin and named exactly as they named it. A script that declares
`-- role analytics` connects as `analytics`, and Postgres decides what it may touch —
grants are ordinary SQL. Windmill answers only "may this caller ask for this role", from
the tenant lists on the data table entry: `u/alice`, `g/analysts`, `f/finance` or `*`.
A data table with no `permissions` block behaves exactly as before.

Everything that opens a connection on someone's behalf goes through one chokepoint,
`get_datatable_resource_from_db`, which takes the identity explicitly and fails closed when
there is none. The role logs in as itself — never `SET ROLE`, which a script could
`RESET ROLE` its way out of.

A fork's data table entry becomes a pointer at the workspace that governs it rather than a
copy of it. The settings clone used to hand a fork a byte-identical entry naming the
parent's database, which a fork admin could edit to grant themselves `admin` there; a
pointer has nothing local to edit, and its tenants are evaluated as a member of the
governing workspace, by email. `permissions` is stripped from the workspace export and
ignored on import: tenants name principals of one workspace, and a settings push is not
where an access decision should be made.

Operations that see the whole database whatever the roles grant stay with the governing
workspace's admins: editing the roles, a migration that declares none, and opening a
replication stream for a Postgres trigger or capture.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012ti5HyeTikPMYyW8YSdiHR

* fix(datatables): gate the paths that reach a whole database as admin

Auditing what still resolved through the unchecked resolver turned up three that act for a
caller and hand back the admin connection: `resolve_pg_source_checked` (behind schema
export, the full-schema read, database creation, import and the forked-database drop), the
connection test, and the schema snapshot a fork clone takes of its parent. On a data table
under roles each let any workspace member — or a fork admin who is nobody in the governing
workspace — read or copy the whole database whatever its roles grant.

All three now require admin reach on the governing workspace. A dump taken under a
restricted role would be a silently truncated copy rather than an error, so refusing is the
only right answer for the copy paths.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012ti5HyeTikPMYyW8YSdiHR

* fix(datatables): confine roles to the instance database, and stop a fork reaching the parent's bookkeeping

A data table role is a login on Windmill's own Postgres. Nothing stopped a workspace admin
putting a *resource-backed* data table under roles, at which point the executor dialled the
host that resource names — one the admin chose — with the role's real cluster password, and
`CONNECT` is granted to every registered instance database. Both ends now refuse: the
permissions endpoint rejects the save, and the chokepoint refuses to substitute credentials
on a non-instance entry rather than trusting the record it read.

Two more places reached the governing database without answering to it. The initial-migration
generator returned a `pg_dump` of the whole schema to any member. And the migration
rename/delete cascade followed a fork's pointer into the parent, so a fork admin renaming or
removing their own local entry relabelled or wiped the parent's `_wm_migrations` — after
which the parent re-runs every migration from zero. The remote half is now skipped when the
entry resolves into another workspace, which is also just correct: a fork renaming what it
calls a data table changes nothing about the data table.

Also: revoking a tenant now bounces the replication streams of every workspace holding an
entry that resolves here, not only the governing one, so a fork's trigger stops rather than
living on inside its open connection; the instance role catalog and the governing workspace's
tenant lists are no longer returned to someone who cannot edit them; and the tenant rename
dedup collapses non-adjacent duplicates, per role rather than once any role changed.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012ti5HyeTikPMYyW8YSdiHR

* fix(datatables): fail loudly where a role or a pointer can be left half-recorded

Three ways the feature could end up in a state nobody could see or undo.

Creating a role writes the cluster first and the catalog second, but the catalog write was an
`UPDATE` that matched nothing when the instance Postgres settings row was absent — leaving a
live login with a password nobody recorded: invisible to the catalog, un-recreatable because
the name is taken, and un-deletable because there is no entry to delete. It now errors, so
the operation is retryable once the row is restored.

Deleting a workspace only nulls the fork lineage; the data table entries pointing at it are
left resolving to nothing. Sweeping them is not an option — turning a pointer back into a copy
would hand each fork the database outright — so the delete now names the data tables it
stranded, and resolving one says which workspace is missing rather than reporting a data table
this workspace never had.

`InstanceDatatableRole` derived `Debug` while holding a Postgres password; it is now
hand-written so `{:?}` on the catalog cannot put a live credential in a log line.

Adds the two branches the reviews found unpinned: a caller who is not a member of the
governing workspace at all, and `NoIdentity` — the compatibility path for an agent worker that
predates this and sends no job id.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012ti5HyeTikPMYyW8YSdiHR

* fix(datatables): unbreak two operator messages and two comments that described other code

The two strings this branch added for states an operator hits once — the catalog write that
matched nothing, and the delete that stranded a pointer — were collapsed from their multi-line
form with the indentation left in, so both rendered with a fourteen-space gap mid-sentence.

`list_datatables` claimed to report a chain it cannot follow and then dropped it; it does drop
it, and the comment now says why that is the right place to stay quiet. The non-superadmin
check in `edit_datatable_config` was introduced as also covering references, which it does not
and need not: `reference` is overwritten from the stored entry for every caller before the
check runs.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012ti5HyeTikPMYyW8YSdiHR

* fix(datatables): serialize role catalog mutations, and state each helper's authorization contract

The catalog is one JSON document, so create, rename, enable and delete are all
read-modify-write. Two concurrent creates read the same snapshot, both succeed in the
cluster, and the second write drops the first — leaving a live Postgres login with a password
nobody recorded, which is the exact state the delete path exists to prevent. Every mutation
now runs in one transaction holding an advisory lock across the read, the cluster DDL and the
write, so a lost update cannot happen and a failure rolls the whole thing back. The DDL
helpers take that transaction rather than the pool, which is what makes the lock cover them.

Their statements moved off `sqlx::raw_sql`: the simple protocol is only needed for genuinely
multi-statement SQL, and its future is not `Send`, which an axum handler holding the
transaction requires. Each of these is one statement anyway.

The new cross-crate surface now says what callers must do. `read_role_catalog` returns
plaintext credentials; `create`/`rename`/`set_login`/`drop_instance_role` and
`converge_connect_grants` mutate cluster-wide state; `read_datatable_entry` reads a workspace's
raw config. All of them are superadmin-gated by their current handlers, but nothing said so at
the definition, which is where the next caller looks.

Also: the roles table reloads after a failed login toggle instead of leaving it claiming a flip
that did not land; the rename affordance is the design-system `Button`, not a raw one; and
`resolve_datatable_pg_as_caller` drops a `role` parameter no caller ever filled — browsing
resolves as the data table's default until the database manager grows a picker.

Why role passwords stay a plain `String` while the instance user's password beside them is a
`StringOrSecretRef`, asked three times across reviews: that one is a secret ref because an
operator supplies it and may want it from their own backend, while these are minted here and
never entered by anyone, so there is nothing for a ref to point at. Encrypting generated
secrets at rest is a separate change that would take the replication password with it. Now
said at the field.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012ti5HyeTikPMYyW8YSdiHR

* fix(datatables): give the role catalog its own row, out of reach of the config machinery

Putting it inside `custom_instance_pg_databases` was the wrong call, and it cost two ways.
The catalog serializes a generated Postgres password per role, and that row is the
operator-facing instance config, so the passwords reached `get_instance_config` and its YAML
editor — a live cluster credential in a response body, a UI field and any log of either.
Worse in the other direction: `to_settings_map` strips the catalog, so a full-row upsert of
that key writes the row back without it and the catalog is gone, while the cluster keeps every
login it described.

`custom_instance_replication_pwd` is the precedent and says exactly why — a generated secret,
written only by the server, never operator-authored, hidden so the config machinery cannot
read, rewrite or drop it. The catalog is the same thing, so it now has the same shape:
`datatable_roles`, in `HIDDEN_SETTINGS`, `PROTECTED_SETTINGS` and the agent-worker denylist.
No redaction to keep in step with three code paths, and no way for a neighbouring write to
take it out.

Two races on the same shared documents. `edit_datatable_config` read the stored data tables
outside its transaction and then wrote the whole `datatable` document, so a permissions save
committing in between was silently rolled back; it now reads under `FOR UPDATE`. And
`set_datatable_permissions` validated role ids against the catalog before opening its
transaction, so a deletion in between let it write a deleted role back — including as the
default, which every later job then fails on; it now holds the catalog lock and the settings
row across validation and write.

Completes the authorization contracts the previous commit claimed but did not finish:
`read_datatable_entry` (which it named and missed), `resolve_governing_datatable`, whose whole
job is to answer for a workspace the caller may not belong to, and
`converge_connect_grants_with`, which had not inherited its wrapper's.

Also the generic Python SDK reference: `_format_py_params` learned the bare `*` last time, but
`extract_py_functions` is a second formatter and still rendered `datatable(name, role)`, so
code written from that page passed a keyword-only argument positionally.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012ti5HyeTikPMYyW8YSdiHR

* fix(datatables): make the concurrency test pin the handlers, and the contracts describe what is enforced

The concurrency test reimplemented the read-modify-write inline, so deleting the lock from all
three handlers left it green — it pinned Postgres, not the code it was written for. It now
drives `create_datatable_role` twice concurrently and asserts the catalog kept both names.
Checked the way the last one should have been: removing the lock from the handler makes it
fail with "wmtest_a_… is a live cluster login the catalog forgot".

The contracts added last commit were stricter than this PR's own callers, which is worse than
none — the next reader sees a rule already broken and learns to ignore it.
`read_role_catalog` said superadmin-only while two of its four callers are open to any
workspace member, and `converge_connect_grants` said superadmin while
`set_datatable_permissions` reaches it as a workspace admin. Both were fine on substance: the
rule that actually holds is about the credential never reaching a response, log, audit record
or export, not about who may call. They now say that. `read_datatable_entry` gets the same
treatment rather than the one the earlier message claimed for it: it is the primitive every
resolution goes through, so it is deliberately open, and what must not escape is `permissions`
— it names the governing workspace's users, groups and folders.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012ti5HyeTikPMYyW8YSdiHR

* fix(datatables): close the last ways a role or a pointer can be left pointing at nothing

The raw settings readers hand back whatever is in the row, so moving the catalog into its own
`global_settings` key protected the config machinery and left `GET /settings/global/datatable_roles`
and the settings listing returning every live password. Both now filter that one key. The
neighbouring `custom_instance_replication_pwd` has the same shape and is not touched here: it
predates this and widening the fix to it is a decision about an operator workflow, not a
consequence of this change.

Three ways a save could leave something resolving to nothing:

A permissioned data table could be moved to a PostgreSQL resource. The block was carried across
as a server-owned field, the runtime refuses roles on a resource-backed table, so the save
succeeded and every job afterwards failed. Refused instead — turning roles off first is one step,
and it keeps discarding an access decision something somebody chose.

Renaming a governing data table left every fork pointing at the old name: the data table
disappears from their pickers and their jobs stop, with nothing in the renaming workspace to
suggest why. The rename now follows into the pointers in the same transaction.

Deleting one cannot be followed the same way, so it is reported instead — the response names what
it stranded, the way deleting a workspace does, and the fork's own error already says which
workspace is gone.

Also: `ensure_instance_db_grant_options_unchecked` claimed superadmin while the permissions
handler reaches it as a workspace admin (the same class fixed last commit, one instance missed);
the role entry kept an `instance_config_schema` derive it no longer needs; `write_role_catalog`
was the one writer of that table not stamping `updated_at`; and the concurrency test dropped its
roles only on success — a failing run is exactly the one that creates them without recording them.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012ti5HyeTikPMYyW8YSdiHR

* refactor(datatables): put the role catalog in its own table, not in global_settings

Five findings across three rounds were all the same choice. A set of live Postgres credentials
was living in `global_settings`, which has generic read, list, write, config-export and CLI
round-trip paths that know nothing about what they carry: the passwords reached the instance
config and its YAML editor, a full-row upsert of a neighbouring key erased the catalog,
`GET /settings/global/{key}` and the settings listing returned them raw, and this round the
redaction that fixed the last two turned `wmill instance push` into something that wipes every
password — a fix breaking the assumption the previous fix made. `POST /settings/global/datatable_roles`
could also empty it outside the lock.

The approved plan offered a table or `global_settings`, so this is the other option it already
allowed rather than a new design. `datatable_role` is a table: no generic settings path can read
it, list it, export it, write it or round-trip it, so none of the five needs a guard. The
redaction, the hidden/protected/agent-denylist entries and the JSON document all go with it.

One row per role also removes the read-modify-write the concurrency work was about: two
concurrent creates are two inserts, and the unique index on `name` is what settles a collision.
The advisory lock stays for the one window rows do not cover — `CREATE ROLE` is invisible to
another transaction until commit, so without it both creates pass their `pg_roles` check.

Also from this round: rename mappings are checked against the configuration they claim to
describe, since fork pointers are rewritten from them — a caller could otherwise submit
`main -> missing` against an unchanged config and repoint every fork of `main` at a name nothing
has, and `A -> B` plus `B -> C` moved what pointed at `A` all the way to `C`. And the warning
naming forks a delete stranded reached the response but not the screen: both the data table
settings save and the workspace delete now show it.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012ti5HyeTikPMYyW8YSdiHR

* fix(datatables): validate a rename against the save it describes, and re-check under the locks

Three from the round, all about deciding on state that could already have moved.

A permission save resolved the data table and checked it was instance-backed before taking any
lock, then wrote under one. A config save committing in between could move the table onto a
PostgreSQL resource — recreating exactly what the transition guard refuses — or rename it, in
which case the write targeted a key that no longer existed and reported success having changed
nothing. It now re-resolves and re-checks on the locked state.

Rename validation checked that the source existed before and the target existed after, which
still accepts `main -> decoy` against a save that keeps both: every fork of `main` then follows
onto a different data table, silently, because it keeps resolving. The rule is now the actual
old-to-new key transition — a source may only survive if another rename took its name, and a
target may only pre-exist if another rename freed it. That also stops two sources sharing one
target, and it admits a swap, which the previous guard refused: `datatables` is keyed by name, so
a swap cannot be done one save at a time, and refusing it was a regression against main. The
pointer cascade now runs in two passes through a temporary name, the way the migration cascade
one layer down already handles the same shape, so `A -> B` with `B -> C` moves each pointer once
from what it named before the save.

The tenant mutators say what they are for: they write an access decision for any workspace named,
with an arbitrary mutation, and exist for the transaction that frees or renames a principal.
Editing a decision on purpose belongs in the permissions endpoint.

Carried in the same change: the stranded-fork list is a field rather than a phrase to grep out of
a success string; the pointer cascade matches with `EXISTS` instead of a `LIKE` over the whole
document, so a workspace whose pointers name something else is not rewritten to a byte-identical
value under an exclusive lock; and `InstanceDatatableRole` drops the serde derives left over from
the JSON document, one of which would emit `pwd`.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012ti5HyeTikPMYyW8YSdiHR

* fix(datatables): cascade on the leave route that is used, gate migrations before the admin connection, and drop a role atomically

The tenant cascade on leaving went onto `/users/leave`. The UI and the generated client call
`/workspaces/leave` — a different handler in a different crate with the same name — which
deleted the membership and left `u/<username>` in the tenant lists. Leaving and rejoining
therefore restored the access the leave was supposed to end, and a later account taking the
username would have inherited it. The regression test drives the route the client actually
calls; without the fix it fails with "leaving kept the tenant".

The migration endpoints authorized too late. `run_datatable_migrations` opened the data table's
admin connection, created `_wm_migrations` and read it before reaching the per-migration role
check — so with nothing pending, nothing was checked at all. Rollback returned before its check
when nothing was applied, and the status endpoint had none. All three now ask, before any
connection is opened, whether the caller can reach the data table as any role at all; which role
a given migration runs as is still decided per migration, and by the executor after that.

Deleting a role committed the cluster drop and the catalog row, then swept the tenant lists in
separate transactions. A sweep failing part-way left workspaces naming a role nothing can connect
as, while the retry answered `NotFound` because the catalog entry was already gone. The sweep now
runs in the same transaction, so the drop, the row and every tenant list commit together.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012ti5HyeTikPMYyW8YSdiHR

* fix(datatables): refuse to copy a data table that is under roles

pg_dump carries no roles and the import runs with --no-privileges, so a copied
data table arrives owned by the admin connection with no GRANT for any role.
The settings clone brings `permissions` across, so the fork's tenants pass
Windmill's check, connect as the role they were given, and are denied by
Postgres on everything: an entry that reads as configured and answers nothing.

Refuse the copy — in the import endpoint before any data moves, and in the fork
path the CLI takes. Replaying the source's owners and ACLs into the clone is
what lifts this, and is a change of its own. Dropping `permissions` from the
copy instead would be the unsafe half, since the copy holds the parent's rows.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012ti5HyeTikPMYyW8YSdiHR

* fix(datatables): refuse the clone's database too, not only its data

A clone is two endpoints: `create_pg_database` then `import_pg_database`. Only
the second refused a data table under roles, so a fork asking to clone one
created and registered an empty `wm_fork_…` instance database and then failed —
and nothing collects it, since `drop_forked_datatable_databases` only drops
entries carrying `forked_from` and no entry names this one.

Refuse in both, so the clone stops before a database exists.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012ti5HyeTikPMYyW8YSdiHR

* nit worker error msg

* fix pg_dump stuck on version 17 on nix

* fix(datatables): refuse a malformed role annotation instead of ignoring it

`-- Role operator`, `-- role operator;` and `-- role operator -- why` all failed
the annotation parser's exact-match rule, so the query fell through to the data
table's default role and ran, silently, under a login the author did not choose.
Naming a role exists precisely to not do that.

A leading comment whose first word is `role` is now an annotation attempt: the
keyword matches case-insensitively, one trailing `;` is tolerated, and anything
else is an error naming the line. Only callers that already know the target is a
`datatable://` reference ever run this, so ordinary SQL keeps its comments.

Also bumps the dev shell's postgres client to 18 — it trailed the server the dev
database runs, which takes out every data table export, clone and fork-with-data.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012ti5HyeTikPMYyW8YSdiHR

* fix(datatables): refuse a malformed role query string instead of ignoring it

`?Role=analytics`, `?role=` and `?x=1&role=…` all fell through the reference
parser's exact-match rule, so the connection resolved to the data table's default
role and ran under a login the caller never asked for — the URI half of the same
trap as a malformed `-- role` annotation.

The key now matches case-insensitively, and anything else in the query string is
an error naming it; `role` is the only parameter a reference takes. Callers that
only need the entry keep a lenient `datatable_ref_name`, since they never act on
the role. The DuckDB `ATTACH` parser propagates it rather than attaching under
the default.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012ti5HyeTikPMYyW8YSdiHR

* fix(datatables): carry the role annotation into the row_to_json retry

The retry rebuilds its SQL from `pruneComments(code)`, so the leading comment
block never reached the second attempt — and with it the `-- role <name>` line
that decides which login the query runs as. The retry connected as the data
table's default role instead, so a query the first attempt was denied could
succeed on the second, reported as "recovered with the row_to_json fix".

Carry the leading comment block over. The retry itself is unchanged.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BjfMkJyKzodxkobqGZ6Lqb

* chore(datatables): don't mount the roles UI until the ACL editor lands

Enforcement ships first. The permissions drawer is what turns roles on, and the
catalog section is what creates them — both are only useful once there is a way
to grant a role the privileges it needs, which arrives with the ACL editor. Left
mounted they would offer a feature whose other half does not exist.

The two components are complete and reviewed; only their call sites here are
commented out, with a note pointing the follow-up PRs at them.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BjfMkJyKzodxkobqGZ6Lqb

* fix(datatables): honour `-- role: x`, and fix the DuckDB attach test

Two review findings, both real.

`attach_datatable_parses_name_and_role` never compiled: `parse_attach_datatable`
returns `Result<Option<_>>` now and one call site kept a single `unwrap`. Its
`?Role=analytics` case also asserted a refusal, contradicting the parser in the
same commit, which matches the key case-insensitively. Replaced with the cases
that are genuinely malformed, and a positive one for the cased key.

`-- role: analytics` fell through to the default role — the silent fallback the
strict parser exists to remove, for the spelling most likely to be typed. The
keyword now accepts an optional colon, attached or spaced, while a word that
merely starts with it (`rolebased`) is still not an attempt.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BjfMkJyKzodxkobqGZ6Lqb

* fix(datatables): clone a fork's pointer instead of failing after the copy

Forking a fork with cloning left an orphan database. The preflight resolves the
pointer and sees the governing entry, so both endpoints ran and filled the new
database; `apply_forked_datatable` then refused the inherited pointer and rolled
the fork back, stranding a registered `wm_fork_*` that no entry names and whose
name blocks the retry.

Refusing earlier would have been the smaller change, but forking a fork and
cloning worked before pointers existed, so it would trade an orphan for a
regression. Resolve what the pointer names and write the terminal entry the
clone needs: the whole `database` object rather than a patch of its
`resource_path`, since a pointer has none, and `reference` removed with it.

Also accepts `-- role=x` and `-- Role = x`, two more spellings that fell through
to the default role.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BjfMkJyKzodxkobqGZ6Lqb

* fix(datatables): refuse to roll back the catalog while roles exist

The down migration dropped the table and left every role behind: live Postgres
logins whose passwords only that table carried, so after a revert Windmill could
neither use, disable nor delete them, and re-applying could not recreate them
because the names were taken. Cleaning up here is not possible either — dropping
a role means reassigning what it owns in every instance database, and a
migration runs in one — so it now refuses while the catalog is non-empty and
says to delete the roles through instance settings, which does the cluster work.

Also enforces the instance-only invariant the resolved-pointer clone relies on
rather than only asserting it in a comment.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BjfMkJyKzodxkobqGZ6Lqb

* refactor(datatables): settle clonability in one place, before anything is created

A clone is three stages a workspace apart — `create_pg_database`, then
`import_pg_database`, then `apply_forked_datatable` inside the fork transaction.
Only the third can roll back, and `CREATE DATABASE` is not transactional, so any
refusal that lives there strands a registered `wm_fork_*` that no entry names
and whose name blocks the retry.

That orphan has now been fixed three times, most recently reintroduced by a
guard added one commit ago. Patching each new refusal into the first endpoint is
not the fix; having two places that can refuse is. `ensure_datatable_is_clonable`
now answers every reason a copy can be refused and returns what it resolved, and
the stage that writes the entry only does the work.

Also takes an ACCESS EXCLUSIVE lock before the rollback guard counts, so a role
created concurrently cannot slip between the check and the drop.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BjfMkJyKzodxkobqGZ6Lqb

* fix(datatables): let a retried clone reclaim its own leftover database

A clone creates its target database one request before it copies into it, and
the fork that would name it is written a request after that. Any failure in
between — a pg_dump error, a bad restore, a dropped connection, the source's
roles changing mid-flow — left a registered `wm_fork_*` that no entry names,
and every retry then failed on its name. This predates data table roles.

`create_pg_database` now reclaims such a leftover before creating: only a
`wm_fork_*` database Windmill registered as a data table database and that no
data table or ducklake entry names, in any workspace, archived ones included.
The drop never terminates connections, so a clone still copying into it makes
the reclaim fail instead of being cut off. It is limited to callers who
administer the source — reaching it is not enough, since on a data table
without roles every member reaches it — and anyone else gets the refusal an
existing database always got.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* Revert "fix(datatables): let a retried clone reclaim its own leftover database"

This reverts commit 7dd3275a10.

The reclaim tied the caller to the source they administer, but not to the
database it dropped. Between another workspace's import and its final fork
request, that workspace's target is full, registered, unnamed and has no open
connection, so an admin of any instance data table could name it and have it
dropped and recreated empty. The victim's fork would then commit pointing at
the empty copy. Safe reclaim needs durable clone ownership and serialization
with the request that names the database; until then the leftover stays, as it
did before this PR.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* docs(datatables): record the stale clone database as a known limitation

A clone is three requests and `CREATE DATABASE` is not transactional, so a
failure after the first leaves a registered `wm_fork_*` behind, as it did
before data table roles. Accepted for this PR: it is harmless to data and goes
away once the clone is a single server-side operation.

The comment also records why the obvious fix is wrong: reclaiming the leftover
on retry, without durable clone ownership, can drop another workspace's fully
copied database between its import and its final fork request.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* fix(datatables): bounce the streams reading a data table when it is deleted

Deleting a governing data table, or the workspace that holds it, only collected
the fork pointers it stranded, for the warning. A Postgres trigger or capture
already streaming through one of those pointers kept the replication connection
it opened while the pointer still resolved, so it went on dispatching the
governing database's rows after the fork lost access — until its connection
happened to restart. The governing workspace's own streams on a deleted entry
did the same.

Both deletion paths now bounce the affected listeners inside their own
transaction, through the helper a permission change already uses, so a
listener that reconnects re-resolves the entry and finds it gone. The helper is
split so a caller can pass the (workspace, local name) pairs it already holds.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* fix(datatables): keep the fork schema baseline, and bounce streams on every removal

Three fixes from review.

`edit_datatable_config` took `forked_from` wholesale from the stored entry, so
the fork schema diff's save of an advanced baseline was silently discarded and
an applied change was offered again. Whether an entry carries a clone stamp is
still carried from the store, since that is what marks its database droppable,
but the baseline inside it is now taken from the request.

The stranded-pointer warning and the stream bounce ran over the optional
`deleted_datatables` hint, which the settings-sync CLI never sends, so removing
a governing data table through `wmill` bounced nothing. Removals are now derived
from the stored configuration against the saved one.

`delete_workspace` read the pointers to bounce before its transaction, so a fork
committing a pointer during the deletion was missed. The read now happens inside
the transaction, after the workspace row is deleted: a fork's insert key-share
locks that row through its parent foreign key, so it is either seen or fails on
the missing parent.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* refactor(datatables): keep Postgres triggers and data table roles apart

A replication stream reads every row of every table whatever the data table's
roles grant, and its listener checks access only when it connects. Rather than
chase every way access can change and bounce the streams each one affects, a
data table now carries one or the other:

- a Postgres trigger or capture cannot be created on, or connect to, a data
  table under roles;
- roles cannot be turned on while an enabled trigger or a live capture reads
  the data table, its own or a fork's through its pointer. The refusal names
  each one to disable.

This removes the stream bounces on roles edits and on data table and workspace
deletion, and the trigger gate that admitted admins. The fork schema baseline
fix from the same review round is kept.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* fix(datatables): refuse a Postgres trigger on a data table under roles when it is saved

Creating or editing a trigger that points at a data table under roles was
accepted, and its listener then retried the refused connection every 30
seconds forever. The save is now refused, and a trigger that reaches such a
data table anyway (re-enabled, or cloned into a fork) is disabled by its
listener with the reason, as a missing replication slot is.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* fix(datatables): disable a data table role before deleting it

Deleting a role reassigns and drops what it owns in each registered
database on its own connection, and each of those passes commits as it
goes. A database failing part-way left the role enabled in the catalog and
able to log in, but already stripped in the databases reached before it.
The role is now disabled in its own commit first, so a failed delete
leaves a disabled role to retry.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* fix(datatables): serialize roles going on with a stream starting

Turning roles on looked for enabled triggers and live captures once,
without a lock anything starting a stream also took. A trigger enabled in
that window could have its listener connect before roles committed, and a
healthy listener never checks again. Both transitions now serialize on one
advisory lock: roles going on hold it exclusive while they look, and
trigger create, edit and enable, and capture setup and ping hold it shared
while they commit. Either the look sees the stream, or the listener
connects after roles are committed and refuses.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* fix(datatables): wait out live listeners, and resolve stored names containing `?`

Turning roles on counted a trigger as gone once disabled, and a capture
once its client stopped pinging, but the listener keeps its replication
connection until its next heartbeat notices. A trigger or capture whose
listener pinged in the last 15 seconds, the window a server holds a
listener for, now still counts as streaming.

Data table names could contain `?` before they were restricted, and such
entries are still stored. Splitting `?role=` off a reference misread them:
`a?b` became `a` with an unknown parameter, and the clone checks looked at
a different entry than the one copied. An entry stored under the whole
reference is now looked up first, in the Postgres executor, DuckDB ATTACH
and the clone checks. Agent workers cannot read the workspace and keep
the strict parse, which refuses such a name rather than misreading it.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* fix(datatables): warn when a settings sync strands fork pointers

A settings save reported the fork pointers left resolving to nothing only
for the names in `deleted_datatables`, which `wmill sync push` never sends.
The save now works out what it removed from the locked entries, and the
CLI prints the stranded pointers it returns.

Also correct the replication helper's contract: no role or admin check
makes a replication connection safe, so a data table under roles is
refused outright rather than gated as an admin operation.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BjfMkJyKzodxkobqGZ6Lqb

* fix(datatables): refuse a save that drops a data table's roles through an undeclared rename

A data table's roles follow its entry only through a declared rename. A
settings sync sends the whole map and never declares one, so renaming a
data table under roles there read as a delete and a new entry on the same
database: the new entry carried no roles, and every caller connected as
admin. Such a save is now refused, naming both entries.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BjfMkJyKzodxkobqGZ6Lqb

* fix(datatables): no entry without roles may newly reach a database under roles

The previous guard only caught a new name replacing an entry under roles.
A whole-map save could also repoint an existing entry without roles at
that database, or another workspace could point one there, and every
caller of that entry would connect as admin. The rule is now stated on
the saved entries: one that carries no roles and newly points at an
instance database any entry under roles uses, in this workspace or
another, is refused. A declared rename carries its roles and passes.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BjfMkJyKzodxkobqGZ6Lqb

* feat(datatables): move data table role catalog and resolution to the enterprise edition

Roles are an Enterprise Edition feature. The catalog, the Postgres logins,
CONNECT convergence, tenant evaluation and the role half of connection
resolution move to windmill-ee-private. Every public function keeps its path
and signature and forwards through datatable_roles_oss, which re-exports the
enterprise implementation or, without it, refuses.

Without the enterprise edition a data table under roles, or a caller naming a
role, is refused a connection rather than resolved as admin, and the reach and
admin-access checks refuse one under roles. A data table not under roles
resolves as before in every edition, and an instance database keeps the
CONNECT grants it was created with. The catalog lock, the stream lock, the
tenant cascades and the permissions stripping stay in OSS: they only restrict.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BjfMkJyKzodxkobqGZ6Lqb

* feat(datatables): move the data table permissions endpoints to the enterprise edition

The permissions read, save and usable-roles handlers move to
windmill-ee-private; the routes stay registered and, without the enterprise
edition, answer that data table roles are an Enterprise Edition feature.
ensure_governs_datatable and ensure_reaches_datatable keep their paths: the
first refuses, the second passes a data table not under roles.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BjfMkJyKzodxkobqGZ6Lqb

* feat(datatables): move the data table role catalog endpoints to the enterprise edition

The superadmin list, create, update and delete handlers move to
windmill-ee-private. The routes stay registered and, without the enterprise
edition, refuse after authentication.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BjfMkJyKzodxkobqGZ6Lqb

* test(datatables): run the roles tests on the enterprise edition, refusals without it

Each test that exercises roles runs with private and enterprise. Two tests run
without them: every roles route answers the Enterprise refusal, and a data
table saved under roles, or a named role, is refused a connection while one
not under roles resolves as before.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BjfMkJyKzodxkobqGZ6Lqb

* feat(datatables): gate the roles UI mount sites on an enterprise license

Both mount sites are still commented out; the gate travels with them.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BjfMkJyKzodxkobqGZ6Lqb

* test(datatables): run the tenant matcher test on the enterprise edition

The matcher it covers is enterprise code now, so without the enterprise
edition the test hit the stub and failed the default windmill-common run. It
runs with private and enterprise, and a counterpart without them asserts that
no tenant list covers anyone, the wildcard and a workspace admin included.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BjfMkJyKzodxkobqGZ6Lqb

* chore: update ee-repo-ref to a1873dbb67f2302b85ff5362f8387b48eccdb607

This commit updates the EE repository reference after PR #783 was merged in windmill-ee-private.

Previous ee-repo-ref: 5c853e2c20eca6b748415fc0d6862a6ebfb5fec4

New ee-repo-ref: a1873dbb67f2302b85ff5362f8387b48eccdb607

Automated by sync-ee-ref workflow.

* fix(datatables): refuse roles while a same-workspace alias reaches the database

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* fix(datatables): let CE migrations connect as an explicitly named admin

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* fix(datatables): serialize roles going on with aliases saved from other workspaces

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* docs(datatables): note that legacy names with ? cannot be migrated

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* feat(datatables): add an ACL editor for data table roles

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* fix: take every pooled connection before the ACL apply locks

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BRoYE5ZeAVvrDYdfhDAYXb

* fix: refresh grant options only after the ACL apply validates its plan

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BRoYE5ZeAVvrDYdfhDAYXb

* fix: add only missing grant options before an ACL apply, never default privileges

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BRoYE5ZeAVvrDYdfhDAYXb

* fix: run one data table ACL apply at a time per server before it connects

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BRoYE5ZeAVvrDYdfhDAYXb

* fix: hold the ACL connection to the database that was authorized

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BRoYE5ZeAVvrDYdfhDAYXb

* fix: build the ACL connection from the authorized data table entry

Resolving the settings again could land on a resource with the same
database name on another server, which the later entry checks never see.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BRoYE5ZeAVvrDYdfhDAYXb

* fix: check ACL read reach against the entry it connects from

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BRoYE5ZeAVvrDYdfhDAYXb

* feat(datatables): Ducklake catalogs on the external instance cluster

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* fix(datatables): never grant CREATEROLE to custom_instance_user on the external cluster

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* docs(datatables): state the authorization contract of external database usage lookups

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* fix(datatables): drop a DuckDB data table secret once its ATTACH has used it

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* perf(datatables): resolve a workspace's data tables per pointer hop, not per entry

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* fix(datatables): hold the parent's settings while a fork points at its data tables

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* fix(datatables): refuse repointing the external cluster while it is in use, and keep verify-ca working for pg_dump

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* fix(datatables): protect external databases pending fork cleanup, and describe Ducklake usage in the API

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* feat(datatables): per-cluster data table role catalogs, with roles on the external instance cluster

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* fix(datatables): write the external cluster setting under the lifecycle lock, and check fork targets are registered

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* fix(datatables): register external fork catalogs under the lifecycle lock, and keep certificate verification in DuckDB attaches

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* fix(datatables): keep certificate verification when DuckDB attaches an external data table

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* fix(datatables): keep certificate verification when DuckDB attaches an external data table

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* fix(datatables): refuse fork cleanup of an external database another workspace uses

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* fix(datatables): refuse rolling back while external data tables are under roles, and type external_instance in the CLI

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* fix(datatables): stop counting storage-only fork cleanup rows as uses of an external database

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* fix(datatables): bind fork database copies to their workspace, and count every use before dropping one

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* fix(datatables): authenticate instance database setup before writing its status, and keep a fork reservation across it

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* fix(datatables): create external databases only on a cluster setup succeeded on, and document the registry reader

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* fix(datatables): keep only the most recently used DuckDB root certificate files

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* fix(datatables): migrate fork reservations on workspace rename, and lock the parent's data tables for the whole fork

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* fix(datatables): take the fork data table lock once, before the external cluster's

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* docs(datatables): describe the external instance cluster and how to run one locally

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* fix(datatables): keep fork reservations private, drop a cleaned-up entry with its database, and serialize cleanup with settings saves

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* fix(datatables): hold the fork lock across a fork import, and carry the reservation inside the setup write

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* fix(datatables): check the external cluster setting on its own transaction, and gate the registry probe

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* feat(datatables): add an ACL editor for data table roles

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* fix: take every pooled connection before the ACL apply locks

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BRoYE5ZeAVvrDYdfhDAYXb

* fix: refresh grant options only after the ACL apply validates its plan

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BRoYE5ZeAVvrDYdfhDAYXb

* fix: add only missing grant options before an ACL apply, never default privileges

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BRoYE5ZeAVvrDYdfhDAYXb

* fix: run one data table ACL apply at a time per server before it connects

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BRoYE5ZeAVvrDYdfhDAYXb

* fix: hold the ACL connection to the database that was authorized

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BRoYE5ZeAVvrDYdfhDAYXb

* fix: build the ACL connection from the authorized data table entry

Resolving the settings again could land on a resource with the same
database name on another server, which the later entry checks never see.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BRoYE5ZeAVvrDYdfhDAYXb

* fix: check ACL read reach against the entry it connects from

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BRoYE5ZeAVvrDYdfhDAYXb

* chore: update ee-repo-ref to 7e338e4dabf91689bfd7fb0333c6534040b17b59

This commit updates the EE repository reference after PR #787 was merged in windmill-ee-private.

Previous ee-repo-ref: 0edd40979cf36bfba59323f3f6a0811ae1369cf5

New ee-repo-ref: 7e338e4dabf91689bfd7fb0333c6534040b17b59

Automated by sync-ee-ref workflow.

* fix(datatables): keep DuckDB root certificate files in the job directory

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* chore: drop the unused json import from the settings crate

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* fix(datatables): drop the serde_json::json import left unused by the fork setup write

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* fix(datatables): accept external_instance data tables in the settings form type

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* fix(datatables): serialize fork database reservations, cleanup, setup and Ducklake saves on the database

- A fork copy is created and registered under its workspace's fork lock, and refused once the
  workspace is archived; a rename re-migrates reservations under that lock after archiving.
- Ducklake saves lock every instance database they newly name, as data table saves do.
- Instance database setup holds the database's lock until its entry is written.
- Fork import also holds the database's lock across the restore.
- Cleanup re-reads the entry under its locks before dropping anything.
- Non-superadmins no longer see fork copies reserved for workspaces they are not in.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* fix(datatables): state the authorization contract of the external cluster status and write check

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* fix(datatables): lock the fork copy at finalization, and order cleanup's registry write after its settings row

- Fork finalization holds the copy's database lock from its availability check to the commit.
- Cleanup removes the registry entry in its own transaction, in a task of its own, instead of on a
  second connection; a rename migrates reservations after its settings rewrites, so both take the
  settings rows before the registry.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* fix(datatables): hold the fork lock across a rename's settings copy

Fork cleanup of the old id could otherwise drop a copy the renamed workspace goes on using. Also
states the authorization contract of the instance database drop helpers.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* fix(datatables): refuse moving the external cluster to another host or port while it holds databases

Every settings writer goes through the same check as removal: the login, TLS and maintenance
database may still change, the cluster may not.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* feat(datatables): clone a data table under roles with its owners and grants (#11120)

Claude-Session: https://claude.ai/code/session_01UbrtwiYNfayrmqouBJHwGV

Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>

* fix: open the raw app data table drawer when the workspace has none

Selecting the first data table of an empty list passed undefined to the name
check, which threw instead of opening the drawer on no data table.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* fix(datatables): offer cloning a data table under roles where its grants can be replayed

The server clones such a data table and replays the source's owners and grants,
which only the Enterprise Edition does, so the fork wizard hid both clone
options everywhere instead of on a build that cannot replay them.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* docs: name the placeholder the empty raw app data drawer renders

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* fix(datatables): replace a job directory file at the DuckDB root certificate path instead of trusting it

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* test: pin the enterprise refusal the role pickers read as 'not under roles'

The server's sentence and the frontend's copy of it were coupled by nothing,
so rewording either one turned every role picker on a community build into a
failed lookup.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* fix(datatables): type Ducklake catalogs on the external instance cluster in the settings form

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* fix(datatables): read a roles answer only for the workspace it was asked in

A fork and its parent each have their own roles on a data table of the same
name, so an answer stamped with the name alone settled the role from the
workspace the editor was acting on before.

Also derive the AI table creation flag from the data replaced into the editor:
data naming no data table left the flag on from before.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* fix(datatables): keep an instance database a settings save is waiting to name

Cleanup for a database whose setup failed took the lock first, read no user,
and dropped it while a save blocked on that same lock was about to commit a
reference to it.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* fix(datatables): check the workspace stamp in the default database selector too

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* fix(datatables): read what a save racing instance-database cleanup committed

A transaction blocked on the lock may still roll back, so keeping the database
for it stranded one whose name then blocks every retry: it is let through and
its outcome read instead. The waiter query also matches this database's locks
only, since pg_locks spans the cluster.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* fix(datatables): tell a waiting request apart from the workspaces using a database

Both callers render what cleanup returns as the workspaces that keep the
database, so a waiting request's pid read as one of them. Each now words that
case itself, and the give-up comment names where the kept name actually goes.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* fix(datatables): take the cleanup lock on a connection the pool cannot reclaim

A session lock outlives the future holding it, so a cancellation between
taking it and releasing it handed a locked session back to the pool, where
every later settings save waits on it. Detached, the connection closes when it
is dropped and the server releases the lock with the session.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* fix(datatables): close the cleanup connection on drop instead of detaching it

Detaching released the pool permit while the session stayed alive, so
concurrent cleanups waiting on their locks could open as many connections as
they liked. Closing on drop covers the same cancellation and keeps them
counted.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* chore: update ee-repo-ref to 179d2454a063ee818a9387a1eafdd354a217a16c

This commit updates the EE repository reference after PR #802 was merged in windmill-ee-private.

Previous ee-repo-ref: c2e43f5b5ff753d339b70e232fd17b6ffecb054d

New ee-repo-ref: 179d2454a063ee818a9387a1eafdd354a217a16c

Automated by sync-ee-ref workflow.

* chore: update ee-repo-ref to fd5b8af748f2c985b13e18d9ea30894f3bd7e9a3

This commit updates the EE repository reference after PR #798 was merged in windmill-ee-private.

Previous ee-repo-ref: 3145e422d61d580f0a82804f075285c112879da0

New ee-repo-ref: fd5b8af748f2c985b13e18d9ea30894f3bd7e9a3

Automated by sync-ee-ref workflow.

* fix(datatables): classify a fork import's target from the resolution that built its connection

A second read could see the entry flipped to a resource and skip the reservation check while the
connection already built still reached the instance cluster.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* fix(datatables): run external database lifecycle on the caller's transaction, one lock order for fork cleanup

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* fix(datatables): check a fork copy's reservation on the locked transaction's connection

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* fix(datatables): make and drop fork copies of external data tables on the external cluster

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* fix(datatables): refuse a fork whose external copy something already names

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* fix(datatables): check an external fork copy's uses before removing its entry, so child fork pointers still count

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* fix(datatables): migrate external cluster fork reservations on workspace rename

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* feat(datatables): configure the external instance cluster and pick its databases from the UI

Instance settings gets an External Postgres tab: the cluster's connection, the setup run
with its report, and the databases Windmill created there. Data table and Ducklake settings
offer the external_instance kind, with a picker over those databases.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* fix(datatables): keep the external cluster form out of the instance settings the page bulk-saves

The form seeded its key into the settings store after the page snapshotted them, so merely
opening the tab made the page send a setting the validator refuses, failing an admin's
unrelated save. The form is local state now, and only the setup writes the key.

The database picker also treats a superadmin-only listing as authoritative: a workspace
admin, who cannot list, no longer sees a saved database as missing.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* fix(tests): app paths are validated, so a guest app cannot be renamed to one with a space

Path validation on apps landed after this test, which still expected a space to be accepted
and created its fixture on a ':' path. The scopable-path guard it exists for is kept by
planting that path on the row, which is now the only way an app can hold one.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* feat(datatables): one Managed Postgres tab, and a switch for Windmill's own database

The instance settings tab covers both substrates Windmill administers: the external cluster,
which can now be disabled once nothing sits on it, and Windmill's own database, which an
operator can turn off so the cluster is the only one a workspace may newly name. A save that
names it then refuses, as the create endpoint does.

Workspace settings name them the way an admin meets them: Postgres Resource, and Managed
instance, qualified as Internal or External only while both are on offer.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* feat(datatables): offer the external cluster in the add-data-table wizard, and give Ducklake its room back

The wizard gets the external cluster as a fourth substrate: pick a database it already holds
or name a new one, which the run creates before writing the entry, so Try again does not trip
on a database the last attempt made.

Ducklake's maintenance column moves into the settings popover next to the extra args, which
the wider catalog select had squeezed the name box out of.

A setting a component saves itself now also moves the page's baseline, or Discard would
restore what it replaced and a later save would send that stale value back.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* fix(datatables): only a database this wizard run made makes its retry idempotent

A retry skipped the create whenever the name was registered, which let a run adopt a database
another superadmin had made and share their data without the warning the existing-database
branch shows. The run tracks what it created instead, and reports it so Discard can say the
database is still there. Creating one now refreshes the registry the next run's default name
and its validation read, and that default counts the external cluster's databases rather than
the instance's.

Row ownership carries the substrate: "instance" and "external_instance" can hold the same name,
and a row repointed between them while a run probes must not read as that run's own.

Also fixes the merge leftovers in the DuckDB executor's tests, which cargo check never compiles:
the new PgDatabase field and the attach helper's job directory.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* fix(datatables): shorten the picked managed kind, and say what an empty database list means

The closed select has room for the qualifier, not the whole name, so a picked managed kind
reads "Managed (Internal)". An external entry gets the tooltip its internal counterpart has,
and a database picker with nothing in it says to type a name rather than "No items found".

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* fix(datatables): keep the wizard's external-database bookkeeping optional and parked

The run deps gained a required field, which every existing caller of runSetup -- the sibling
test suite included -- does not pass. It defaults to empty instead, the parked payload carries
it across a Supabase redirect as it claims to, and a resumed run counts it as something left
behind. Two regressions pin the create: made on a first attempt, skipped only for a database
this run made.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* fix(datatables): review an external-cluster database as one, not as a Postgres resource

The review step fell through to the resource branch for the new provider, so it announced a
connection already in the workspace for a database that does not exist yet.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* fix(datatables): show the server's refusal instead of an object in the toast

Saving data table settings passed the whole error to the toast, which rendered the request
and response as JSON, and the instance-database wizard replaced it with "check console".
Both surface the message the API sent, through the helper that exists for it.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* chore: point at the EE commit dropping the unused import

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* fix(datatables): define a role on the cluster its data table sits on

The permissions drawer created and listed roles without naming a cluster, so a role defined
from an external data table became a login on Windmill's own, and the refresh then replaced
the correct catalog with the internal one. The drawer passes the data table's cluster to both.

The wizard also read a database it had just created as a name collision, which blocked
returning to the failed attempt and finishing under the same name.

And the merge had left the CONNECT-grant pass and its log in both the caller and the callee.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* feat(datatables): manage the external cluster's role catalog, and name the cluster in the copy

The roles drawer offered one catalog, so the external cluster's roles could only be reached
through a data table sitting on it — and became unreachable once the last one was gone, while
the cluster could not be unset until they were dropped. Opened from the page it now offers the
clusters the instance has; a data table's own drawer still pins its cluster, and its copy names
that cluster rather than "the instance".

A DuckDB attach also decides whether to verify certificates the way every other Postgres
connection does, so a resource carrying a root certificate is no longer downgraded to require
here alone.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* fix(datatables): bind the roles list to the cluster it was read for

Switching catalogs left the previous cluster's rows on screen and applied whichever response
landed last, so a slower read could seat one cluster's logins under the other's heading while
every control acted on the wrong id — invisible where a name exists on both. The switch clears
the rows, a token discards a response the selection has moved past, and the controls stay inert
while a catalog is being read.

The drop confirmation also names the cluster whose databases it reaches.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* chore: update ee-repo-ref to dfe7b8b9204d21e0264fbea1c6f6eedf9e738d56

This commit updates the EE repository reference after PR #810 was merged in windmill-ee-private.

Previous ee-repo-ref: 30db1b33bda0446f5fc5dfe353fbb226d57a26d4

New ee-repo-ref: dfe7b8b9204d21e0264fbea1c6f6eedf9e738d56

Automated by sync-ee-ref workflow.

---------

Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
Co-authored-by: windmill-internal-app[bot] <windmill-internal-app[bot]@users.noreply.github.com>
Co-authored-by: Ruben Fiszel <ruben@windmill.dev>
2026-09-29 14:45:47 +02:00
Ruben FiszelandClaude Opus 5.5 1fd729ac1a feat: add an instance-wide accent color setting with sidebar tint (#11335)
* feat: add an instance-wide accent color setting with sidebar tint

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

* fix: paint the cached accent before the license resolves

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

* feat: serve the banner and accent color from one cached endpoint

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

* fix: invalidate the instance ui cache and bound it with a ttl

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

* fix: keep the instance ui ttl under the client poll period

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

* refactor: load the banner and accent color once per page load

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

* refactor: read the banner and accent color without a server cache

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

* fix: show a cleared accent color as off

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-09-25 11:23:30 +02:00
Ruben FiszelandClaude Opus 5 87761d1797 feat: let every EE instance set the announcement banner (#11238)
* feat: let every EE instance set the announcement banner, not only cloud

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* feat: move the announcement banner setting to the bottom of the general tab

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-18 21:03:00 +02:00
Ruben FiszelandClaude Opus 5 a9b0d871a4 surface OTEL env vars in the settings page and startup log (#11074)
* feat: surface OTEL env vars in the settings page and startup log

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01RbLzVfFDZ9pjBGHSzCSrNZ

* chore: log OTEL compression and per-signal timeouts in the startup config

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01RbLzVfFDZ9pjBGHSzCSrNZ

---------

Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-10 22:54:54 +02:00
Ruben FiszelandClaude Opus 5 abf4c6c234 feat: add a dismissible instance-wide announcement banner (#11037)
* feat: add a dismissible instance-wide announcement banner

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01ExBp57hUoB8hQm36bJuUEs

* fix: harden instance banner validation and mandatory-banner visibility

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01ExBp57hUoB8hQm36bJuUEs

* fix: sequence instance banner loads and match the backend character cap

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01ExBp57hUoB8hQm36bJuUEs

* fix: gate the settings save on a valid instance banner link

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01ExBp57hUoB8hQm36bJuUEs

* feat: restrict the announcement banner to the managed cloud

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01ExBp57hUoB8hQm36bJuUEs

---------

Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-09 11:20:25 +02:00
hugocasaandClaude Opus 5 d472193e5b feat: add retention cleanup for the otel_traces table (#10949)
* feat: add retention cleanup for the otel_traces table

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01NLhUaCPpLRAa29rSZDjS28

* fix: vacuum otel_traces and badge its retention setting EE

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01NLhUaCPpLRAa29rSZDjS28

---------

Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-02 19:41:46 +02:00
318c9f0073 feat(git-sync): dedicated base url for GitHub webhook delivery (#10411)
* feat(git-sync): let GitHub webhooks register a dedicated base url

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* fix(git-sync): validate the webhook base url and apply it on change

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* chore: pin ee ref for the git-sync webhook base url change

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* fix(git-sync): validate and reconcile the webhook base url on every write path

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* fix(git-sync): route every declarative settings writer through the same rules

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* fix(git-sync): let the reconciler own the webhook field write-back

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* fix(git-sync): make the webhook base url validators agree across UI and server

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* fix(git-sync): lock the workspace row across git_sync read-modify-writes

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* test: pin the webhook base url validator to its server counterpart

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* fix(git-sync): retry a failed webhook move on every re-apply of the setting

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* fix(git-sync): retry pending webhook moves on every declarative re-apply

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* fix(git-sync): reject non-string webhook base urls and bound the sweep

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* fix(git-sync): reject credential-bearing webhook base urls

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* fix(git-sync): keep credentials out of webhook base url validation errors

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* fix(git-sync): redact through the last authority @ when reporting a bad url

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* fix(git-sync): stop echoing unparsed webhook base urls instead of scrubbing them

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* fix(git-sync): never echo a submitted webhook base url in validation errors

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* fix(git-sync): keep the submitted scheme out of validation errors

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* refactor(git-sync): drop the webhook sweep, surface stale receivers in settings

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* fix(git-sync): refresh the stale webhook list when settings are saved

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* fix(git-sync): mark registered_url nullable and drop the duplicated field error

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* chore: pin ee ref after dropping the reconcile lock and CAS

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* fix(git-sync): refresh the stale webhook list on category saves too

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* chore: update ee-repo-ref to aa05ca8e97fc8265cd724753a80db37f83243254

This commit updates the EE repository reference after PR #695 was merged in windmill-ee-private.

Previous ee-repo-ref: 3e6cd9226b68707233ae2434511fe5131dce808b

New ee-repo-ref: aa05ca8e97fc8265cd724753a80db37f83243254

Automated by sync-ee-ref workflow.

---------

Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
Co-authored-by: windmill-internal-app[bot] <windmill-internal-app[bot]@users.noreply.github.com>
2026-07-30 23:27:12 +02:00
396fb1c475 feat(otel-tracing-proxy): trust internal endpoints with untrusted CAs (#10139)
* [ee] feat(otel-tracing-proxy): trust internal endpoints with untrusted CAs

Add `insecure_upstream_hosts` and `upstream_ca_certs` to the HTTP Request Tracing
settings so the OTEL tracing proxy can reach internal endpoints with untrusted or
private-CA certificates while keeping them traced. Wires the two settings through
the worker config and live reload, adds the inputs to the instance settings UI,
and pulls in the rustls upstream-client deps (hyper-rustls/tokio-rustls/rustls/
rustls-native-certs/rustls-pemfile; hyper-http-proxy switched to its rustls
feature).

The proxy-side implementation lives in the companion EE PR.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* chore: bump ee-repo-ref to otel_ca companion commit

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* fix(otel-tracing-proxy): expose new fields in declarative config; bump ee-ref

Addresses code-review findings:
- Add `insecure_upstream_hosts` and `upstream_ca_certs` to the declarative
  `OtelTracingProxySettings` in instance_config.rs so operator/GitOps-managed
  installs can set them and reconciliation no longer drops values saved via the UI.
- Restore the trailing newline on ee-repo-ref.txt and bump it to the companion
  EE commit carrying the strict host-matching / port-ordering fixes.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* chore: update ee-repo-ref to 51e50629f48dbc4f5520a787b4bdfb76f4cd38d3

This commit updates the EE repository reference after PR #665 was merged in windmill-ee-private.

Previous ee-repo-ref: 49f458e4446395e98915c220baa757ab3b2ed2d8

New ee-repo-ref: 51e50629f48dbc4f5520a787b4bdfb76f4cd38d3

Automated by sync-ee-ref workflow.

---------

Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Co-authored-by: windmill-internal-app[bot] <windmill-internal-app[bot]@users.noreply.github.com>
2026-07-17 15:57:00 +02:00
ff774c46bf feat: add per-workspace job-retention override (#10050)
* feat: add per-workspace job-retention override (EE)

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* chore: update ee-repo-ref to 2ba6a2a75b6fc97858b306b2c98ada481e363c10

This commit updates the EE repository reference after PR #658 was merged in windmill-ee-private.

Previous ee-repo-ref: e7fb36acd813cd717bcf05f5aafbf81de271d618

New ee-repo-ref: 2ba6a2a75b6fc97858b306b2c98ada481e363c10

Automated by sync-ee-ref workflow.

---------

Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Co-authored-by: windmill-internal-app[bot] <windmill-internal-app[bot]@users.noreply.github.com>
2026-07-11 23:46:34 +02:00
Ruben Fiszel b656dc6cdc feat(nsjail): optional disk-backed /tmp via instance setting (#9272)
* feat(nsjail): optional disk-backed /tmp via instance setting

* test(nsjail): unit-test tmp mount resolver and narrow visibility

* refactor(nsjail): switch tmp backing to select + conditional UI

* ui(nsjail): make tmpfs the visible default in /tmp backing select

* fix(nsjail): refuse preexisting jail_tmp to block symlink escape

* fix(nsjail): allow jail_tmp reuse on sequential nsjail calls

Codex flagged that python/ruby/rust executors invoke nsjail twice per
job_dir (install then run). The previous resolver treated any preexisting
jail_tmp as hostile and silently fell back to tmpfs on the second call,
so disk-backed mode never reached the main script run for those langs.

Use symlink_metadata().is_dir() to distinguish a real directory left by
an earlier call in the same job_dir (safe to reuse) from a symlink or
other entity (still refused, as the codebase-tar escape requires).

Also loosen the frontend visibility predicate: only hide nsjail settings
when job_isolation is explicitly 'none' or 'unshare', so deployments
that enable nsjail via DISABLE_NSJAIL=false with no DB setting can
still see the controls.
2026-05-21 15:34:49 +00:00
hugocasaandClaude Opus 4.7 f51b51a9a1 fix(frontend): open customer portal in popup synchronously to bypass Safari blocker (#9242)
* fix(frontend): open customer portal in popup synchronously to bypass Safari blocker

Safari blocks window.open() called after an await because it loses the
user-gesture context. Open a blank tab synchronously on click, then
assign location.href once the portal URL resolves.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* chore(backend): wire dev_override feature flag in backend crate

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-19 15:26:32 +00:00
Ruben Fiszel d48d61cc79 feat(otel-tracing-proxy): configurable tracing MITM NO_PROXY hosts (#9169)
* feat(otel-tracing-proxy): configurable NO_PROXY hosts

* refactor(otel-tracing-proxy): NO_PROXY only governs job-side bypass

* fix(otel-tracing-proxy): restore empty NO_PROXY default

* test(otel-tracing-proxy): unit tests for NO_PROXY normalization

* fix(otel-tracing-proxy): gate normalize_no_proxy_hosts to EE features
2026-05-14 14:03:47 +00:00
9c6cd8c852 offline (URL-bound) license keys (#9089)
* [ee] feat(license): offline (URL-bound) license keys

Offline keys are a 4-segment variant for air-gapped customers — no
phone-home, embedded seat/CU caps, locked to the instance's base_url.
Existing 3-segment online keys are unchanged.

Companion PRs:
- windmill-labs/windmill-ee-private (full design + EE impl)
- windmill-labs/windmill-customer-service (issuance + portal)
- windmill-labs/windmill-cf-worker-keygen (signing)

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* [ee] refactor(license): bind offline keys via instance hash; simpler CU enforcement

- /settings/license_status now surfaces an `instance_hash` superadmins share
  with support when requesting an offline key
- OfflineMetadata: `hash` replaces `base_url`; OfflineCapStatus reports
  `current_cu` (last 2min) and drops the grace-period fields
- verify_license_key now takes a db so EE can recheck the hash
- InstanceSetting.svelte: hash copy-block + simpler status panel
- Bump ee-repo-ref

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* [ee] chore(license): bump ee-repo-ref

Pulls in the current_cu clamp + prod public key restoration.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* [ee] refactor(license): split instance_hash endpoint; minimal cap UI; restore workers expiry toast

- `instance_hash` is no longer part of /settings/license_status responses; it
  lives at GET /settings/instance_hash (super-admin only) so it isn't re-emitted
  on every status poll. The UI doesn't show it — admins fetch it explicitly when
  requesting a key from support.
- InstanceSetting offline cap UI is now two compact green/red status lines
  (Seats X.X/Y and CUs X.X/Y) placed above the action buttons, matching the
  existing "Latest key renewal" badge style. The block-panel is gone.
- "Latest key renewal" line and the "Renew key" button are now hidden when an
  offline key is loaded (renewal is server-disabled for offline keys).
- Restore parseLicenseKey + checkLicenseExpiration toast on /workers
  (works for both 3- and 4-segment keys).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* [ee] chore(license): bump ee-repo-ref

Pulls in the plain-SHA256 instance hash + stats_ee revert.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* [ee] chore(license): bump ee-repo-ref

Picks up the alert wording change.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* [ee] chore(license): bump ee-repo-ref

Picks up the instance_uid cache so the periodic verify_license_key cycle
no longer hits global_settings.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* [ee] refactor(license): rename /settings/license_status → /offline_license_status

The endpoint was only used by the offline-license UI; the other fields it
returned (license_key_id, license_key_valid, kind, offline metadata) were
unused. Rename to clarify scope and flatten the response — it now returns
just the OfflineCapStatus (or null when no offline license is loaded).

Frontend uses `offlineCapStatus != null` as the "is offline" check.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* [ee] fix(ci): regenerate sqlx cache for the inline worker_ping query

After reverting unused stats_ee helpers (fetch_worker_pings*), the
inline `sqlx::query_as!(WorkerPingRecord, ...)` in get_stats_payload
lost its cache entry — CI's check_ee_full + cargo_test were failing
under SQLX_OFFLINE=true with E0282 type-inference errors.

Re-running update_sqlx.sh regenerates the cache file under its
current hash and prunes a couple of stale entries.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* [ee] fix(license): address cubic-bot review

- get_offline_license_status: propagate enforce_offline_caps errors as 500
  instead of swallowing into a "no offline license" (Option::None) response
- canonical_base_url: rewrite the doc to match the actual fallback behavior
  (lowercase + trailing-slash strip on URL parse failure); the original
  cross-service contract is gone since the customer-service no longer
  canonicalizes (treats the instance hash as opaque)
- check_seat_cap_for_new_user: take an email and short-circuit when the
  email is already in `usr ∪ workspace_invite` so net-zero invite upserts
  and invite→user transitions aren't spuriously blocked at cap. Mirrors
  the dedup rule the count itself uses.
- Bump ee-repo-ref to pull in the EE-side change

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* [ee] chore(license): bump ee-repo-ref

Picks up the exact-delta seat-cap check (replaces the simple existence
short-circuit). Regenerates the new sqlx cache for the bool_and query.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* [ee] fix(license): propagate get_instance_hash errors; bump ee-repo-ref

- get_instance_hash: replace `.ok().flatten()` with map_err+? so DB errors
  during instance_uid lookup surface as 500 instead of silently returning
  `{"instance_hash": null}` (same pattern get_offline_license_status already uses)
- Bump ee-repo-ref to pull in the enforce_offline_caps cached-state preservation

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* chore: update ee-repo-ref to c6cd1afe2d9e04809b30751cd1687b28a65e62b1

This commit updates the EE repository reference after PR #566 was merged in windmill-ee-private.

Previous ee-repo-ref: a6d91016ae0d43c46604313aecae3aa9c778c8e0

New ee-repo-ref: c6cd1afe2d9e04809b30751cd1687b28a65e62b1

Automated by sync-ee-ref workflow.

---------

Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Co-authored-by: Ruben Fiszel <ruben@windmill.dev>
Co-authored-by: windmill-internal-app[bot] <windmill-internal-app[bot]@users.noreply.github.com>
2026-05-11 22:09:21 +00:00
0aea49f960 feat: add http/protobuf support for OTEL exporters (#8702)
* feat: add --main flag to write_latest_ee_ref.sh to point to latest EE main

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* [ee] feat: add http/protobuf support for OTEL exporters

Add http-proto and reqwest-client features to opentelemetry-otlp to
enable HTTP/protobuf transport as an alternative to gRPC.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* chore: update ee-repo-ref.txt

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* feat: expose OTEL protocol selector in instance settings UI

Replace the hardcoded "gRPC" label with a dropdown allowing users to
select between grpc (default) and http/protobuf.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* chore: update ee-repo-ref to 681b725781611510ed3040c00e8f9b8497d6feda

This commit updates the EE repository reference after PR #509 was merged in windmill-ee-private.

Previous ee-repo-ref: 50051ded8183e662a9e932d87d17258501f3e944

New ee-repo-ref: 681b725781611510ed3040c00e8f9b8497d6feda

Automated by sync-ee-ref workflow.

* fix: remove reqwest-client feature to avoid conflict with default reqwest-blocking-client

The opentelemetry-otlp crate only activates the reqwest-client HTTP client
when reqwest-blocking-client is NOT also enabled. Since the default features
include reqwest-blocking-client, having both resulted in no HTTP client being
created. The default reqwest-blocking-client works correctly.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* chore: update ee-repo-ref.txt

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* iterate

---------

Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Co-authored-by: windmill-internal-app[bot] <windmill-internal-app[bot]@users.noreply.github.com>
2026-04-03 19:30:25 +00:00
7de98c0df4 feat: add OTel metrics support (#8442)
* [ee] feat: add OTel metrics support

Add OpenTelemetry metrics export for Windmill operational metrics.
When the OTel metrics toggle is enabled in instance settings (EE),
Windmill exports 16 metrics to any OTLP-compatible collector, letting
users observe queue depths, worker execution, DB pool state, and health
without a separate Prometheus setup.

Changes:
- otel_oss.rs: no-op stubs for OSS builds
- monitor.rs: queue count/running count gauges, zombie counters, DB pool
  monitoring (shared single DB query and loop with Prometheus)
- worker.rs: execution count/duration, worker busy, pull duration
- jobs.rs: queue push/delete/pull counters
- health.rs: DB latency gauge
- main.rs: call monitor_pool_otel unconditionally
- InstanceSetting.svelte: enable metrics toggle for EE licenses

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* chore: update ee-repo-ref.txt for OTel metrics

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* feat: add worker_started, worker_uptime, health_status, health_db_unresponsive OTel metrics

Wire up 5 additional metrics to reach parity with Prometheus:
- worker_execution_failed: wired in add_completed_job_error (was defined but unused)
- worker.started: incremented on worker startup
- worker.uptime: recorded each loop iteration
- health.status: phase gauge (healthy/degraded/unhealthy)
- health.db_unresponsive: flag (0/1)

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* chore: update ee-repo-ref.txt

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* chore: update ee-repo-ref to fbe68e4aa621e30378995cfd328a6ccf74176614

This commit updates the EE repository reference after PR #469 was merged in windmill-ee-private.

Previous ee-repo-ref: 6fa1881aafdfb60f4abf11a37f01f6fedaecb3ec

New ee-repo-ref: fbe68e4aa621e30378995cfd328a6ccf74176614

Automated by sync-ee-ref workflow.

* fix: remove duplicate cfg attr and duplicate OTel pool reporting

- Remove duplicate #[cfg(feature = "prometheus")] on monitor_pool
- Remove OTel block from monitor_pool; monitor_pool_otel is the sole
  OTel reporter, eliminating duplicate windmill.db.pool.* metrics in
  EE builds
- Simplify monitor_pool back to its original Prometheus-only structure

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Co-authored-by: windmill-internal-app[bot] <windmill-internal-app[bot]@users.noreply.github.com>
2026-03-19 08:10:37 +00:00
Ruben FiszelandClaude Opus 4.6 372023e995 feat: add ws_base_url instance setting for WebSocket URL override (#8405)
* feat: add ws_base_url instance setting to override WebSocket base URL

Allow deployments behind reverse proxies to route WebSocket traffic
(LSP, debugger, multiplayer) to a different host/port than the main
frontend via a new instance setting.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* feat: move ws_base_url to Advanced section with toggle and connectivity test

- Move setting from Core to Advanced > WebSocket section
- Render as toggle "Custom websocket base url from frontend to
  multiplayer/lsp/debugger" with conditional URL text field
- Add Test connectivity button (always visible) that checks HTTP health
  and WebSocket ping for all three services (LSP, Multiplayer, Debugger)
- Add /ws/ping and /ws/health endpoints to LSP service
- Add /ws_mp/health HTTP and __ping__ WS handlers to multiplayer service
- Add /ping WS handler to debugger service
- Add CORS headers to health endpoints for cross-origin testing

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix: toggle enabled check and testWs promise resolution

- Fix enabled derived to check only for null (not empty string),
  otherwise the toggle never turns on since toggleEnabled sets ''
- Fix testWs onclose handler to resolve(false) so the promise
  doesn't hang if the server closes without sending a message

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix: make connectivity test work with existing services

- HTTP test: accept plain text "ok"/"okay" (old services) in addition
  to JSON {"status": "ok"} (new services), reject HTML (SPA fallback)
- WS test: resolve on onopen (connection established) instead of
  waiting for a specific pong message, so the test works even with
  services that don't have the new /ping handler yet

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-17 12:48:59 +00:00
2e430c4c0b feat: add GitHub Enterprise Server (GHES) support for GitHub App git sync (#8344)
* feat: add GitHub Enterprise Server (GHES) support for GitHub App git sync

Add a self-managed GitHub App mode alongside the existing managed
(stats.windmill.dev) mode, enabling git sync for GitHub Enterprise Server
and custom GitHub App installations.

Backend:
- Parameterize GitHub API URLs (no more hardcoded github.com)
- Add GITHUB_ENTERPRISE_APP_SETTING global setting
- Add OpenAPI specs for ghes_installation_callback and ghes_config endpoints

Frontend:
- Add instance settings UI for configuring self-managed GitHub Apps
  with setup instructions and validation
- GHES installation flow in gh_success page
- Dynamic installation URL based on GHES config
- Increase git sync test connection timeout to 10s
- Block "Review changes" save when settings are invalid

EE companion PR: windmill-labs/windmill-ee-private#<PR_NUMBER>

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* chore: update ee-repo-ref to c74c86b78a66b976fd9968b21f77903723e668ec

This commit updates the EE repository reference after PR #459 was merged in windmill-ee-private.

Previous ee-repo-ref: 45e4550110799525b5502cf072c8af8132492638

New ee-repo-ref: c74c86b78a66b976fd9968b21f77903723e668ec

Automated by sync-ee-ref workflow.

* sqlx

---------

Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
Co-authored-by: windmill-internal-app[bot] <windmill-internal-app[bot]@users.noreply.github.com>
Co-authored-by: Ruben Fiszel <ruben@windmill.dev>
2026-03-13 08:21:15 +00:00
fe1519f128 feat: support minimal telemetry mode (#8243)
* feat: support minimal telemetry mode for EE

When EE customers disable telemetry, send a reduced payload with only
license-compliance data instead of ignoring the setting. Job usage data
is excluded in minimal mode. The telemetry settings UI now shows in EE
with context-appropriate descriptions for both CE and EE.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* chore: update ee-repo-ref for telemetry-minimal

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* feat: make telemetry toggle label and description license-aware

Show "Minimal telemetry" with EE-specific description on EE, and
"Disable telemetry" with CE-specific description on CE.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* chore: update ee-repo-ref

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* Update commit hash in ee-repo-ref.txt

* Update reference hash in ee-repo-ref.txt

* chore: update ee-repo-ref to 2f52c015bc6c81391234fa87b27ee1d4cd3a48a3

This commit updates the EE repository reference after PR #440 was merged in windmill-ee-private.

Previous ee-repo-ref: 3628ed51426d8d29b3d5c62864ba256b7f9eab17

New ee-repo-ref: 2f52c015bc6c81391234fa87b27ee1d4cd3a48a3

Automated by sync-ee-ref workflow.

---------

Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
Co-authored-by: windmill-internal-app[bot] <windmill-internal-app[bot]@users.noreply.github.com>
2026-03-11 19:12:47 +01:00
GuilhemandClaude Opus 4.6 57ca7dbca0 improve instance settings drawer UX (#8002)
* fix(frontend): prevent false dirty state in instance settings on load

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix(frontend): handle undefined python version in select binding

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* refactor(frontend): extract SaveButton component and improve drawer header UX

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* refactor(frontend): replace inline diff with diff drawer and simplify save flow

Save now saves immediately instead of requiring a two-step confirm flow.
Diff view opens in a separate drawer with split/unified toggle instead of
replacing the form content inline.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix(frontend): preserve dirty state when toggling YAML mode in instance settings

syncFormToYaml() was setting yamlCodeInitial to the current modified YAML,
causing hasUnsavedChanges to become false when entering YAML mode with
pending form changes. Build yamlCodeInitial from initialValues instead.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix(frontend): clear dirty state after saving in YAML mode

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* reduce save button timeout

* feat(frontend): add review changes button to unsaved changes confirmation modal

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* fix(frontend): address code review issues from PR #8002

Remove unnecessary IIFE wrappers in handleSave/handleSaveAndCloseDiff,
fix stale on:close reference on diff drawer, clip SaveButton overlay with
overflow-hidden, make DiffEditor respond reactively to inlineDiff prop
instead of using {#key} destroy/recreate, and revert normalizeValue
object check to original simpler behavior.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix(frontend): remove tab-switch confirmation modal in full settings mode

In full mode, the save button saves all settings across all categories,
so switching tabs cannot lose unsaved changes. Remove the per-category
dirty check, confirmation modal, and unused ConfirmationModal import.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix(frontend): prevent SMTP toggles from creating false dirty state

Use getter/setter bind:checked so Toggle reads undefined as false
without writing it back to the store. This prevents visiting the SMTP
tab from mutating smtp_settings and triggering a false unsaved diff.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix(frontend): prevent OTEL toggles from creating false dirty state

Same fix as SMTP toggles: use getter/setter bind:checked so Toggle
reads undefined as false without writing it back to the store.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* refactor(frontend): use recursive normalizeValue for dirty state instead of per-component fixes

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* refactor(frontend): replace save button with always-visible review changes button

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* fix(frontend): address PR review comments on DiffEditor and SaveButton

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-18 21:12:20 +00:00
GuilhemandClaude Opus 4.6 4d1d17580b add fuzzy search to instance settings (#8000)
* feat: add fuzzy search to instance settings sidebar

Adds a search input at the top of the superadmin settings sidebar that
uses uFuzzy for fuzzy matching against all setting labels, descriptions,
and categories. Selecting a result navigates to the correct tab and
scrolls to the specific setting card with a brief highlight.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix: improve settings search display and description matching

- Extract only the label portion from uFuzzy highlighted text for
  cleaner dropdown display
- Show description only when the match is in the description and NOT
  in the label
- Truncate descriptions to 80 chars in searchable items
- Add maxHeight prop to SelectDropdown for configurable height

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix: fix search description truncation and handle undefined marked values

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix: remove description from settings search dropdown

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* feat: add smooth outline transition for setting highlight animation

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* feat: highlight first search result by default for enter-to-select

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* nit

* clean code

* fix: address review feedback - sanitize html, remove max-w-40, document description field

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix: constrain search dropdown width to prevent long title overflow

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* perf: add 150ms debounce to settings search filter

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* fix: clean up timeouts on destroy and re-invocation

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* refactor: extract settings search into reusable SettingsSearchInput component

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* fix: use twMerge for class prop in SettingsSearchInput

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix: wrap debounced state write in untrack to prevent re-triggering

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-18 17:12:27 +00:00
centdixandClaude Opus 4.6 e7b0b00f56 fix: mark base_url as unsaved when using browser fallback (#7964)
* fix: mark base_url as unsaved when using browser fallback in instance settings

When base_url is not set in the database, the frontend silently fills in
window.location.origin but also snapshots it as the initial value. This
makes the dirty-check see no change, so the Save button stays disabled
and the user cannot persist the auto-detected value.

Fix by snapshotting initialValues before applying the fallback, and show
a yellow warning indicating the value is auto-detected and unsaved.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix: render base_url fallback warning inside the setting card

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-16 17:01:18 +00:00
Ruben FiszelandClaude Opus 4.6 82e5f6de48 feat: add Kubernetes operator and instance settings YAML editor (#7836)
* Add windmill-operator crate for Kubernetes CRD-based instance config

Introduces a new `windmill-operator` crate that enables declarative
management of Windmill instance configuration via a Kubernetes
`WindmillInstance` CRD. The operator watches CRD resources and performs
full declarative sync of global_settings and worker configs to the
database, supporting GitOps workflows for instance-level configuration.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* Add tests for windmill-operator CRD and db_sync

- 9 unit tests for CRD serialization, deserialization, metadata, and
  status field behavior
- 15 integration tests for db_sync using #[sqlx::test] with full
  declarative sync coverage: upsert, delete, protected keys,
  idempotency, worker config prefix handling, and end-to-end sync

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* Replace untyped BTreeMap CRD fields with typed structs for schema validation

GlobalSettings, SmtpSettings, IndexerSettings, and WorkerGroupConfig now
have explicit typed fields with serde(flatten) catch-all for forward
compatibility. The generated CRD YAML includes a full OpenAPI v3 schema
that Kubernetes validates on kubectl apply.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* Type opaque serde_json::Value CRD fields with real structs

Replace most remaining serde_json::Value fields in WindmillInstance CRD
with properly typed structs derived from the codebase:
- oauths: BTreeMap<String, OAuthClient>
- otel: OtelSettings
- otel_tracing_proxy: OtelTracingProxySettings with ScriptLang enum
- critical_error_channels: Vec<CriticalErrorChannel> (untagged enum)
- critical_alerts_on_db_oversize: DbOversizeAlert
- ducklake_settings: DucklakeSettings with nested catalog/storage types
- custom_instance_pg_databases: CustomInstancePgDatabases
- autoscaling (worker config): AutoscalingConfig with integration struct
- custom_tags, default_tags_workspaces: Vec<String>
- default_tags_per_workspace: bool

Still opaque (serde_json::Value): object_store_cache_config (kube-core
can't generate schemas for internally-tagged enums), secret_backend
(EE-private), slack, teams (no clear struct definitions).

Regenerated CRD YAML with full OpenAPI schema (352→703 lines).

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* Unify instance config types and add bulk GET/PUT API

Move all typed settings (GlobalSettings, WorkerGroupConfig, etc.) from
windmill-operator/crd.rs into windmill-common/instance_config.rs so both
the API server and operator share a single source of truth. Add diff/apply
logic (Merge mode for UI, Replace mode for operator) and InstanceConfig::from_db().

Add GET/PUT /settings/instance_config endpoints so the frontend loads all
settings in 1 call instead of 42, and saves with a single bulk PUT. The
backend handles the diff internally, running pre-write hooks for changed keys.

Refactor windmill-operator/db_sync.rs to use the shared diff+apply functions
and slim crd.rs down to the CRD wrapper with re-exports.

Includes 32 unit tests and 30 integration tests covering serialization,
diff logic, DB roundtrips, protected settings, and edge cases.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* feat: add Form/YAML toggle to instance settings UI

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* fix: show Form/YAML toggle regardless of hideTabs prop

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* refactor: replace toggle button group with simple YAML toggle

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* feat: mask sensitive fields in YAML view with show/hide toggle

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* feat: hide internal settings and mask sensitive fields in YAML view

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* feat: hide jwt_secret and min_keep_alive_version from API and config exports

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* all

* feat: add secretKeyRef support for sensitive fields in operator CRD

Allow sensitive fields (license_key, hub_api_secret, scim_token,
smtp_password, OAuthClient.secret, custom PG user_pwd) to reference
Kubernetes Secrets via the standard secretKeyRef pattern instead of
inlining values as plaintext YAML. The reconciler resolves all refs
by reading K8s Secrets before syncing to the database.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* all

* all

* all

* fix: merge main and update dev environment docs

Resolve merge conflicts from origin/main, fix duplicate
UV_INDEX_STRATEGY_SETTING import, and add Playwright MCP
testing instructions to CLAUDE.md.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* all

* fix: init tracing for CLI subcommands and deduplicate setting side-effects

Initialize tracing subscriber before early-return CLI paths (sync-config,
operator) so tracing calls are not silently dropped. Refactor
set_global_setting_internal to call run_setting_pre_write_hook instead of
duplicating the side-effect logic.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* feat: add `wmill instance get-config` CLI command

Dumps the current instance config (global settings + worker configs) as
YAML. Supports --output-file to write to a file instead of stdout.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* all

---------

Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-14 22:42:01 +00:00
Ruben Fiszel 93be81d3e8 improve on-boarding II 2026-02-13 00:00:36 +00:00
Ruben Fiszel 4e38a4f108 fix: improve on-boarding experience 2026-02-12 23:02:05 +00:00
Guilhem dd419ade94 fix(frontend): redesign instance settings (#7916)
* handle dirty config

* Move instance update in drawer actions

* Put windmill version in drawer header

* Use sidebar instead of tabs

* Rework user section

* improve user table

* Handle EE

* Add settings section header

* test-1

* option 2

* create new settings group

* harmonize all settings inputs

* improve members/users table styling

* refactor instance setup

* show user count

* nit

* Create setting card component

* harmonize instance settings and workspace settings

* fix python version loader

* nit

* clean code

* nit

* add email validation

* fix reactivity issue on default value

* fix dirty config check

* fix object storage dirty config check

* Fix object storage settings reactivity

* fix indexer dirty reactivity

* Add validation for indexer

* fix sso dirty issues

* clean

* nit
2026-02-12 09:50:01 +00:00
Pyraandclaude[bot] 097d9288c5 feat(uv): index resolve strategy (#7885)
* fix

* fix

Signed-off-by: pyranota <pyra@duck.com>

* Update backend/windmill-worker/src/python_executor.rs

Co-authored-by: claude[bot] <209825114+claude[bot]@users.noreply.github.com>

* Update backend/windmill-worker/src/python_executor.rs

Co-authored-by: claude[bot] <209825114+claude[bot]@users.noreply.github.com>

---------

Signed-off-by: pyranota <pyra@duck.com>
Co-authored-by: claude[bot] <209825114+claude[bot]@users.noreply.github.com>
2026-02-11 11:18:51 +00:00
Guilhem 31bfccc745 fix(frontend): improve time picker (#7893)
* clean layout

* remove useless toggle

* improve layout

* nit

* Fix input type seconds

* convert secondInputs to svelte 5

* redesign seconds input

* Adapt layout where time input is used

* nit
2026-02-11 00:48:18 +00:00
hugocasa 8363ff1eee feat: download encrypted usage (#7804) 2026-02-10 09:50:24 +00:00
Guilhem dd421845ba fix(frontend): reorganize workspace settings (#7788)
* Add vertical nav bar to workspace settings

* harmonize settings content titles

* remove sidebar icons

* add background to sidebar

* nit user section

* EEonly display

* Workspace settings general design

* Add schema validation and dirty detection

* Put critical alerts in a separated tab

* separate error success handler

* only enable save when there is some changes

* Fix dirty detection for deployment UI

* Only enable save button when changes for datatables ws storage

* Add setting footer component

* Use new footer setting for saving configs

* nit

* apply setting footer

* improve save button

* nit

* nit

* nit

* make ws app use same pattern as other tabs

* Separate scrolling between sidebar and content

* Gather error handlers

* use universal save button for object storage

* Title sentence case

* nit

* nit

* improve dirty config logic

* nit

* nit

* clean dead code

* Use settings footer for deployment settings

* Git sync settings

* move tabs

* fix dirty stats of error handlers

* nit

* nit
2026-02-09 18:22:22 +00:00
Pyra 95df7b9a6a feat: otel REST tracing (#7571) 2026-01-20 13:38:02 +00:00
Ruben Fiszel 1b9d1c56c7 feat: add HashiCorp Vault secret storage integration (#7599)
* feat: add HashiCorp Vault secret storage integration

- Create SecretBackend trait abstraction for secret storage
- Add VaultBackend implementation with CRUD operations
- Integrate secret backend into variable CRUD operations
- Add migration functions (DB → Vault and Vault → DB)
- Add frontend configuration UI for secret backend
- Add test connection and migration endpoints
2026-01-18 23:08:29 +00:00
GuilhemandAlexander Petric 210b8285d4 fix(frontend): settings redesign (#7406)
* improve collapsible link

* do not show superadmin ws link when already in it

* improve OAuth UI

* sso/oauth instance settings ui

* refactor instance settings alerts WIP

* Indexer and Oauth to brand guidelines

* refactor ws error handler page

* Create a tab SMTP in the Instance Settings

* Ractivity isssue fix for tabs

* nit

* Add smtp settings status in Error handler

* Add smtp configuration status

* Display teams connection status for instance alerts

* nit

* Add critical alerts description

* nit

* nit

* improve ee display

* nit

* nit

* fix typo

* nit

* restore vit config

---------

Co-authored-by: Alexander Petric <alex@windmill.dev>
2025-12-19 20:33:03 +00:00
Alexander Petricandwindmill-internal-app[bot] 96aaceef95 fix: improve teams search ux (#7407)
* feat: improve teams search ux

* ee ref

* claude review

* chore: update ee-repo-ref to e218dfce97dcea56c6ef6032592dab812a3f5047

This commit updates the EE repository reference after PR #363 was merged in windmill-ee-private.

Previous ee-repo-ref: 1b95a24ab25d96e59d2f22588901e9d3ce6c72b3

New ee-repo-ref: e218dfce97dcea56c6ef6032592dab812a3f5047

Automated by sync-ee-ref workflow.

---------

Co-authored-by: windmill-internal-app[bot] <windmill-internal-app[bot]@users.noreply.github.com>
2025-12-19 13:03:15 +00:00
Guilhem 0594257a25 fix(frontend): update workers page ui (#7264)
* Improve workers page

* Update group config drawer

* improve dirty workergroup config

* Make layout reactive

* fix section animation

* prevent opening dropdown while clicking New group config

* migrate workers page to svelte 5

* Open drawer upon adding a worker group

* nit critical alert table

* improve queue metrics drawer

* improve agent worker drawer

* harmonize copy icon

* improve agent worker doc

* improve layout

* Improve autoscaling event list

* Improve tags managment

* Remove default tags

* fix npm check

* Add info for agent workers

* improve agent worker jwt token creation

* Improve token display

* nit

* improve tag display

* create EE component

* nit

* harmonize tag overflow

* handle permission better

* improve env var presets

* handle permission for config

* nit alerts

* nit

* Improve custom tag creation in tag select

* optimistic tag addition

* nit

* nit

* fix typo

* improve workers table

* Group config tags

* show mismatch

* fix typo

* optimistic update when adding tag

* do not allow to create tag when picking a tag to watch in alerts
2025-12-04 22:09:25 +00:00
Alexander Petric 408911dbf6 camelcase (#7091) 2025-11-07 18:51:33 +00:00
Alexander Petric 6045f0c406 fix: teams selector svelte5 (#7087) 2025-11-07 18:21:55 +00:00
9063945161 trim whitespaces from license key input field (#7084)
- Add onBlur prop support to Password component
- Trim license key on blur in InstanceSetting component
- Trim license key before saving in InstanceSettings component

This ensures leading and trailing whitespace is always removed
from the license key input field, both when the user leaves the
field and when settings are saved.

Co-authored-by: claude[bot] <41898282+claude[bot]@users.noreply.github.com>
Co-authored-by: windmill-internal-app[bot] <windmill-internal-app[bot]@users.noreply.github.com>
2025-11-07 16:22:13 +00:00
Diego Imbert f33ae2a4b1 Remove whitespace-nowrap from Button (#6955) 2025-10-27 15:49:27 +00:00
Guilhem 032f0c1f8c feat: UX/UI full overhaul to meet new design system 2025-10-22 15:56:36 +00:00
hugocasa c7fb178190 fix(frontend): interaction with code instance settings on chrome (#6859) 2025-10-17 14:22:19 +00:00
hugocasa e1d2eb2870 feat: support for protected private hub (#6762)
* feat: support for protected private hub

* nits

* nits
2025-10-08 09:33:28 +00:00
claude[bot]andwindmill-internal-app[bot] c7f6ff14af correct typos in user-facing text (#6725)
- Fixed 'occured' -> 'occurred' (10 instances)
- Fixed 'commited' -> 'committed' (2 instances)

Co-authored-by: claude[bot] <209825114+claude[bot]@users.noreply.github.com>
Co-authored-by: windmill-internal-app[bot] <windmill-internal-app[bot]@users.noreply.github.com>
2025-10-01 16:04:15 +00:00
hugocasa c751a5d6aa fix(frontend): prevent label interference with monaco editor in instance settings (#6701) 2025-09-29 12:04:52 +00:00
Ruben Fiszel 4cb06ebe99 frontend nits 2025-09-20 13:46:55 +00:00
Alexander Petric 70e9ae14a9 fix: teams api improvements (#6643)
* init

* adding cache + ui nits

* improve caching

* sqlx

* ee repo ref

* remove useless comments

* ci errors

* pr comments

* ee repo ref
2025-09-19 17:43:16 +00:00
hugocasa 5a97258375 fix: do not save license key when renewing if the expiry date is earlier than that of the current key (#6346)
* fix: do not save license key when renewing if the expiry date is earlier than that of the current key

* tmp ref

* final ref
2025-08-08 14:23:31 +02:00
HugoCasa 1d62dd0fc3 feat: better explanation for dev key renewal (#6209)
* fix: hide s3 catalog picker in anonymous apps

* nit reactivity upload text

* no signed s3 object warning

* remove warning

* fix check

* feat: better dev key explanation
2025-07-16 17:26:37 +00:00
c51e128920 fix: add support for GCS object storage (#6083)
* AI: Updates to files (run 15923369101)

* all

* ee ref

* unneeded

---------

Co-authored-by: windmill-internal-app[bot] <windmill-internal-app[bot]@users.noreply.github.com>
Co-authored-by: HugoCasa <hugo@casademont.ch>
2025-06-30 20:11:57 +00:00
Ruben Fiszel 283167264e more svelte 5 migrations 2025-06-13 08:47:27 +02:00
HugoCasa f412ede6ed feat: multiple azure models support (#5920)
* feat: multiple azure models support

* all
2025-06-12 17:34:33 +02:00