* fix(datatables): refuse a save that drops a data table's roles through an undeclared rename
A data table's roles follow its entry only through a declared rename. A
settings sync sends the whole map and never declares one, so renaming a
data table under roles there read as a delete and a new entry on the same
database: the new entry carried no roles, and every caller connected as
admin. Such a save is now refused, naming both entries.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BjfMkJyKzodxkobqGZ6Lqb
* fix(datatables): no entry without roles may newly reach a database under roles
The previous guard only caught a new name replacing an entry under roles.
A whole-map save could also repoint an existing entry without roles at
that database, or another workspace could point one there, and every
caller of that entry would connect as admin. The rule is now stated on
the saved entries: one that carries no roles and newly points at an
instance database any entry under roles uses, in this workspace or
another, is refused. A declared rename carries its roles and passes.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BjfMkJyKzodxkobqGZ6Lqb
* feat(datatables): move data table role catalog and resolution to the enterprise edition
Roles are an Enterprise Edition feature. The catalog, the Postgres logins,
CONNECT convergence, tenant evaluation and the role half of connection
resolution move to windmill-ee-private. Every public function keeps its path
and signature and forwards through datatable_roles_oss, which re-exports the
enterprise implementation or, without it, refuses.
Without the enterprise edition a data table under roles, or a caller naming a
role, is refused a connection rather than resolved as admin, and the reach and
admin-access checks refuse one under roles. A data table not under roles
resolves as before in every edition, and an instance database keeps the
CONNECT grants it was created with. The catalog lock, the stream lock, the
tenant cascades and the permissions stripping stay in OSS: they only restrict.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BjfMkJyKzodxkobqGZ6Lqb
* feat(datatables): move the data table permissions endpoints to the enterprise edition
The permissions read, save and usable-roles handlers move to
windmill-ee-private; the routes stay registered and, without the enterprise
edition, answer that data table roles are an Enterprise Edition feature.
ensure_governs_datatable and ensure_reaches_datatable keep their paths: the
first refuses, the second passes a data table not under roles.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BjfMkJyKzodxkobqGZ6Lqb
* feat(datatables): move the data table role catalog endpoints to the enterprise edition
The superadmin list, create, update and delete handlers move to
windmill-ee-private. The routes stay registered and, without the enterprise
edition, refuse after authentication.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BjfMkJyKzodxkobqGZ6Lqb
* test(datatables): run the roles tests on the enterprise edition, refusals without it
Each test that exercises roles runs with private and enterprise. Two tests run
without them: every roles route answers the Enterprise refusal, and a data
table saved under roles, or a named role, is refused a connection while one
not under roles resolves as before.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BjfMkJyKzodxkobqGZ6Lqb
* feat(datatables): gate the roles UI mount sites on an enterprise license
Both mount sites are still commented out; the gate travels with them.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BjfMkJyKzodxkobqGZ6Lqb
* test(datatables): run the tenant matcher test on the enterprise edition
The matcher it covers is enterprise code now, so without the enterprise
edition the test hit the stub and failed the default windmill-common run. It
runs with private and enterprise, and a counterpart without them asserts that
no tenant list covers anyone, the wildcard and a workspace admin included.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BjfMkJyKzodxkobqGZ6Lqb
* chore: update ee-repo-ref to a1873dbb67f2302b85ff5362f8387b48eccdb607
This commit updates the EE repository reference after PR #783 was merged in windmill-ee-private.
Previous ee-repo-ref: 5c853e2c20eca6b748415fc0d6862a6ebfb5fec4
New ee-repo-ref: a1873dbb67f2302b85ff5362f8387b48eccdb607
Automated by sync-ee-ref workflow.
* fix(datatables): refuse roles while a same-workspace alias reaches the database
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* feat(datatables): add an ACL editor for data table roles
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* feat(datatables): data table roles in the DB manager and raw apps
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* fix: never add a role to the reference of a data table whose name contains '?'
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* fix: read the roles of a data table whose name contains '?'
The generated client leaves a '?' in a path param unencoded, so the lookup
404'd and the raw-app picker blocked Start on such a data table.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* fix: take every pooled connection before the ACL apply locks
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BRoYE5ZeAVvrDYdfhDAYXb
* fix: refresh grant options only after the ACL apply validates its plan
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BRoYE5ZeAVvrDYdfhDAYXb
* fix(datatables): refuse a reference naming both a legacy data table and a role
When a workspace stores both `sales` and a legacy `sales?role=analytics`, the
reference resolved to the legacy entry without a role, so browsing `sales` as
`analytics` reached another data table.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* fix: declare the default role in migrations written for a data table whose name contains '?'
Such a data table connects as its default role without naming it, so the
migrations the manager wrote for it declared no role and ran as admin.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* fix(datatables): let CE migrations connect as an explicitly named admin
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* fix: add only missing grant options before an ACL apply, never default privileges
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BRoYE5ZeAVvrDYdfhDAYXb
* fix(datatables): serialize roles going on with aliases saved from other workspaces
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* docs(datatables): note that legacy names with ? cannot be migrated
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* fix: run one data table ACL apply at a time per server before it connects
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BRoYE5ZeAVvrDYdfhDAYXb
* feat(datatables): set up an external instance cluster for data tables
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* fix(datatables): send external cluster passwords as SCRAM verifiers
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* fix(datatables): scope external cluster credential readers to the crate
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* [ee] feat(datatables): external_instance data tables on the external cluster
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* fix: hold the ACL connection to the database that was authorized
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BRoYE5ZeAVvrDYdfhDAYXb
* fix(datatables): compare the external cluster settings under a row lock before storing setup
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* fix(datatables): only drop external databases Windmill marked, and check use under the lock
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* fix: build the ACL connection from the authorized data table entry
Resolving the settings again could land on a resource with the same
database name on another server, which the later entry checks never see.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BRoYE5ZeAVvrDYdfhDAYXb
* fix: check ACL read reach against the entry it connects from
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BRoYE5ZeAVvrDYdfhDAYXb
* feat(datatables): put a data table's connection under Postgres roles
A data table backed by the instance database resolved to exactly one Postgres connection,
`custom_instance_user`, for everyone who could reach it at all. There was no way to say
this job reads, that one writes, this one never sees the salaries table.
A data table role is now a real Postgres login on the cluster, defined once for the
instance by a superadmin and named exactly as they named it. A script that declares
`-- role analytics` connects as `analytics`, and Postgres decides what it may touch —
grants are ordinary SQL. Windmill answers only "may this caller ask for this role", from
the tenant lists on the data table entry: `u/alice`, `g/analysts`, `f/finance` or `*`.
A data table with no `permissions` block behaves exactly as before.
Everything that opens a connection on someone's behalf goes through one chokepoint,
`get_datatable_resource_from_db`, which takes the identity explicitly and fails closed when
there is none. The role logs in as itself — never `SET ROLE`, which a script could
`RESET ROLE` its way out of.
A fork's data table entry becomes a pointer at the workspace that governs it rather than a
copy of it. The settings clone used to hand a fork a byte-identical entry naming the
parent's database, which a fork admin could edit to grant themselves `admin` there; a
pointer has nothing local to edit, and its tenants are evaluated as a member of the
governing workspace, by email. `permissions` is stripped from the workspace export and
ignored on import: tenants name principals of one workspace, and a settings push is not
where an access decision should be made.
Operations that see the whole database whatever the roles grant stay with the governing
workspace's admins: editing the roles, a migration that declares none, and opening a
replication stream for a Postgres trigger or capture.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012ti5HyeTikPMYyW8YSdiHR
* fix(datatables): gate the paths that reach a whole database as admin
Auditing what still resolved through the unchecked resolver turned up three that act for a
caller and hand back the admin connection: `resolve_pg_source_checked` (behind schema
export, the full-schema read, database creation, import and the forked-database drop), the
connection test, and the schema snapshot a fork clone takes of its parent. On a data table
under roles each let any workspace member — or a fork admin who is nobody in the governing
workspace — read or copy the whole database whatever its roles grant.
All three now require admin reach on the governing workspace. A dump taken under a
restricted role would be a silently truncated copy rather than an error, so refusing is the
only right answer for the copy paths.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012ti5HyeTikPMYyW8YSdiHR
* fix(datatables): confine roles to the instance database, and stop a fork reaching the parent's bookkeeping
A data table role is a login on Windmill's own Postgres. Nothing stopped a workspace admin
putting a *resource-backed* data table under roles, at which point the executor dialled the
host that resource names — one the admin chose — with the role's real cluster password, and
`CONNECT` is granted to every registered instance database. Both ends now refuse: the
permissions endpoint rejects the save, and the chokepoint refuses to substitute credentials
on a non-instance entry rather than trusting the record it read.
Two more places reached the governing database without answering to it. The initial-migration
generator returned a `pg_dump` of the whole schema to any member. And the migration
rename/delete cascade followed a fork's pointer into the parent, so a fork admin renaming or
removing their own local entry relabelled or wiped the parent's `_wm_migrations` — after
which the parent re-runs every migration from zero. The remote half is now skipped when the
entry resolves into another workspace, which is also just correct: a fork renaming what it
calls a data table changes nothing about the data table.
Also: revoking a tenant now bounces the replication streams of every workspace holding an
entry that resolves here, not only the governing one, so a fork's trigger stops rather than
living on inside its open connection; the instance role catalog and the governing workspace's
tenant lists are no longer returned to someone who cannot edit them; and the tenant rename
dedup collapses non-adjacent duplicates, per role rather than once any role changed.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012ti5HyeTikPMYyW8YSdiHR
* fix(datatables): fail loudly where a role or a pointer can be left half-recorded
Three ways the feature could end up in a state nobody could see or undo.
Creating a role writes the cluster first and the catalog second, but the catalog write was an
`UPDATE` that matched nothing when the instance Postgres settings row was absent — leaving a
live login with a password nobody recorded: invisible to the catalog, un-recreatable because
the name is taken, and un-deletable because there is no entry to delete. It now errors, so
the operation is retryable once the row is restored.
Deleting a workspace only nulls the fork lineage; the data table entries pointing at it are
left resolving to nothing. Sweeping them is not an option — turning a pointer back into a copy
would hand each fork the database outright — so the delete now names the data tables it
stranded, and resolving one says which workspace is missing rather than reporting a data table
this workspace never had.
`InstanceDatatableRole` derived `Debug` while holding a Postgres password; it is now
hand-written so `{:?}` on the catalog cannot put a live credential in a log line.
Adds the two branches the reviews found unpinned: a caller who is not a member of the
governing workspace at all, and `NoIdentity` — the compatibility path for an agent worker that
predates this and sends no job id.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012ti5HyeTikPMYyW8YSdiHR
* fix(datatables): unbreak two operator messages and two comments that described other code
The two strings this branch added for states an operator hits once — the catalog write that
matched nothing, and the delete that stranded a pointer — were collapsed from their multi-line
form with the indentation left in, so both rendered with a fourteen-space gap mid-sentence.
`list_datatables` claimed to report a chain it cannot follow and then dropped it; it does drop
it, and the comment now says why that is the right place to stay quiet. The non-superadmin
check in `edit_datatable_config` was introduced as also covering references, which it does not
and need not: `reference` is overwritten from the stored entry for every caller before the
check runs.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012ti5HyeTikPMYyW8YSdiHR
* fix(datatables): serialize role catalog mutations, and state each helper's authorization contract
The catalog is one JSON document, so create, rename, enable and delete are all
read-modify-write. Two concurrent creates read the same snapshot, both succeed in the
cluster, and the second write drops the first — leaving a live Postgres login with a password
nobody recorded, which is the exact state the delete path exists to prevent. Every mutation
now runs in one transaction holding an advisory lock across the read, the cluster DDL and the
write, so a lost update cannot happen and a failure rolls the whole thing back. The DDL
helpers take that transaction rather than the pool, which is what makes the lock cover them.
Their statements moved off `sqlx::raw_sql`: the simple protocol is only needed for genuinely
multi-statement SQL, and its future is not `Send`, which an axum handler holding the
transaction requires. Each of these is one statement anyway.
The new cross-crate surface now says what callers must do. `read_role_catalog` returns
plaintext credentials; `create`/`rename`/`set_login`/`drop_instance_role` and
`converge_connect_grants` mutate cluster-wide state; `read_datatable_entry` reads a workspace's
raw config. All of them are superadmin-gated by their current handlers, but nothing said so at
the definition, which is where the next caller looks.
Also: the roles table reloads after a failed login toggle instead of leaving it claiming a flip
that did not land; the rename affordance is the design-system `Button`, not a raw one; and
`resolve_datatable_pg_as_caller` drops a `role` parameter no caller ever filled — browsing
resolves as the data table's default until the database manager grows a picker.
Why role passwords stay a plain `String` while the instance user's password beside them is a
`StringOrSecretRef`, asked three times across reviews: that one is a secret ref because an
operator supplies it and may want it from their own backend, while these are minted here and
never entered by anyone, so there is nothing for a ref to point at. Encrypting generated
secrets at rest is a separate change that would take the replication password with it. Now
said at the field.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012ti5HyeTikPMYyW8YSdiHR
* fix(datatables): give the role catalog its own row, out of reach of the config machinery
Putting it inside `custom_instance_pg_databases` was the wrong call, and it cost two ways.
The catalog serializes a generated Postgres password per role, and that row is the
operator-facing instance config, so the passwords reached `get_instance_config` and its YAML
editor — a live cluster credential in a response body, a UI field and any log of either.
Worse in the other direction: `to_settings_map` strips the catalog, so a full-row upsert of
that key writes the row back without it and the catalog is gone, while the cluster keeps every
login it described.
`custom_instance_replication_pwd` is the precedent and says exactly why — a generated secret,
written only by the server, never operator-authored, hidden so the config machinery cannot
read, rewrite or drop it. The catalog is the same thing, so it now has the same shape:
`datatable_roles`, in `HIDDEN_SETTINGS`, `PROTECTED_SETTINGS` and the agent-worker denylist.
No redaction to keep in step with three code paths, and no way for a neighbouring write to
take it out.
Two races on the same shared documents. `edit_datatable_config` read the stored data tables
outside its transaction and then wrote the whole `datatable` document, so a permissions save
committing in between was silently rolled back; it now reads under `FOR UPDATE`. And
`set_datatable_permissions` validated role ids against the catalog before opening its
transaction, so a deletion in between let it write a deleted role back — including as the
default, which every later job then fails on; it now holds the catalog lock and the settings
row across validation and write.
Completes the authorization contracts the previous commit claimed but did not finish:
`read_datatable_entry` (which it named and missed), `resolve_governing_datatable`, whose whole
job is to answer for a workspace the caller may not belong to, and
`converge_connect_grants_with`, which had not inherited its wrapper's.
Also the generic Python SDK reference: `_format_py_params` learned the bare `*` last time, but
`extract_py_functions` is a second formatter and still rendered `datatable(name, role)`, so
code written from that page passed a keyword-only argument positionally.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012ti5HyeTikPMYyW8YSdiHR
* fix(datatables): make the concurrency test pin the handlers, and the contracts describe what is enforced
The concurrency test reimplemented the read-modify-write inline, so deleting the lock from all
three handlers left it green — it pinned Postgres, not the code it was written for. It now
drives `create_datatable_role` twice concurrently and asserts the catalog kept both names.
Checked the way the last one should have been: removing the lock from the handler makes it
fail with "wmtest_a_… is a live cluster login the catalog forgot".
The contracts added last commit were stricter than this PR's own callers, which is worse than
none — the next reader sees a rule already broken and learns to ignore it.
`read_role_catalog` said superadmin-only while two of its four callers are open to any
workspace member, and `converge_connect_grants` said superadmin while
`set_datatable_permissions` reaches it as a workspace admin. Both were fine on substance: the
rule that actually holds is about the credential never reaching a response, log, audit record
or export, not about who may call. They now say that. `read_datatable_entry` gets the same
treatment rather than the one the earlier message claimed for it: it is the primitive every
resolution goes through, so it is deliberately open, and what must not escape is `permissions`
— it names the governing workspace's users, groups and folders.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012ti5HyeTikPMYyW8YSdiHR
* fix(datatables): close the last ways a role or a pointer can be left pointing at nothing
The raw settings readers hand back whatever is in the row, so moving the catalog into its own
`global_settings` key protected the config machinery and left `GET /settings/global/datatable_roles`
and the settings listing returning every live password. Both now filter that one key. The
neighbouring `custom_instance_replication_pwd` has the same shape and is not touched here: it
predates this and widening the fix to it is a decision about an operator workflow, not a
consequence of this change.
Three ways a save could leave something resolving to nothing:
A permissioned data table could be moved to a PostgreSQL resource. The block was carried across
as a server-owned field, the runtime refuses roles on a resource-backed table, so the save
succeeded and every job afterwards failed. Refused instead — turning roles off first is one step,
and it keeps discarding an access decision something somebody chose.
Renaming a governing data table left every fork pointing at the old name: the data table
disappears from their pickers and their jobs stop, with nothing in the renaming workspace to
suggest why. The rename now follows into the pointers in the same transaction.
Deleting one cannot be followed the same way, so it is reported instead — the response names what
it stranded, the way deleting a workspace does, and the fork's own error already says which
workspace is gone.
Also: `ensure_instance_db_grant_options_unchecked` claimed superadmin while the permissions
handler reaches it as a workspace admin (the same class fixed last commit, one instance missed);
the role entry kept an `instance_config_schema` derive it no longer needs; `write_role_catalog`
was the one writer of that table not stamping `updated_at`; and the concurrency test dropped its
roles only on success — a failing run is exactly the one that creates them without recording them.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012ti5HyeTikPMYyW8YSdiHR
* refactor(datatables): put the role catalog in its own table, not in global_settings
Five findings across three rounds were all the same choice. A set of live Postgres credentials
was living in `global_settings`, which has generic read, list, write, config-export and CLI
round-trip paths that know nothing about what they carry: the passwords reached the instance
config and its YAML editor, a full-row upsert of a neighbouring key erased the catalog,
`GET /settings/global/{key}` and the settings listing returned them raw, and this round the
redaction that fixed the last two turned `wmill instance push` into something that wipes every
password — a fix breaking the assumption the previous fix made. `POST /settings/global/datatable_roles`
could also empty it outside the lock.
The approved plan offered a table or `global_settings`, so this is the other option it already
allowed rather than a new design. `datatable_role` is a table: no generic settings path can read
it, list it, export it, write it or round-trip it, so none of the five needs a guard. The
redaction, the hidden/protected/agent-denylist entries and the JSON document all go with it.
One row per role also removes the read-modify-write the concurrency work was about: two
concurrent creates are two inserts, and the unique index on `name` is what settles a collision.
The advisory lock stays for the one window rows do not cover — `CREATE ROLE` is invisible to
another transaction until commit, so without it both creates pass their `pg_roles` check.
Also from this round: rename mappings are checked against the configuration they claim to
describe, since fork pointers are rewritten from them — a caller could otherwise submit
`main -> missing` against an unchanged config and repoint every fork of `main` at a name nothing
has, and `A -> B` plus `B -> C` moved what pointed at `A` all the way to `C`. And the warning
naming forks a delete stranded reached the response but not the screen: both the data table
settings save and the workspace delete now show it.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012ti5HyeTikPMYyW8YSdiHR
* fix(datatables): validate a rename against the save it describes, and re-check under the locks
Three from the round, all about deciding on state that could already have moved.
A permission save resolved the data table and checked it was instance-backed before taking any
lock, then wrote under one. A config save committing in between could move the table onto a
PostgreSQL resource — recreating exactly what the transition guard refuses — or rename it, in
which case the write targeted a key that no longer existed and reported success having changed
nothing. It now re-resolves and re-checks on the locked state.
Rename validation checked that the source existed before and the target existed after, which
still accepts `main -> decoy` against a save that keeps both: every fork of `main` then follows
onto a different data table, silently, because it keeps resolving. The rule is now the actual
old-to-new key transition — a source may only survive if another rename took its name, and a
target may only pre-exist if another rename freed it. That also stops two sources sharing one
target, and it admits a swap, which the previous guard refused: `datatables` is keyed by name, so
a swap cannot be done one save at a time, and refusing it was a regression against main. The
pointer cascade now runs in two passes through a temporary name, the way the migration cascade
one layer down already handles the same shape, so `A -> B` with `B -> C` moves each pointer once
from what it named before the save.
The tenant mutators say what they are for: they write an access decision for any workspace named,
with an arbitrary mutation, and exist for the transaction that frees or renames a principal.
Editing a decision on purpose belongs in the permissions endpoint.
Carried in the same change: the stranded-fork list is a field rather than a phrase to grep out of
a success string; the pointer cascade matches with `EXISTS` instead of a `LIKE` over the whole
document, so a workspace whose pointers name something else is not rewritten to a byte-identical
value under an exclusive lock; and `InstanceDatatableRole` drops the serde derives left over from
the JSON document, one of which would emit `pwd`.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012ti5HyeTikPMYyW8YSdiHR
* fix(datatables): cascade on the leave route that is used, gate migrations before the admin connection, and drop a role atomically
The tenant cascade on leaving went onto `/users/leave`. The UI and the generated client call
`/workspaces/leave` — a different handler in a different crate with the same name — which
deleted the membership and left `u/<username>` in the tenant lists. Leaving and rejoining
therefore restored the access the leave was supposed to end, and a later account taking the
username would have inherited it. The regression test drives the route the client actually
calls; without the fix it fails with "leaving kept the tenant".
The migration endpoints authorized too late. `run_datatable_migrations` opened the data table's
admin connection, created `_wm_migrations` and read it before reaching the per-migration role
check — so with nothing pending, nothing was checked at all. Rollback returned before its check
when nothing was applied, and the status endpoint had none. All three now ask, before any
connection is opened, whether the caller can reach the data table as any role at all; which role
a given migration runs as is still decided per migration, and by the executor after that.
Deleting a role committed the cluster drop and the catalog row, then swept the tenant lists in
separate transactions. A sweep failing part-way left workspaces naming a role nothing can connect
as, while the retry answered `NotFound` because the catalog entry was already gone. The sweep now
runs in the same transaction, so the drop, the row and every tenant list commit together.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012ti5HyeTikPMYyW8YSdiHR
* fix(datatables): refuse to copy a data table that is under roles
pg_dump carries no roles and the import runs with --no-privileges, so a copied
data table arrives owned by the admin connection with no GRANT for any role.
The settings clone brings `permissions` across, so the fork's tenants pass
Windmill's check, connect as the role they were given, and are denied by
Postgres on everything: an entry that reads as configured and answers nothing.
Refuse the copy — in the import endpoint before any data moves, and in the fork
path the CLI takes. Replaying the source's owners and ACLs into the clone is
what lifts this, and is a change of its own. Dropping `permissions` from the
copy instead would be the unsafe half, since the copy holds the parent's rows.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012ti5HyeTikPMYyW8YSdiHR
* fix(datatables): refuse the clone's database too, not only its data
A clone is two endpoints: `create_pg_database` then `import_pg_database`. Only
the second refused a data table under roles, so a fork asking to clone one
created and registered an empty `wm_fork_…` instance database and then failed —
and nothing collects it, since `drop_forked_datatable_databases` only drops
entries carrying `forked_from` and no entry names this one.
Refuse in both, so the clone stops before a database exists.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012ti5HyeTikPMYyW8YSdiHR
* nit worker error msg
* fix pg_dump stuck on version 17 on nix
* fix(datatables): refuse a malformed role annotation instead of ignoring it
`-- Role operator`, `-- role operator;` and `-- role operator -- why` all failed
the annotation parser's exact-match rule, so the query fell through to the data
table's default role and ran, silently, under a login the author did not choose.
Naming a role exists precisely to not do that.
A leading comment whose first word is `role` is now an annotation attempt: the
keyword matches case-insensitively, one trailing `;` is tolerated, and anything
else is an error naming the line. Only callers that already know the target is a
`datatable://` reference ever run this, so ordinary SQL keeps its comments.
Also bumps the dev shell's postgres client to 18 — it trailed the server the dev
database runs, which takes out every data table export, clone and fork-with-data.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012ti5HyeTikPMYyW8YSdiHR
* fix(datatables): refuse a malformed role query string instead of ignoring it
`?Role=analytics`, `?role=` and `?x=1&role=…` all fell through the reference
parser's exact-match rule, so the connection resolved to the data table's default
role and ran under a login the caller never asked for — the URI half of the same
trap as a malformed `-- role` annotation.
The key now matches case-insensitively, and anything else in the query string is
an error naming it; `role` is the only parameter a reference takes. Callers that
only need the entry keep a lenient `datatable_ref_name`, since they never act on
the role. The DuckDB `ATTACH` parser propagates it rather than attaching under
the default.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012ti5HyeTikPMYyW8YSdiHR
* fix(datatables): carry the role annotation into the row_to_json retry
The retry rebuilds its SQL from `pruneComments(code)`, so the leading comment
block never reached the second attempt — and with it the `-- role <name>` line
that decides which login the query runs as. The retry connected as the data
table's default role instead, so a query the first attempt was denied could
succeed on the second, reported as "recovered with the row_to_json fix".
Carry the leading comment block over. The retry itself is unchanged.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BjfMkJyKzodxkobqGZ6Lqb
* chore(datatables): don't mount the roles UI until the ACL editor lands
Enforcement ships first. The permissions drawer is what turns roles on, and the
catalog section is what creates them — both are only useful once there is a way
to grant a role the privileges it needs, which arrives with the ACL editor. Left
mounted they would offer a feature whose other half does not exist.
The two components are complete and reviewed; only their call sites here are
commented out, with a note pointing the follow-up PRs at them.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BjfMkJyKzodxkobqGZ6Lqb
* fix(datatables): honour `-- role: x`, and fix the DuckDB attach test
Two review findings, both real.
`attach_datatable_parses_name_and_role` never compiled: `parse_attach_datatable`
returns `Result<Option<_>>` now and one call site kept a single `unwrap`. Its
`?Role=analytics` case also asserted a refusal, contradicting the parser in the
same commit, which matches the key case-insensitively. Replaced with the cases
that are genuinely malformed, and a positive one for the cased key.
`-- role: analytics` fell through to the default role — the silent fallback the
strict parser exists to remove, for the spelling most likely to be typed. The
keyword now accepts an optional colon, attached or spaced, while a word that
merely starts with it (`rolebased`) is still not an attempt.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BjfMkJyKzodxkobqGZ6Lqb
* fix(datatables): clone a fork's pointer instead of failing after the copy
Forking a fork with cloning left an orphan database. The preflight resolves the
pointer and sees the governing entry, so both endpoints ran and filled the new
database; `apply_forked_datatable` then refused the inherited pointer and rolled
the fork back, stranding a registered `wm_fork_*` that no entry names and whose
name blocks the retry.
Refusing earlier would have been the smaller change, but forking a fork and
cloning worked before pointers existed, so it would trade an orphan for a
regression. Resolve what the pointer names and write the terminal entry the
clone needs: the whole `database` object rather than a patch of its
`resource_path`, since a pointer has none, and `reference` removed with it.
Also accepts `-- role=x` and `-- Role = x`, two more spellings that fell through
to the default role.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BjfMkJyKzodxkobqGZ6Lqb
* fix(datatables): refuse to roll back the catalog while roles exist
The down migration dropped the table and left every role behind: live Postgres
logins whose passwords only that table carried, so after a revert Windmill could
neither use, disable nor delete them, and re-applying could not recreate them
because the names were taken. Cleaning up here is not possible either — dropping
a role means reassigning what it owns in every instance database, and a
migration runs in one — so it now refuses while the catalog is non-empty and
says to delete the roles through instance settings, which does the cluster work.
Also enforces the instance-only invariant the resolved-pointer clone relies on
rather than only asserting it in a comment.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BjfMkJyKzodxkobqGZ6Lqb
* refactor(datatables): settle clonability in one place, before anything is created
A clone is three stages a workspace apart — `create_pg_database`, then
`import_pg_database`, then `apply_forked_datatable` inside the fork transaction.
Only the third can roll back, and `CREATE DATABASE` is not transactional, so any
refusal that lives there strands a registered `wm_fork_*` that no entry names
and whose name blocks the retry.
That orphan has now been fixed three times, most recently reintroduced by a
guard added one commit ago. Patching each new refusal into the first endpoint is
not the fix; having two places that can refuse is. `ensure_datatable_is_clonable`
now answers every reason a copy can be refused and returns what it resolved, and
the stage that writes the entry only does the work.
Also takes an ACCESS EXCLUSIVE lock before the rollback guard counts, so a role
created concurrently cannot slip between the check and the drop.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BjfMkJyKzodxkobqGZ6Lqb
* fix(datatables): let a retried clone reclaim its own leftover database
A clone creates its target database one request before it copies into it, and
the fork that would name it is written a request after that. Any failure in
between — a pg_dump error, a bad restore, a dropped connection, the source's
roles changing mid-flow — left a registered `wm_fork_*` that no entry names,
and every retry then failed on its name. This predates data table roles.
`create_pg_database` now reclaims such a leftover before creating: only a
`wm_fork_*` database Windmill registered as a data table database and that no
data table or ducklake entry names, in any workspace, archived ones included.
The drop never terminates connections, so a clone still copying into it makes
the reclaim fail instead of being cut off. It is limited to callers who
administer the source — reaching it is not enough, since on a data table
without roles every member reaches it — and anyone else gets the refusal an
existing database always got.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* Revert "fix(datatables): let a retried clone reclaim its own leftover database"
This reverts commit 7dd3275a10.
The reclaim tied the caller to the source they administer, but not to the
database it dropped. Between another workspace's import and its final fork
request, that workspace's target is full, registered, unnamed and has no open
connection, so an admin of any instance data table could name it and have it
dropped and recreated empty. The victim's fork would then commit pointing at
the empty copy. Safe reclaim needs durable clone ownership and serialization
with the request that names the database; until then the leftover stays, as it
did before this PR.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* docs(datatables): record the stale clone database as a known limitation
A clone is three requests and `CREATE DATABASE` is not transactional, so a
failure after the first leaves a registered `wm_fork_*` behind, as it did
before data table roles. Accepted for this PR: it is harmless to data and goes
away once the clone is a single server-side operation.
The comment also records why the obvious fix is wrong: reclaiming the leftover
on retry, without durable clone ownership, can drop another workspace's fully
copied database between its import and its final fork request.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* fix(datatables): bounce the streams reading a data table when it is deleted
Deleting a governing data table, or the workspace that holds it, only collected
the fork pointers it stranded, for the warning. A Postgres trigger or capture
already streaming through one of those pointers kept the replication connection
it opened while the pointer still resolved, so it went on dispatching the
governing database's rows after the fork lost access — until its connection
happened to restart. The governing workspace's own streams on a deleted entry
did the same.
Both deletion paths now bounce the affected listeners inside their own
transaction, through the helper a permission change already uses, so a
listener that reconnects re-resolves the entry and finds it gone. The helper is
split so a caller can pass the (workspace, local name) pairs it already holds.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* fix(datatables): keep the fork schema baseline, and bounce streams on every removal
Three fixes from review.
`edit_datatable_config` took `forked_from` wholesale from the stored entry, so
the fork schema diff's save of an advanced baseline was silently discarded and
an applied change was offered again. Whether an entry carries a clone stamp is
still carried from the store, since that is what marks its database droppable,
but the baseline inside it is now taken from the request.
The stranded-pointer warning and the stream bounce ran over the optional
`deleted_datatables` hint, which the settings-sync CLI never sends, so removing
a governing data table through `wmill` bounced nothing. Removals are now derived
from the stored configuration against the saved one.
`delete_workspace` read the pointers to bounce before its transaction, so a fork
committing a pointer during the deletion was missed. The read now happens inside
the transaction, after the workspace row is deleted: a fork's insert key-share
locks that row through its parent foreign key, so it is either seen or fails on
the missing parent.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* refactor(datatables): keep Postgres triggers and data table roles apart
A replication stream reads every row of every table whatever the data table's
roles grant, and its listener checks access only when it connects. Rather than
chase every way access can change and bounce the streams each one affects, a
data table now carries one or the other:
- a Postgres trigger or capture cannot be created on, or connect to, a data
table under roles;
- roles cannot be turned on while an enabled trigger or a live capture reads
the data table, its own or a fork's through its pointer. The refusal names
each one to disable.
This removes the stream bounces on roles edits and on data table and workspace
deletion, and the trigger gate that admitted admins. The fork schema baseline
fix from the same review round is kept.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* fix(datatables): refuse a Postgres trigger on a data table under roles when it is saved
Creating or editing a trigger that points at a data table under roles was
accepted, and its listener then retried the refused connection every 30
seconds forever. The save is now refused, and a trigger that reaches such a
data table anyway (re-enabled, or cloned into a fork) is disabled by its
listener with the reason, as a missing replication slot is.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* fix(datatables): disable a data table role before deleting it
Deleting a role reassigns and drops what it owns in each registered
database on its own connection, and each of those passes commits as it
goes. A database failing part-way left the role enabled in the catalog and
able to log in, but already stripped in the databases reached before it.
The role is now disabled in its own commit first, so a failed delete
leaves a disabled role to retry.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* fix(datatables): serialize roles going on with a stream starting
Turning roles on looked for enabled triggers and live captures once,
without a lock anything starting a stream also took. A trigger enabled in
that window could have its listener connect before roles committed, and a
healthy listener never checks again. Both transitions now serialize on one
advisory lock: roles going on hold it exclusive while they look, and
trigger create, edit and enable, and capture setup and ping hold it shared
while they commit. Either the look sees the stream, or the listener
connects after roles are committed and refuses.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* fix(datatables): wait out live listeners, and resolve stored names containing `?`
Turning roles on counted a trigger as gone once disabled, and a capture
once its client stopped pinging, but the listener keeps its replication
connection until its next heartbeat notices. A trigger or capture whose
listener pinged in the last 15 seconds, the window a server holds a
listener for, now still counts as streaming.
Data table names could contain `?` before they were restricted, and such
entries are still stored. Splitting `?role=` off a reference misread them:
`a?b` became `a` with an unknown parameter, and the clone checks looked at
a different entry than the one copied. An entry stored under the whole
reference is now looked up first, in the Postgres executor, DuckDB ATTACH
and the clone checks. Agent workers cannot read the workspace and keep
the strict parse, which refuses such a name rather than misreading it.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* fix(datatables): warn when a settings sync strands fork pointers
A settings save reported the fork pointers left resolving to nothing only
for the names in `deleted_datatables`, which `wmill sync push` never sends.
The save now works out what it removed from the locked entries, and the
CLI prints the stranded pointers it returns.
Also correct the replication helper's contract: no role or admin check
makes a replication connection safe, so a data table under roles is
refused outright rather than gated as an admin operation.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BjfMkJyKzodxkobqGZ6Lqb
* fix(datatables): refuse a save that drops a data table's roles through an undeclared rename
A data table's roles follow its entry only through a declared rename. A
settings sync sends the whole map and never declares one, so renaming a
data table under roles there read as a delete and a new entry on the same
database: the new entry carried no roles, and every caller connected as
admin. Such a save is now refused, naming both entries.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BjfMkJyKzodxkobqGZ6Lqb
* fix(datatables): no entry without roles may newly reach a database under roles
The previous guard only caught a new name replacing an entry under roles.
A whole-map save could also repoint an existing entry without roles at
that database, or another workspace could point one there, and every
caller of that entry would connect as admin. The rule is now stated on
the saved entries: one that carries no roles and newly points at an
instance database any entry under roles uses, in this workspace or
another, is refused. A declared rename carries its roles and passes.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BjfMkJyKzodxkobqGZ6Lqb
* feat(datatables): move data table role catalog and resolution to the enterprise edition
Roles are an Enterprise Edition feature. The catalog, the Postgres logins,
CONNECT convergence, tenant evaluation and the role half of connection
resolution move to windmill-ee-private. Every public function keeps its path
and signature and forwards through datatable_roles_oss, which re-exports the
enterprise implementation or, without it, refuses.
Without the enterprise edition a data table under roles, or a caller naming a
role, is refused a connection rather than resolved as admin, and the reach and
admin-access checks refuse one under roles. A data table not under roles
resolves as before in every edition, and an instance database keeps the
CONNECT grants it was created with. The catalog lock, the stream lock, the
tenant cascades and the permissions stripping stay in OSS: they only restrict.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BjfMkJyKzodxkobqGZ6Lqb
* feat(datatables): move the data table permissions endpoints to the enterprise edition
The permissions read, save and usable-roles handlers move to
windmill-ee-private; the routes stay registered and, without the enterprise
edition, answer that data table roles are an Enterprise Edition feature.
ensure_governs_datatable and ensure_reaches_datatable keep their paths: the
first refuses, the second passes a data table not under roles.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BjfMkJyKzodxkobqGZ6Lqb
* feat(datatables): move the data table role catalog endpoints to the enterprise edition
The superadmin list, create, update and delete handlers move to
windmill-ee-private. The routes stay registered and, without the enterprise
edition, refuse after authentication.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BjfMkJyKzodxkobqGZ6Lqb
* test(datatables): run the roles tests on the enterprise edition, refusals without it
Each test that exercises roles runs with private and enterprise. Two tests run
without them: every roles route answers the Enterprise refusal, and a data
table saved under roles, or a named role, is refused a connection while one
not under roles resolves as before.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BjfMkJyKzodxkobqGZ6Lqb
* feat(datatables): gate the roles UI mount sites on an enterprise license
Both mount sites are still commented out; the gate travels with them.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BjfMkJyKzodxkobqGZ6Lqb
* test(datatables): run the tenant matcher test on the enterprise edition
The matcher it covers is enterprise code now, so without the enterprise
edition the test hit the stub and failed the default windmill-common run. It
runs with private and enterprise, and a counterpart without them asserts that
no tenant list covers anyone, the wildcard and a workspace admin included.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BjfMkJyKzodxkobqGZ6Lqb
* chore: update ee-repo-ref to a1873dbb67f2302b85ff5362f8387b48eccdb607
This commit updates the EE repository reference after PR #783 was merged in windmill-ee-private.
Previous ee-repo-ref: 5c853e2c20eca6b748415fc0d6862a6ebfb5fec4
New ee-repo-ref: a1873dbb67f2302b85ff5362f8387b48eccdb607
Automated by sync-ee-ref workflow.
* fix(datatables): refuse roles while a same-workspace alias reaches the database
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* fix(datatables): let CE migrations connect as an explicitly named admin
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* fix(datatables): serialize roles going on with aliases saved from other workspaces
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* docs(datatables): note that legacy names with ? cannot be migrated
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* feat(datatables): add an ACL editor for data table roles
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* fix: take every pooled connection before the ACL apply locks
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BRoYE5ZeAVvrDYdfhDAYXb
* fix: refresh grant options only after the ACL apply validates its plan
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BRoYE5ZeAVvrDYdfhDAYXb
* fix: add only missing grant options before an ACL apply, never default privileges
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BRoYE5ZeAVvrDYdfhDAYXb
* fix: run one data table ACL apply at a time per server before it connects
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BRoYE5ZeAVvrDYdfhDAYXb
* fix: hold the ACL connection to the database that was authorized
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BRoYE5ZeAVvrDYdfhDAYXb
* fix: build the ACL connection from the authorized data table entry
Resolving the settings again could land on a resource with the same
database name on another server, which the later entry checks never see.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BRoYE5ZeAVvrDYdfhDAYXb
* fix: check ACL read reach against the entry it connects from
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BRoYE5ZeAVvrDYdfhDAYXb
* feat(datatables): Ducklake catalogs on the external instance cluster
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* fix(datatables): never grant CREATEROLE to custom_instance_user on the external cluster
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* docs(datatables): state the authorization contract of external database usage lookups
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* fix(datatables): drop a DuckDB data table secret once its ATTACH has used it
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* perf(datatables): resolve a workspace's data tables per pointer hop, not per entry
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* fix(datatables): hold the parent's settings while a fork points at its data tables
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* fix(datatables): refuse repointing the external cluster while it is in use, and keep verify-ca working for pg_dump
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* fix(datatables): protect external databases pending fork cleanup, and describe Ducklake usage in the API
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* feat(datatables): per-cluster data table role catalogs, with roles on the external instance cluster
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* fix(datatables): write the external cluster setting under the lifecycle lock, and check fork targets are registered
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* fix(datatables): register external fork catalogs under the lifecycle lock, and keep certificate verification in DuckDB attaches
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* fix(datatables): keep certificate verification when DuckDB attaches an external data table
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* fix(datatables): keep certificate verification when DuckDB attaches an external data table
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* fix(datatables): refuse fork cleanup of an external database another workspace uses
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* fix(datatables): refuse rolling back while external data tables are under roles, and type external_instance in the CLI
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* fix(datatables): stop counting storage-only fork cleanup rows as uses of an external database
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* fix(datatables): bind fork database copies to their workspace, and count every use before dropping one
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* fix(datatables): authenticate instance database setup before writing its status, and keep a fork reservation across it
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* fix(datatables): create external databases only on a cluster setup succeeded on, and document the registry reader
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* fix(datatables): keep only the most recently used DuckDB root certificate files
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* fix(datatables): migrate fork reservations on workspace rename, and lock the parent's data tables for the whole fork
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* fix(datatables): take the fork data table lock once, before the external cluster's
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* docs(datatables): describe the external instance cluster and how to run one locally
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* fix(datatables): keep fork reservations private, drop a cleaned-up entry with its database, and serialize cleanup with settings saves
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* fix(datatables): hold the fork lock across a fork import, and carry the reservation inside the setup write
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* fix(datatables): check the external cluster setting on its own transaction, and gate the registry probe
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* feat(datatables): add an ACL editor for data table roles
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* fix: take every pooled connection before the ACL apply locks
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BRoYE5ZeAVvrDYdfhDAYXb
* fix: refresh grant options only after the ACL apply validates its plan
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BRoYE5ZeAVvrDYdfhDAYXb
* fix: add only missing grant options before an ACL apply, never default privileges
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BRoYE5ZeAVvrDYdfhDAYXb
* fix: run one data table ACL apply at a time per server before it connects
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BRoYE5ZeAVvrDYdfhDAYXb
* fix: hold the ACL connection to the database that was authorized
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BRoYE5ZeAVvrDYdfhDAYXb
* fix: build the ACL connection from the authorized data table entry
Resolving the settings again could land on a resource with the same
database name on another server, which the later entry checks never see.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BRoYE5ZeAVvrDYdfhDAYXb
* fix: check ACL read reach against the entry it connects from
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BRoYE5ZeAVvrDYdfhDAYXb
* chore: update ee-repo-ref to 7e338e4dabf91689bfd7fb0333c6534040b17b59
This commit updates the EE repository reference after PR #787 was merged in windmill-ee-private.
Previous ee-repo-ref: 0edd40979cf36bfba59323f3f6a0811ae1369cf5
New ee-repo-ref: 7e338e4dabf91689bfd7fb0333c6534040b17b59
Automated by sync-ee-ref workflow.
* fix(datatables): keep DuckDB root certificate files in the job directory
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* chore: drop the unused json import from the settings crate
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* fix(datatables): drop the serde_json::json import left unused by the fork setup write
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* fix(datatables): accept external_instance data tables in the settings form type
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* fix(datatables): serialize fork database reservations, cleanup, setup and Ducklake saves on the database
- A fork copy is created and registered under its workspace's fork lock, and refused once the
workspace is archived; a rename re-migrates reservations under that lock after archiving.
- Ducklake saves lock every instance database they newly name, as data table saves do.
- Instance database setup holds the database's lock until its entry is written.
- Fork import also holds the database's lock across the restore.
- Cleanup re-reads the entry under its locks before dropping anything.
- Non-superadmins no longer see fork copies reserved for workspaces they are not in.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* fix(datatables): state the authorization contract of the external cluster status and write check
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* fix(datatables): lock the fork copy at finalization, and order cleanup's registry write after its settings row
- Fork finalization holds the copy's database lock from its availability check to the commit.
- Cleanup removes the registry entry in its own transaction, in a task of its own, instead of on a
second connection; a rename migrates reservations after its settings rewrites, so both take the
settings rows before the registry.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* fix(datatables): hold the fork lock across a rename's settings copy
Fork cleanup of the old id could otherwise drop a copy the renamed workspace goes on using. Also
states the authorization contract of the instance database drop helpers.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* fix(datatables): refuse moving the external cluster to another host or port while it holds databases
Every settings writer goes through the same check as removal: the login, TLS and maintenance
database may still change, the cluster may not.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* feat(datatables): clone a data table under roles with its owners and grants (#11120)
Claude-Session: https://claude.ai/code/session_01UbrtwiYNfayrmqouBJHwGV
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
* fix: open the raw app data table drawer when the workspace has none
Selecting the first data table of an empty list passed undefined to the name
check, which threw instead of opening the drawer on no data table.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* fix(datatables): offer cloning a data table under roles where its grants can be replayed
The server clones such a data table and replays the source's owners and grants,
which only the Enterprise Edition does, so the fork wizard hid both clone
options everywhere instead of on a build that cannot replay them.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* docs: name the placeholder the empty raw app data drawer renders
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* fix(datatables): replace a job directory file at the DuckDB root certificate path instead of trusting it
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* test: pin the enterprise refusal the role pickers read as 'not under roles'
The server's sentence and the frontend's copy of it were coupled by nothing,
so rewording either one turned every role picker on a community build into a
failed lookup.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* fix(datatables): type Ducklake catalogs on the external instance cluster in the settings form
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* fix(datatables): read a roles answer only for the workspace it was asked in
A fork and its parent each have their own roles on a data table of the same
name, so an answer stamped with the name alone settled the role from the
workspace the editor was acting on before.
Also derive the AI table creation flag from the data replaced into the editor:
data naming no data table left the flag on from before.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* fix(datatables): keep an instance database a settings save is waiting to name
Cleanup for a database whose setup failed took the lock first, read no user,
and dropped it while a save blocked on that same lock was about to commit a
reference to it.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* fix(datatables): check the workspace stamp in the default database selector too
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* fix(datatables): read what a save racing instance-database cleanup committed
A transaction blocked on the lock may still roll back, so keeping the database
for it stranded one whose name then blocks every retry: it is let through and
its outcome read instead. The waiter query also matches this database's locks
only, since pg_locks spans the cluster.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* fix(datatables): tell a waiting request apart from the workspaces using a database
Both callers render what cleanup returns as the workspaces that keep the
database, so a waiting request's pid read as one of them. Each now words that
case itself, and the give-up comment names where the kept name actually goes.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* fix(datatables): take the cleanup lock on a connection the pool cannot reclaim
A session lock outlives the future holding it, so a cancellation between
taking it and releasing it handed a locked session back to the pool, where
every later settings save waits on it. Detached, the connection closes when it
is dropped and the server releases the lock with the session.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* fix(datatables): close the cleanup connection on drop instead of detaching it
Detaching released the pool permit while the session stayed alive, so
concurrent cleanups waiting on their locks could open as many connections as
they liked. Closing on drop covers the same cancellation and keeps them
counted.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* chore: update ee-repo-ref to 179d2454a063ee818a9387a1eafdd354a217a16c
This commit updates the EE repository reference after PR #802 was merged in windmill-ee-private.
Previous ee-repo-ref: c2e43f5b5ff753d339b70e232fd17b6ffecb054d
New ee-repo-ref: 179d2454a063ee818a9387a1eafdd354a217a16c
Automated by sync-ee-ref workflow.
* chore: update ee-repo-ref to fd5b8af748f2c985b13e18d9ea30894f3bd7e9a3
This commit updates the EE repository reference after PR #798 was merged in windmill-ee-private.
Previous ee-repo-ref: 3145e422d61d580f0a82804f075285c112879da0
New ee-repo-ref: fd5b8af748f2c985b13e18d9ea30894f3bd7e9a3
Automated by sync-ee-ref workflow.
* fix(datatables): classify a fork import's target from the resolution that built its connection
A second read could see the entry flipped to a resource and skip the reservation check while the
connection already built still reached the instance cluster.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* fix(datatables): run external database lifecycle on the caller's transaction, one lock order for fork cleanup
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* fix(datatables): check a fork copy's reservation on the locked transaction's connection
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* fix(datatables): make and drop fork copies of external data tables on the external cluster
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* fix(datatables): refuse a fork whose external copy something already names
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* fix(datatables): check an external fork copy's uses before removing its entry, so child fork pointers still count
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* fix(datatables): migrate external cluster fork reservations on workspace rename
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* feat(datatables): configure the external instance cluster and pick its databases from the UI
Instance settings gets an External Postgres tab: the cluster's connection, the setup run
with its report, and the databases Windmill created there. Data table and Ducklake settings
offer the external_instance kind, with a picker over those databases.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* fix(datatables): keep the external cluster form out of the instance settings the page bulk-saves
The form seeded its key into the settings store after the page snapshotted them, so merely
opening the tab made the page send a setting the validator refuses, failing an admin's
unrelated save. The form is local state now, and only the setup writes the key.
The database picker also treats a superadmin-only listing as authoritative: a workspace
admin, who cannot list, no longer sees a saved database as missing.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* fix(tests): app paths are validated, so a guest app cannot be renamed to one with a space
Path validation on apps landed after this test, which still expected a space to be accepted
and created its fixture on a ':' path. The scopable-path guard it exists for is kept by
planting that path on the row, which is now the only way an app can hold one.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* feat(datatables): one Managed Postgres tab, and a switch for Windmill's own database
The instance settings tab covers both substrates Windmill administers: the external cluster,
which can now be disabled once nothing sits on it, and Windmill's own database, which an
operator can turn off so the cluster is the only one a workspace may newly name. A save that
names it then refuses, as the create endpoint does.
Workspace settings name them the way an admin meets them: Postgres Resource, and Managed
instance, qualified as Internal or External only while both are on offer.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* feat(datatables): offer the external cluster in the add-data-table wizard, and give Ducklake its room back
The wizard gets the external cluster as a fourth substrate: pick a database it already holds
or name a new one, which the run creates before writing the entry, so Try again does not trip
on a database the last attempt made.
Ducklake's maintenance column moves into the settings popover next to the extra args, which
the wider catalog select had squeezed the name box out of.
A setting a component saves itself now also moves the page's baseline, or Discard would
restore what it replaced and a later save would send that stale value back.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* fix(datatables): only a database this wizard run made makes its retry idempotent
A retry skipped the create whenever the name was registered, which let a run adopt a database
another superadmin had made and share their data without the warning the existing-database
branch shows. The run tracks what it created instead, and reports it so Discard can say the
database is still there. Creating one now refreshes the registry the next run's default name
and its validation read, and that default counts the external cluster's databases rather than
the instance's.
Row ownership carries the substrate: "instance" and "external_instance" can hold the same name,
and a row repointed between them while a run probes must not read as that run's own.
Also fixes the merge leftovers in the DuckDB executor's tests, which cargo check never compiles:
the new PgDatabase field and the attach helper's job directory.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* fix(datatables): shorten the picked managed kind, and say what an empty database list means
The closed select has room for the qualifier, not the whole name, so a picked managed kind
reads "Managed (Internal)". An external entry gets the tooltip its internal counterpart has,
and a database picker with nothing in it says to type a name rather than "No items found".
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* fix(datatables): keep the wizard's external-database bookkeeping optional and parked
The run deps gained a required field, which every existing caller of runSetup -- the sibling
test suite included -- does not pass. It defaults to empty instead, the parked payload carries
it across a Supabase redirect as it claims to, and a resumed run counts it as something left
behind. Two regressions pin the create: made on a first attempt, skipped only for a database
this run made.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* fix(datatables): review an external-cluster database as one, not as a Postgres resource
The review step fell through to the resource branch for the new provider, so it announced a
connection already in the workspace for a database that does not exist yet.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* fix(datatables): show the server's refusal instead of an object in the toast
Saving data table settings passed the whole error to the toast, which rendered the request
and response as JSON, and the instance-database wizard replaced it with "check console".
Both surface the message the API sent, through the helper that exists for it.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* chore: point at the EE commit dropping the unused import
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* fix(datatables): define a role on the cluster its data table sits on
The permissions drawer created and listed roles without naming a cluster, so a role defined
from an external data table became a login on Windmill's own, and the refresh then replaced
the correct catalog with the internal one. The drawer passes the data table's cluster to both.
The wizard also read a database it had just created as a name collision, which blocked
returning to the failed attempt and finishing under the same name.
And the merge had left the CONNECT-grant pass and its log in both the caller and the callee.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* feat(datatables): manage the external cluster's role catalog, and name the cluster in the copy
The roles drawer offered one catalog, so the external cluster's roles could only be reached
through a data table sitting on it — and became unreachable once the last one was gone, while
the cluster could not be unset until they were dropped. Opened from the page it now offers the
clusters the instance has; a data table's own drawer still pins its cluster, and its copy names
that cluster rather than "the instance".
A DuckDB attach also decides whether to verify certificates the way every other Postgres
connection does, so a resource carrying a root certificate is no longer downgraded to require
here alone.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* fix(datatables): bind the roles list to the cluster it was read for
Switching catalogs left the previous cluster's rows on screen and applied whichever response
landed last, so a slower read could seat one cluster's logins under the other's heading while
every control acted on the wrong id — invisible where a name exists on both. The switch clears
the rows, a token discards a response the selection has moved past, and the controls stay inert
while a catalog is being read.
The drop confirmation also names the cluster whose databases it reaches.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* feat: show data tables as cards and enable the add wizard by default
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
* feat: offer managed vs resource in the data table wizard, with external setup hints
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
* feat: move the external database hint next to the wizard's primary action
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
* feat: open managed postgres settings in place, prefilled from the picked resource
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
* feat: preview data table usage in python, duckdb and typescript in the wizard review
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
* feat: lay a data table card's fields out on one row, with manage set apart
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
* fix: hide data table grants while the data table is not under roles
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
* fix: default a new data table to the workspace name, and frame the usage preview
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
* feat: warn when disabling the internal instance while data still sits on it
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
* feat: disable the unavailable cluster in the instance roles toggle
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
* feat: make the data table manage buttons accent
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
* fix: stop resolving data tables on windmill's database once it is disabled
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
* feat: hint superadmins that typing in the add-role picker creates a role
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
* style: border the assets page cards instead of elevating them
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
* feat: confirm data table deletions on save, and show no success when a save is called off
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
* test: pin that instance data tables stop resolving once disabled
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
* fix: give the import wizard real managed availability, and lock the roles catalog when no cluster is on offer
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
* feat: mark the managed database as recommended in the data table wizard
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
* feat: skip the data table wizard's first step when a resource is the only option
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
Co-authored-by: windmill-internal-app[bot] <windmill-internal-app[bot]@users.noreply.github.com>
* fix pg_dump stuck on version 17 on nix
* fix(datatables): refuse a malformed role annotation instead of ignoring it
`-- Role operator`, `-- role operator;` and `-- role operator -- why` all failed
the annotation parser's exact-match rule, so the query fell through to the data
table's default role and ran, silently, under a login the author did not choose.
Naming a role exists precisely to not do that.
A leading comment whose first word is `role` is now an annotation attempt: the
keyword matches case-insensitively, one trailing `;` is tolerated, and anything
else is an error naming the line. Only callers that already know the target is a
`datatable://` reference ever run this, so ordinary SQL keeps its comments.
Also bumps the dev shell's postgres client to 18 — it trailed the server the dev
database runs, which takes out every data table export, clone and fork-with-data.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012ti5HyeTikPMYyW8YSdiHR
* fix(datatables): refuse a malformed role query string instead of ignoring it
`?Role=analytics`, `?role=` and `?x=1&role=…` all fell through the reference
parser's exact-match rule, so the connection resolved to the data table's default
role and ran under a login the caller never asked for — the URI half of the same
trap as a malformed `-- role` annotation.
The key now matches case-insensitively, and anything else in the query string is
an error naming it; `role` is the only parameter a reference takes. Callers that
only need the entry keep a lenient `datatable_ref_name`, since they never act on
the role. The DuckDB `ATTACH` parser propagates it rather than attaching under
the default.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012ti5HyeTikPMYyW8YSdiHR
* fix(datatables): carry the role annotation into the row_to_json retry
The retry rebuilds its SQL from `pruneComments(code)`, so the leading comment
block never reached the second attempt — and with it the `-- role <name>` line
that decides which login the query runs as. The retry connected as the data
table's default role instead, so a query the first attempt was denied could
succeed on the second, reported as "recovered with the row_to_json fix".
Carry the leading comment block over. The retry itself is unchanged.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BjfMkJyKzodxkobqGZ6Lqb
* chore(datatables): don't mount the roles UI until the ACL editor lands
Enforcement ships first. The permissions drawer is what turns roles on, and the
catalog section is what creates them — both are only useful once there is a way
to grant a role the privileges it needs, which arrives with the ACL editor. Left
mounted they would offer a feature whose other half does not exist.
The two components are complete and reviewed; only their call sites here are
commented out, with a note pointing the follow-up PRs at them.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BjfMkJyKzodxkobqGZ6Lqb
* fix(datatables): honour `-- role: x`, and fix the DuckDB attach test
Two review findings, both real.
`attach_datatable_parses_name_and_role` never compiled: `parse_attach_datatable`
returns `Result<Option<_>>` now and one call site kept a single `unwrap`. Its
`?Role=analytics` case also asserted a refusal, contradicting the parser in the
same commit, which matches the key case-insensitively. Replaced with the cases
that are genuinely malformed, and a positive one for the cased key.
`-- role: analytics` fell through to the default role — the silent fallback the
strict parser exists to remove, for the spelling most likely to be typed. The
keyword now accepts an optional colon, attached or spaced, while a word that
merely starts with it (`rolebased`) is still not an attempt.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BjfMkJyKzodxkobqGZ6Lqb
* fix(datatables): clone a fork's pointer instead of failing after the copy
Forking a fork with cloning left an orphan database. The preflight resolves the
pointer and sees the governing entry, so both endpoints ran and filled the new
database; `apply_forked_datatable` then refused the inherited pointer and rolled
the fork back, stranding a registered `wm_fork_*` that no entry names and whose
name blocks the retry.
Refusing earlier would have been the smaller change, but forking a fork and
cloning worked before pointers existed, so it would trade an orphan for a
regression. Resolve what the pointer names and write the terminal entry the
clone needs: the whole `database` object rather than a patch of its
`resource_path`, since a pointer has none, and `reference` removed with it.
Also accepts `-- role=x` and `-- Role = x`, two more spellings that fell through
to the default role.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BjfMkJyKzodxkobqGZ6Lqb
* fix(datatables): refuse to roll back the catalog while roles exist
The down migration dropped the table and left every role behind: live Postgres
logins whose passwords only that table carried, so after a revert Windmill could
neither use, disable nor delete them, and re-applying could not recreate them
because the names were taken. Cleaning up here is not possible either — dropping
a role means reassigning what it owns in every instance database, and a
migration runs in one — so it now refuses while the catalog is non-empty and
says to delete the roles through instance settings, which does the cluster work.
Also enforces the instance-only invariant the resolved-pointer clone relies on
rather than only asserting it in a comment.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BjfMkJyKzodxkobqGZ6Lqb
* refactor(datatables): settle clonability in one place, before anything is created
A clone is three stages a workspace apart — `create_pg_database`, then
`import_pg_database`, then `apply_forked_datatable` inside the fork transaction.
Only the third can roll back, and `CREATE DATABASE` is not transactional, so any
refusal that lives there strands a registered `wm_fork_*` that no entry names
and whose name blocks the retry.
That orphan has now been fixed three times, most recently reintroduced by a
guard added one commit ago. Patching each new refusal into the first endpoint is
not the fix; having two places that can refuse is. `ensure_datatable_is_clonable`
now answers every reason a copy can be refused and returns what it resolved, and
the stage that writes the entry only does the work.
Also takes an ACCESS EXCLUSIVE lock before the rollback guard counts, so a role
created concurrently cannot slip between the check and the drop.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BjfMkJyKzodxkobqGZ6Lqb
* fix(datatables): let a retried clone reclaim its own leftover database
A clone creates its target database one request before it copies into it, and
the fork that would name it is written a request after that. Any failure in
between — a pg_dump error, a bad restore, a dropped connection, the source's
roles changing mid-flow — left a registered `wm_fork_*` that no entry names,
and every retry then failed on its name. This predates data table roles.
`create_pg_database` now reclaims such a leftover before creating: only a
`wm_fork_*` database Windmill registered as a data table database and that no
data table or ducklake entry names, in any workspace, archived ones included.
The drop never terminates connections, so a clone still copying into it makes
the reclaim fail instead of being cut off. It is limited to callers who
administer the source — reaching it is not enough, since on a data table
without roles every member reaches it — and anyone else gets the refusal an
existing database always got.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* Revert "fix(datatables): let a retried clone reclaim its own leftover database"
This reverts commit 7dd3275a10.
The reclaim tied the caller to the source they administer, but not to the
database it dropped. Between another workspace's import and its final fork
request, that workspace's target is full, registered, unnamed and has no open
connection, so an admin of any instance data table could name it and have it
dropped and recreated empty. The victim's fork would then commit pointing at
the empty copy. Safe reclaim needs durable clone ownership and serialization
with the request that names the database; until then the leftover stays, as it
did before this PR.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* docs(datatables): record the stale clone database as a known limitation
A clone is three requests and `CREATE DATABASE` is not transactional, so a
failure after the first leaves a registered `wm_fork_*` behind, as it did
before data table roles. Accepted for this PR: it is harmless to data and goes
away once the clone is a single server-side operation.
The comment also records why the obvious fix is wrong: reclaiming the leftover
on retry, without durable clone ownership, can drop another workspace's fully
copied database between its import and its final fork request.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* fix(datatables): bounce the streams reading a data table when it is deleted
Deleting a governing data table, or the workspace that holds it, only collected
the fork pointers it stranded, for the warning. A Postgres trigger or capture
already streaming through one of those pointers kept the replication connection
it opened while the pointer still resolved, so it went on dispatching the
governing database's rows after the fork lost access — until its connection
happened to restart. The governing workspace's own streams on a deleted entry
did the same.
Both deletion paths now bounce the affected listeners inside their own
transaction, through the helper a permission change already uses, so a
listener that reconnects re-resolves the entry and finds it gone. The helper is
split so a caller can pass the (workspace, local name) pairs it already holds.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* fix(datatables): keep the fork schema baseline, and bounce streams on every removal
Three fixes from review.
`edit_datatable_config` took `forked_from` wholesale from the stored entry, so
the fork schema diff's save of an advanced baseline was silently discarded and
an applied change was offered again. Whether an entry carries a clone stamp is
still carried from the store, since that is what marks its database droppable,
but the baseline inside it is now taken from the request.
The stranded-pointer warning and the stream bounce ran over the optional
`deleted_datatables` hint, which the settings-sync CLI never sends, so removing
a governing data table through `wmill` bounced nothing. Removals are now derived
from the stored configuration against the saved one.
`delete_workspace` read the pointers to bounce before its transaction, so a fork
committing a pointer during the deletion was missed. The read now happens inside
the transaction, after the workspace row is deleted: a fork's insert key-share
locks that row through its parent foreign key, so it is either seen or fails on
the missing parent.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* refactor(datatables): keep Postgres triggers and data table roles apart
A replication stream reads every row of every table whatever the data table's
roles grant, and its listener checks access only when it connects. Rather than
chase every way access can change and bounce the streams each one affects, a
data table now carries one or the other:
- a Postgres trigger or capture cannot be created on, or connect to, a data
table under roles;
- roles cannot be turned on while an enabled trigger or a live capture reads
the data table, its own or a fork's through its pointer. The refusal names
each one to disable.
This removes the stream bounces on roles edits and on data table and workspace
deletion, and the trigger gate that admitted admins. The fork schema baseline
fix from the same review round is kept.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* fix(datatables): refuse a Postgres trigger on a data table under roles when it is saved
Creating or editing a trigger that points at a data table under roles was
accepted, and its listener then retried the refused connection every 30
seconds forever. The save is now refused, and a trigger that reaches such a
data table anyway (re-enabled, or cloned into a fork) is disabled by its
listener with the reason, as a missing replication slot is.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* fix(datatables): disable a data table role before deleting it
Deleting a role reassigns and drops what it owns in each registered
database on its own connection, and each of those passes commits as it
goes. A database failing part-way left the role enabled in the catalog and
able to log in, but already stripped in the databases reached before it.
The role is now disabled in its own commit first, so a failed delete
leaves a disabled role to retry.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* fix(datatables): serialize roles going on with a stream starting
Turning roles on looked for enabled triggers and live captures once,
without a lock anything starting a stream also took. A trigger enabled in
that window could have its listener connect before roles committed, and a
healthy listener never checks again. Both transitions now serialize on one
advisory lock: roles going on hold it exclusive while they look, and
trigger create, edit and enable, and capture setup and ping hold it shared
while they commit. Either the look sees the stream, or the listener
connects after roles are committed and refuses.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* fix(datatables): wait out live listeners, and resolve stored names containing `?`
Turning roles on counted a trigger as gone once disabled, and a capture
once its client stopped pinging, but the listener keeps its replication
connection until its next heartbeat notices. A trigger or capture whose
listener pinged in the last 15 seconds, the window a server holds a
listener for, now still counts as streaming.
Data table names could contain `?` before they were restricted, and such
entries are still stored. Splitting `?role=` off a reference misread them:
`a?b` became `a` with an unknown parameter, and the clone checks looked at
a different entry than the one copied. An entry stored under the whole
reference is now looked up first, in the Postgres executor, DuckDB ATTACH
and the clone checks. Agent workers cannot read the workspace and keep
the strict parse, which refuses such a name rather than misreading it.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* fix(datatables): warn when a settings sync strands fork pointers
A settings save reported the fork pointers left resolving to nothing only
for the names in `deleted_datatables`, which `wmill sync push` never sends.
The save now works out what it removed from the locked entries, and the
CLI prints the stranded pointers it returns.
Also correct the replication helper's contract: no role or admin check
makes a replication connection safe, so a data table under roles is
refused outright rather than gated as an admin operation.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BjfMkJyKzodxkobqGZ6Lqb
* fix(datatables): refuse a save that drops a data table's roles through an undeclared rename
A data table's roles follow its entry only through a declared rename. A
settings sync sends the whole map and never declares one, so renaming a
data table under roles there read as a delete and a new entry on the same
database: the new entry carried no roles, and every caller connected as
admin. Such a save is now refused, naming both entries.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BjfMkJyKzodxkobqGZ6Lqb
* fix(datatables): no entry without roles may newly reach a database under roles
The previous guard only caught a new name replacing an entry under roles.
A whole-map save could also repoint an existing entry without roles at
that database, or another workspace could point one there, and every
caller of that entry would connect as admin. The rule is now stated on
the saved entries: one that carries no roles and newly points at an
instance database any entry under roles uses, in this workspace or
another, is refused. A declared rename carries its roles and passes.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BjfMkJyKzodxkobqGZ6Lqb
* feat(datatables): move data table role catalog and resolution to the enterprise edition
Roles are an Enterprise Edition feature. The catalog, the Postgres logins,
CONNECT convergence, tenant evaluation and the role half of connection
resolution move to windmill-ee-private. Every public function keeps its path
and signature and forwards through datatable_roles_oss, which re-exports the
enterprise implementation or, without it, refuses.
Without the enterprise edition a data table under roles, or a caller naming a
role, is refused a connection rather than resolved as admin, and the reach and
admin-access checks refuse one under roles. A data table not under roles
resolves as before in every edition, and an instance database keeps the
CONNECT grants it was created with. The catalog lock, the stream lock, the
tenant cascades and the permissions stripping stay in OSS: they only restrict.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BjfMkJyKzodxkobqGZ6Lqb
* feat(datatables): move the data table permissions endpoints to the enterprise edition
The permissions read, save and usable-roles handlers move to
windmill-ee-private; the routes stay registered and, without the enterprise
edition, answer that data table roles are an Enterprise Edition feature.
ensure_governs_datatable and ensure_reaches_datatable keep their paths: the
first refuses, the second passes a data table not under roles.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BjfMkJyKzodxkobqGZ6Lqb
* feat(datatables): move the data table role catalog endpoints to the enterprise edition
The superadmin list, create, update and delete handlers move to
windmill-ee-private. The routes stay registered and, without the enterprise
edition, refuse after authentication.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BjfMkJyKzodxkobqGZ6Lqb
* test(datatables): run the roles tests on the enterprise edition, refusals without it
Each test that exercises roles runs with private and enterprise. Two tests run
without them: every roles route answers the Enterprise refusal, and a data
table saved under roles, or a named role, is refused a connection while one
not under roles resolves as before.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BjfMkJyKzodxkobqGZ6Lqb
* feat(datatables): gate the roles UI mount sites on an enterprise license
Both mount sites are still commented out; the gate travels with them.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BjfMkJyKzodxkobqGZ6Lqb
* test(datatables): run the tenant matcher test on the enterprise edition
The matcher it covers is enterprise code now, so without the enterprise
edition the test hit the stub and failed the default windmill-common run. It
runs with private and enterprise, and a counterpart without them asserts that
no tenant list covers anyone, the wildcard and a workspace admin included.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BjfMkJyKzodxkobqGZ6Lqb
* chore: update ee-repo-ref to a1873dbb67f2302b85ff5362f8387b48eccdb607
This commit updates the EE repository reference after PR #783 was merged in windmill-ee-private.
Previous ee-repo-ref: 5c853e2c20eca6b748415fc0d6862a6ebfb5fec4
New ee-repo-ref: a1873dbb67f2302b85ff5362f8387b48eccdb607
Automated by sync-ee-ref workflow.
* fix(datatables): refuse roles while a same-workspace alias reaches the database
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* feat(datatables): add an ACL editor for data table roles
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* feat(datatables): data table roles in the DB manager and raw apps
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* fix: never add a role to the reference of a data table whose name contains '?'
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* fix: read the roles of a data table whose name contains '?'
The generated client leaves a '?' in a path param unencoded, so the lookup
404'd and the raw-app picker blocked Start on such a data table.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* fix: take every pooled connection before the ACL apply locks
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BRoYE5ZeAVvrDYdfhDAYXb
* fix: refresh grant options only after the ACL apply validates its plan
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BRoYE5ZeAVvrDYdfhDAYXb
* fix(datatables): refuse a reference naming both a legacy data table and a role
When a workspace stores both `sales` and a legacy `sales?role=analytics`, the
reference resolved to the legacy entry without a role, so browsing `sales` as
`analytics` reached another data table.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* fix: declare the default role in migrations written for a data table whose name contains '?'
Such a data table connects as its default role without naming it, so the
migrations the manager wrote for it declared no role and ran as admin.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* fix(datatables): let CE migrations connect as an explicitly named admin
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* fix: add only missing grant options before an ACL apply, never default privileges
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BRoYE5ZeAVvrDYdfhDAYXb
* fix(datatables): serialize roles going on with aliases saved from other workspaces
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* docs(datatables): note that legacy names with ? cannot be migrated
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* fix: run one data table ACL apply at a time per server before it connects
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BRoYE5ZeAVvrDYdfhDAYXb
* feat(datatables): set up an external instance cluster for data tables
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* fix(datatables): send external cluster passwords as SCRAM verifiers
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* fix(datatables): scope external cluster credential readers to the crate
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* [ee] feat(datatables): external_instance data tables on the external cluster
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* fix: hold the ACL connection to the database that was authorized
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BRoYE5ZeAVvrDYdfhDAYXb
* fix(datatables): compare the external cluster settings under a row lock before storing setup
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* fix(datatables): only drop external databases Windmill marked, and check use under the lock
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* fix: build the ACL connection from the authorized data table entry
Resolving the settings again could land on a resource with the same
database name on another server, which the later entry checks never see.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BRoYE5ZeAVvrDYdfhDAYXb
* fix: check ACL read reach against the entry it connects from
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BRoYE5ZeAVvrDYdfhDAYXb
* feat(datatables): put a data table's connection under Postgres roles
A data table backed by the instance database resolved to exactly one Postgres connection,
`custom_instance_user`, for everyone who could reach it at all. There was no way to say
this job reads, that one writes, this one never sees the salaries table.
A data table role is now a real Postgres login on the cluster, defined once for the
instance by a superadmin and named exactly as they named it. A script that declares
`-- role analytics` connects as `analytics`, and Postgres decides what it may touch —
grants are ordinary SQL. Windmill answers only "may this caller ask for this role", from
the tenant lists on the data table entry: `u/alice`, `g/analysts`, `f/finance` or `*`.
A data table with no `permissions` block behaves exactly as before.
Everything that opens a connection on someone's behalf goes through one chokepoint,
`get_datatable_resource_from_db`, which takes the identity explicitly and fails closed when
there is none. The role logs in as itself — never `SET ROLE`, which a script could
`RESET ROLE` its way out of.
A fork's data table entry becomes a pointer at the workspace that governs it rather than a
copy of it. The settings clone used to hand a fork a byte-identical entry naming the
parent's database, which a fork admin could edit to grant themselves `admin` there; a
pointer has nothing local to edit, and its tenants are evaluated as a member of the
governing workspace, by email. `permissions` is stripped from the workspace export and
ignored on import: tenants name principals of one workspace, and a settings push is not
where an access decision should be made.
Operations that see the whole database whatever the roles grant stay with the governing
workspace's admins: editing the roles, a migration that declares none, and opening a
replication stream for a Postgres trigger or capture.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012ti5HyeTikPMYyW8YSdiHR
* fix(datatables): gate the paths that reach a whole database as admin
Auditing what still resolved through the unchecked resolver turned up three that act for a
caller and hand back the admin connection: `resolve_pg_source_checked` (behind schema
export, the full-schema read, database creation, import and the forked-database drop), the
connection test, and the schema snapshot a fork clone takes of its parent. On a data table
under roles each let any workspace member — or a fork admin who is nobody in the governing
workspace — read or copy the whole database whatever its roles grant.
All three now require admin reach on the governing workspace. A dump taken under a
restricted role would be a silently truncated copy rather than an error, so refusing is the
only right answer for the copy paths.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012ti5HyeTikPMYyW8YSdiHR
* fix(datatables): confine roles to the instance database, and stop a fork reaching the parent's bookkeeping
A data table role is a login on Windmill's own Postgres. Nothing stopped a workspace admin
putting a *resource-backed* data table under roles, at which point the executor dialled the
host that resource names — one the admin chose — with the role's real cluster password, and
`CONNECT` is granted to every registered instance database. Both ends now refuse: the
permissions endpoint rejects the save, and the chokepoint refuses to substitute credentials
on a non-instance entry rather than trusting the record it read.
Two more places reached the governing database without answering to it. The initial-migration
generator returned a `pg_dump` of the whole schema to any member. And the migration
rename/delete cascade followed a fork's pointer into the parent, so a fork admin renaming or
removing their own local entry relabelled or wiped the parent's `_wm_migrations` — after
which the parent re-runs every migration from zero. The remote half is now skipped when the
entry resolves into another workspace, which is also just correct: a fork renaming what it
calls a data table changes nothing about the data table.
Also: revoking a tenant now bounces the replication streams of every workspace holding an
entry that resolves here, not only the governing one, so a fork's trigger stops rather than
living on inside its open connection; the instance role catalog and the governing workspace's
tenant lists are no longer returned to someone who cannot edit them; and the tenant rename
dedup collapses non-adjacent duplicates, per role rather than once any role changed.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012ti5HyeTikPMYyW8YSdiHR
* fix(datatables): fail loudly where a role or a pointer can be left half-recorded
Three ways the feature could end up in a state nobody could see or undo.
Creating a role writes the cluster first and the catalog second, but the catalog write was an
`UPDATE` that matched nothing when the instance Postgres settings row was absent — leaving a
live login with a password nobody recorded: invisible to the catalog, un-recreatable because
the name is taken, and un-deletable because there is no entry to delete. It now errors, so
the operation is retryable once the row is restored.
Deleting a workspace only nulls the fork lineage; the data table entries pointing at it are
left resolving to nothing. Sweeping them is not an option — turning a pointer back into a copy
would hand each fork the database outright — so the delete now names the data tables it
stranded, and resolving one says which workspace is missing rather than reporting a data table
this workspace never had.
`InstanceDatatableRole` derived `Debug` while holding a Postgres password; it is now
hand-written so `{:?}` on the catalog cannot put a live credential in a log line.
Adds the two branches the reviews found unpinned: a caller who is not a member of the
governing workspace at all, and `NoIdentity` — the compatibility path for an agent worker that
predates this and sends no job id.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012ti5HyeTikPMYyW8YSdiHR
* fix(datatables): unbreak two operator messages and two comments that described other code
The two strings this branch added for states an operator hits once — the catalog write that
matched nothing, and the delete that stranded a pointer — were collapsed from their multi-line
form with the indentation left in, so both rendered with a fourteen-space gap mid-sentence.
`list_datatables` claimed to report a chain it cannot follow and then dropped it; it does drop
it, and the comment now says why that is the right place to stay quiet. The non-superadmin
check in `edit_datatable_config` was introduced as also covering references, which it does not
and need not: `reference` is overwritten from the stored entry for every caller before the
check runs.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012ti5HyeTikPMYyW8YSdiHR
* fix(datatables): serialize role catalog mutations, and state each helper's authorization contract
The catalog is one JSON document, so create, rename, enable and delete are all
read-modify-write. Two concurrent creates read the same snapshot, both succeed in the
cluster, and the second write drops the first — leaving a live Postgres login with a password
nobody recorded, which is the exact state the delete path exists to prevent. Every mutation
now runs in one transaction holding an advisory lock across the read, the cluster DDL and the
write, so a lost update cannot happen and a failure rolls the whole thing back. The DDL
helpers take that transaction rather than the pool, which is what makes the lock cover them.
Their statements moved off `sqlx::raw_sql`: the simple protocol is only needed for genuinely
multi-statement SQL, and its future is not `Send`, which an axum handler holding the
transaction requires. Each of these is one statement anyway.
The new cross-crate surface now says what callers must do. `read_role_catalog` returns
plaintext credentials; `create`/`rename`/`set_login`/`drop_instance_role` and
`converge_connect_grants` mutate cluster-wide state; `read_datatable_entry` reads a workspace's
raw config. All of them are superadmin-gated by their current handlers, but nothing said so at
the definition, which is where the next caller looks.
Also: the roles table reloads after a failed login toggle instead of leaving it claiming a flip
that did not land; the rename affordance is the design-system `Button`, not a raw one; and
`resolve_datatable_pg_as_caller` drops a `role` parameter no caller ever filled — browsing
resolves as the data table's default until the database manager grows a picker.
Why role passwords stay a plain `String` while the instance user's password beside them is a
`StringOrSecretRef`, asked three times across reviews: that one is a secret ref because an
operator supplies it and may want it from their own backend, while these are minted here and
never entered by anyone, so there is nothing for a ref to point at. Encrypting generated
secrets at rest is a separate change that would take the replication password with it. Now
said at the field.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012ti5HyeTikPMYyW8YSdiHR
* fix(datatables): give the role catalog its own row, out of reach of the config machinery
Putting it inside `custom_instance_pg_databases` was the wrong call, and it cost two ways.
The catalog serializes a generated Postgres password per role, and that row is the
operator-facing instance config, so the passwords reached `get_instance_config` and its YAML
editor — a live cluster credential in a response body, a UI field and any log of either.
Worse in the other direction: `to_settings_map` strips the catalog, so a full-row upsert of
that key writes the row back without it and the catalog is gone, while the cluster keeps every
login it described.
`custom_instance_replication_pwd` is the precedent and says exactly why — a generated secret,
written only by the server, never operator-authored, hidden so the config machinery cannot
read, rewrite or drop it. The catalog is the same thing, so it now has the same shape:
`datatable_roles`, in `HIDDEN_SETTINGS`, `PROTECTED_SETTINGS` and the agent-worker denylist.
No redaction to keep in step with three code paths, and no way for a neighbouring write to
take it out.
Two races on the same shared documents. `edit_datatable_config` read the stored data tables
outside its transaction and then wrote the whole `datatable` document, so a permissions save
committing in between was silently rolled back; it now reads under `FOR UPDATE`. And
`set_datatable_permissions` validated role ids against the catalog before opening its
transaction, so a deletion in between let it write a deleted role back — including as the
default, which every later job then fails on; it now holds the catalog lock and the settings
row across validation and write.
Completes the authorization contracts the previous commit claimed but did not finish:
`read_datatable_entry` (which it named and missed), `resolve_governing_datatable`, whose whole
job is to answer for a workspace the caller may not belong to, and
`converge_connect_grants_with`, which had not inherited its wrapper's.
Also the generic Python SDK reference: `_format_py_params` learned the bare `*` last time, but
`extract_py_functions` is a second formatter and still rendered `datatable(name, role)`, so
code written from that page passed a keyword-only argument positionally.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012ti5HyeTikPMYyW8YSdiHR
* fix(datatables): make the concurrency test pin the handlers, and the contracts describe what is enforced
The concurrency test reimplemented the read-modify-write inline, so deleting the lock from all
three handlers left it green — it pinned Postgres, not the code it was written for. It now
drives `create_datatable_role` twice concurrently and asserts the catalog kept both names.
Checked the way the last one should have been: removing the lock from the handler makes it
fail with "wmtest_a_… is a live cluster login the catalog forgot".
The contracts added last commit were stricter than this PR's own callers, which is worse than
none — the next reader sees a rule already broken and learns to ignore it.
`read_role_catalog` said superadmin-only while two of its four callers are open to any
workspace member, and `converge_connect_grants` said superadmin while
`set_datatable_permissions` reaches it as a workspace admin. Both were fine on substance: the
rule that actually holds is about the credential never reaching a response, log, audit record
or export, not about who may call. They now say that. `read_datatable_entry` gets the same
treatment rather than the one the earlier message claimed for it: it is the primitive every
resolution goes through, so it is deliberately open, and what must not escape is `permissions`
— it names the governing workspace's users, groups and folders.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012ti5HyeTikPMYyW8YSdiHR
* fix(datatables): close the last ways a role or a pointer can be left pointing at nothing
The raw settings readers hand back whatever is in the row, so moving the catalog into its own
`global_settings` key protected the config machinery and left `GET /settings/global/datatable_roles`
and the settings listing returning every live password. Both now filter that one key. The
neighbouring `custom_instance_replication_pwd` has the same shape and is not touched here: it
predates this and widening the fix to it is a decision about an operator workflow, not a
consequence of this change.
Three ways a save could leave something resolving to nothing:
A permissioned data table could be moved to a PostgreSQL resource. The block was carried across
as a server-owned field, the runtime refuses roles on a resource-backed table, so the save
succeeded and every job afterwards failed. Refused instead — turning roles off first is one step,
and it keeps discarding an access decision something somebody chose.
Renaming a governing data table left every fork pointing at the old name: the data table
disappears from their pickers and their jobs stop, with nothing in the renaming workspace to
suggest why. The rename now follows into the pointers in the same transaction.
Deleting one cannot be followed the same way, so it is reported instead — the response names what
it stranded, the way deleting a workspace does, and the fork's own error already says which
workspace is gone.
Also: `ensure_instance_db_grant_options_unchecked` claimed superadmin while the permissions
handler reaches it as a workspace admin (the same class fixed last commit, one instance missed);
the role entry kept an `instance_config_schema` derive it no longer needs; `write_role_catalog`
was the one writer of that table not stamping `updated_at`; and the concurrency test dropped its
roles only on success — a failing run is exactly the one that creates them without recording them.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012ti5HyeTikPMYyW8YSdiHR
* refactor(datatables): put the role catalog in its own table, not in global_settings
Five findings across three rounds were all the same choice. A set of live Postgres credentials
was living in `global_settings`, which has generic read, list, write, config-export and CLI
round-trip paths that know nothing about what they carry: the passwords reached the instance
config and its YAML editor, a full-row upsert of a neighbouring key erased the catalog,
`GET /settings/global/{key}` and the settings listing returned them raw, and this round the
redaction that fixed the last two turned `wmill instance push` into something that wipes every
password — a fix breaking the assumption the previous fix made. `POST /settings/global/datatable_roles`
could also empty it outside the lock.
The approved plan offered a table or `global_settings`, so this is the other option it already
allowed rather than a new design. `datatable_role` is a table: no generic settings path can read
it, list it, export it, write it or round-trip it, so none of the five needs a guard. The
redaction, the hidden/protected/agent-denylist entries and the JSON document all go with it.
One row per role also removes the read-modify-write the concurrency work was about: two
concurrent creates are two inserts, and the unique index on `name` is what settles a collision.
The advisory lock stays for the one window rows do not cover — `CREATE ROLE` is invisible to
another transaction until commit, so without it both creates pass their `pg_roles` check.
Also from this round: rename mappings are checked against the configuration they claim to
describe, since fork pointers are rewritten from them — a caller could otherwise submit
`main -> missing` against an unchanged config and repoint every fork of `main` at a name nothing
has, and `A -> B` plus `B -> C` moved what pointed at `A` all the way to `C`. And the warning
naming forks a delete stranded reached the response but not the screen: both the data table
settings save and the workspace delete now show it.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012ti5HyeTikPMYyW8YSdiHR
* fix(datatables): validate a rename against the save it describes, and re-check under the locks
Three from the round, all about deciding on state that could already have moved.
A permission save resolved the data table and checked it was instance-backed before taking any
lock, then wrote under one. A config save committing in between could move the table onto a
PostgreSQL resource — recreating exactly what the transition guard refuses — or rename it, in
which case the write targeted a key that no longer existed and reported success having changed
nothing. It now re-resolves and re-checks on the locked state.
Rename validation checked that the source existed before and the target existed after, which
still accepts `main -> decoy` against a save that keeps both: every fork of `main` then follows
onto a different data table, silently, because it keeps resolving. The rule is now the actual
old-to-new key transition — a source may only survive if another rename took its name, and a
target may only pre-exist if another rename freed it. That also stops two sources sharing one
target, and it admits a swap, which the previous guard refused: `datatables` is keyed by name, so
a swap cannot be done one save at a time, and refusing it was a regression against main. The
pointer cascade now runs in two passes through a temporary name, the way the migration cascade
one layer down already handles the same shape, so `A -> B` with `B -> C` moves each pointer once
from what it named before the save.
The tenant mutators say what they are for: they write an access decision for any workspace named,
with an arbitrary mutation, and exist for the transaction that frees or renames a principal.
Editing a decision on purpose belongs in the permissions endpoint.
Carried in the same change: the stranded-fork list is a field rather than a phrase to grep out of
a success string; the pointer cascade matches with `EXISTS` instead of a `LIKE` over the whole
document, so a workspace whose pointers name something else is not rewritten to a byte-identical
value under an exclusive lock; and `InstanceDatatableRole` drops the serde derives left over from
the JSON document, one of which would emit `pwd`.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012ti5HyeTikPMYyW8YSdiHR
* fix(datatables): cascade on the leave route that is used, gate migrations before the admin connection, and drop a role atomically
The tenant cascade on leaving went onto `/users/leave`. The UI and the generated client call
`/workspaces/leave` — a different handler in a different crate with the same name — which
deleted the membership and left `u/<username>` in the tenant lists. Leaving and rejoining
therefore restored the access the leave was supposed to end, and a later account taking the
username would have inherited it. The regression test drives the route the client actually
calls; without the fix it fails with "leaving kept the tenant".
The migration endpoints authorized too late. `run_datatable_migrations` opened the data table's
admin connection, created `_wm_migrations` and read it before reaching the per-migration role
check — so with nothing pending, nothing was checked at all. Rollback returned before its check
when nothing was applied, and the status endpoint had none. All three now ask, before any
connection is opened, whether the caller can reach the data table as any role at all; which role
a given migration runs as is still decided per migration, and by the executor after that.
Deleting a role committed the cluster drop and the catalog row, then swept the tenant lists in
separate transactions. A sweep failing part-way left workspaces naming a role nothing can connect
as, while the retry answered `NotFound` because the catalog entry was already gone. The sweep now
runs in the same transaction, so the drop, the row and every tenant list commit together.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012ti5HyeTikPMYyW8YSdiHR
* fix(datatables): refuse to copy a data table that is under roles
pg_dump carries no roles and the import runs with --no-privileges, so a copied
data table arrives owned by the admin connection with no GRANT for any role.
The settings clone brings `permissions` across, so the fork's tenants pass
Windmill's check, connect as the role they were given, and are denied by
Postgres on everything: an entry that reads as configured and answers nothing.
Refuse the copy — in the import endpoint before any data moves, and in the fork
path the CLI takes. Replaying the source's owners and ACLs into the clone is
what lifts this, and is a change of its own. Dropping `permissions` from the
copy instead would be the unsafe half, since the copy holds the parent's rows.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012ti5HyeTikPMYyW8YSdiHR
* fix(datatables): refuse the clone's database too, not only its data
A clone is two endpoints: `create_pg_database` then `import_pg_database`. Only
the second refused a data table under roles, so a fork asking to clone one
created and registered an empty `wm_fork_…` instance database and then failed —
and nothing collects it, since `drop_forked_datatable_databases` only drops
entries carrying `forked_from` and no entry names this one.
Refuse in both, so the clone stops before a database exists.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012ti5HyeTikPMYyW8YSdiHR
* nit worker error msg
* fix pg_dump stuck on version 17 on nix
* fix(datatables): refuse a malformed role annotation instead of ignoring it
`-- Role operator`, `-- role operator;` and `-- role operator -- why` all failed
the annotation parser's exact-match rule, so the query fell through to the data
table's default role and ran, silently, under a login the author did not choose.
Naming a role exists precisely to not do that.
A leading comment whose first word is `role` is now an annotation attempt: the
keyword matches case-insensitively, one trailing `;` is tolerated, and anything
else is an error naming the line. Only callers that already know the target is a
`datatable://` reference ever run this, so ordinary SQL keeps its comments.
Also bumps the dev shell's postgres client to 18 — it trailed the server the dev
database runs, which takes out every data table export, clone and fork-with-data.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012ti5HyeTikPMYyW8YSdiHR
* fix(datatables): refuse a malformed role query string instead of ignoring it
`?Role=analytics`, `?role=` and `?x=1&role=…` all fell through the reference
parser's exact-match rule, so the connection resolved to the data table's default
role and ran under a login the caller never asked for — the URI half of the same
trap as a malformed `-- role` annotation.
The key now matches case-insensitively, and anything else in the query string is
an error naming it; `role` is the only parameter a reference takes. Callers that
only need the entry keep a lenient `datatable_ref_name`, since they never act on
the role. The DuckDB `ATTACH` parser propagates it rather than attaching under
the default.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012ti5HyeTikPMYyW8YSdiHR
* fix(datatables): carry the role annotation into the row_to_json retry
The retry rebuilds its SQL from `pruneComments(code)`, so the leading comment
block never reached the second attempt — and with it the `-- role <name>` line
that decides which login the query runs as. The retry connected as the data
table's default role instead, so a query the first attempt was denied could
succeed on the second, reported as "recovered with the row_to_json fix".
Carry the leading comment block over. The retry itself is unchanged.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BjfMkJyKzodxkobqGZ6Lqb
* chore(datatables): don't mount the roles UI until the ACL editor lands
Enforcement ships first. The permissions drawer is what turns roles on, and the
catalog section is what creates them — both are only useful once there is a way
to grant a role the privileges it needs, which arrives with the ACL editor. Left
mounted they would offer a feature whose other half does not exist.
The two components are complete and reviewed; only their call sites here are
commented out, with a note pointing the follow-up PRs at them.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BjfMkJyKzodxkobqGZ6Lqb
* fix(datatables): honour `-- role: x`, and fix the DuckDB attach test
Two review findings, both real.
`attach_datatable_parses_name_and_role` never compiled: `parse_attach_datatable`
returns `Result<Option<_>>` now and one call site kept a single `unwrap`. Its
`?Role=analytics` case also asserted a refusal, contradicting the parser in the
same commit, which matches the key case-insensitively. Replaced with the cases
that are genuinely malformed, and a positive one for the cased key.
`-- role: analytics` fell through to the default role — the silent fallback the
strict parser exists to remove, for the spelling most likely to be typed. The
keyword now accepts an optional colon, attached or spaced, while a word that
merely starts with it (`rolebased`) is still not an attempt.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BjfMkJyKzodxkobqGZ6Lqb
* fix(datatables): clone a fork's pointer instead of failing after the copy
Forking a fork with cloning left an orphan database. The preflight resolves the
pointer and sees the governing entry, so both endpoints ran and filled the new
database; `apply_forked_datatable` then refused the inherited pointer and rolled
the fork back, stranding a registered `wm_fork_*` that no entry names and whose
name blocks the retry.
Refusing earlier would have been the smaller change, but forking a fork and
cloning worked before pointers existed, so it would trade an orphan for a
regression. Resolve what the pointer names and write the terminal entry the
clone needs: the whole `database` object rather than a patch of its
`resource_path`, since a pointer has none, and `reference` removed with it.
Also accepts `-- role=x` and `-- Role = x`, two more spellings that fell through
to the default role.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BjfMkJyKzodxkobqGZ6Lqb
* fix(datatables): refuse to roll back the catalog while roles exist
The down migration dropped the table and left every role behind: live Postgres
logins whose passwords only that table carried, so after a revert Windmill could
neither use, disable nor delete them, and re-applying could not recreate them
because the names were taken. Cleaning up here is not possible either — dropping
a role means reassigning what it owns in every instance database, and a
migration runs in one — so it now refuses while the catalog is non-empty and
says to delete the roles through instance settings, which does the cluster work.
Also enforces the instance-only invariant the resolved-pointer clone relies on
rather than only asserting it in a comment.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BjfMkJyKzodxkobqGZ6Lqb
* refactor(datatables): settle clonability in one place, before anything is created
A clone is three stages a workspace apart — `create_pg_database`, then
`import_pg_database`, then `apply_forked_datatable` inside the fork transaction.
Only the third can roll back, and `CREATE DATABASE` is not transactional, so any
refusal that lives there strands a registered `wm_fork_*` that no entry names
and whose name blocks the retry.
That orphan has now been fixed three times, most recently reintroduced by a
guard added one commit ago. Patching each new refusal into the first endpoint is
not the fix; having two places that can refuse is. `ensure_datatable_is_clonable`
now answers every reason a copy can be refused and returns what it resolved, and
the stage that writes the entry only does the work.
Also takes an ACCESS EXCLUSIVE lock before the rollback guard counts, so a role
created concurrently cannot slip between the check and the drop.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BjfMkJyKzodxkobqGZ6Lqb
* fix(datatables): let a retried clone reclaim its own leftover database
A clone creates its target database one request before it copies into it, and
the fork that would name it is written a request after that. Any failure in
between — a pg_dump error, a bad restore, a dropped connection, the source's
roles changing mid-flow — left a registered `wm_fork_*` that no entry names,
and every retry then failed on its name. This predates data table roles.
`create_pg_database` now reclaims such a leftover before creating: only a
`wm_fork_*` database Windmill registered as a data table database and that no
data table or ducklake entry names, in any workspace, archived ones included.
The drop never terminates connections, so a clone still copying into it makes
the reclaim fail instead of being cut off. It is limited to callers who
administer the source — reaching it is not enough, since on a data table
without roles every member reaches it — and anyone else gets the refusal an
existing database always got.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* Revert "fix(datatables): let a retried clone reclaim its own leftover database"
This reverts commit 7dd3275a10.
The reclaim tied the caller to the source they administer, but not to the
database it dropped. Between another workspace's import and its final fork
request, that workspace's target is full, registered, unnamed and has no open
connection, so an admin of any instance data table could name it and have it
dropped and recreated empty. The victim's fork would then commit pointing at
the empty copy. Safe reclaim needs durable clone ownership and serialization
with the request that names the database; until then the leftover stays, as it
did before this PR.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* docs(datatables): record the stale clone database as a known limitation
A clone is three requests and `CREATE DATABASE` is not transactional, so a
failure after the first leaves a registered `wm_fork_*` behind, as it did
before data table roles. Accepted for this PR: it is harmless to data and goes
away once the clone is a single server-side operation.
The comment also records why the obvious fix is wrong: reclaiming the leftover
on retry, without durable clone ownership, can drop another workspace's fully
copied database between its import and its final fork request.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* fix(datatables): bounce the streams reading a data table when it is deleted
Deleting a governing data table, or the workspace that holds it, only collected
the fork pointers it stranded, for the warning. A Postgres trigger or capture
already streaming through one of those pointers kept the replication connection
it opened while the pointer still resolved, so it went on dispatching the
governing database's rows after the fork lost access — until its connection
happened to restart. The governing workspace's own streams on a deleted entry
did the same.
Both deletion paths now bounce the affected listeners inside their own
transaction, through the helper a permission change already uses, so a
listener that reconnects re-resolves the entry and finds it gone. The helper is
split so a caller can pass the (workspace, local name) pairs it already holds.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* fix(datatables): keep the fork schema baseline, and bounce streams on every removal
Three fixes from review.
`edit_datatable_config` took `forked_from` wholesale from the stored entry, so
the fork schema diff's save of an advanced baseline was silently discarded and
an applied change was offered again. Whether an entry carries a clone stamp is
still carried from the store, since that is what marks its database droppable,
but the baseline inside it is now taken from the request.
The stranded-pointer warning and the stream bounce ran over the optional
`deleted_datatables` hint, which the settings-sync CLI never sends, so removing
a governing data table through `wmill` bounced nothing. Removals are now derived
from the stored configuration against the saved one.
`delete_workspace` read the pointers to bounce before its transaction, so a fork
committing a pointer during the deletion was missed. The read now happens inside
the transaction, after the workspace row is deleted: a fork's insert key-share
locks that row through its parent foreign key, so it is either seen or fails on
the missing parent.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* refactor(datatables): keep Postgres triggers and data table roles apart
A replication stream reads every row of every table whatever the data table's
roles grant, and its listener checks access only when it connects. Rather than
chase every way access can change and bounce the streams each one affects, a
data table now carries one or the other:
- a Postgres trigger or capture cannot be created on, or connect to, a data
table under roles;
- roles cannot be turned on while an enabled trigger or a live capture reads
the data table, its own or a fork's through its pointer. The refusal names
each one to disable.
This removes the stream bounces on roles edits and on data table and workspace
deletion, and the trigger gate that admitted admins. The fork schema baseline
fix from the same review round is kept.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* fix(datatables): refuse a Postgres trigger on a data table under roles when it is saved
Creating or editing a trigger that points at a data table under roles was
accepted, and its listener then retried the refused connection every 30
seconds forever. The save is now refused, and a trigger that reaches such a
data table anyway (re-enabled, or cloned into a fork) is disabled by its
listener with the reason, as a missing replication slot is.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* fix(datatables): disable a data table role before deleting it
Deleting a role reassigns and drops what it owns in each registered
database on its own connection, and each of those passes commits as it
goes. A database failing part-way left the role enabled in the catalog and
able to log in, but already stripped in the databases reached before it.
The role is now disabled in its own commit first, so a failed delete
leaves a disabled role to retry.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* fix(datatables): serialize roles going on with a stream starting
Turning roles on looked for enabled triggers and live captures once,
without a lock anything starting a stream also took. A trigger enabled in
that window could have its listener connect before roles committed, and a
healthy listener never checks again. Both transitions now serialize on one
advisory lock: roles going on hold it exclusive while they look, and
trigger create, edit and enable, and capture setup and ping hold it shared
while they commit. Either the look sees the stream, or the listener
connects after roles are committed and refuses.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* fix(datatables): wait out live listeners, and resolve stored names containing `?`
Turning roles on counted a trigger as gone once disabled, and a capture
once its client stopped pinging, but the listener keeps its replication
connection until its next heartbeat notices. A trigger or capture whose
listener pinged in the last 15 seconds, the window a server holds a
listener for, now still counts as streaming.
Data table names could contain `?` before they were restricted, and such
entries are still stored. Splitting `?role=` off a reference misread them:
`a?b` became `a` with an unknown parameter, and the clone checks looked at
a different entry than the one copied. An entry stored under the whole
reference is now looked up first, in the Postgres executor, DuckDB ATTACH
and the clone checks. Agent workers cannot read the workspace and keep
the strict parse, which refuses such a name rather than misreading it.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* fix(datatables): warn when a settings sync strands fork pointers
A settings save reported the fork pointers left resolving to nothing only
for the names in `deleted_datatables`, which `wmill sync push` never sends.
The save now works out what it removed from the locked entries, and the
CLI prints the stranded pointers it returns.
Also correct the replication helper's contract: no role or admin check
makes a replication connection safe, so a data table under roles is
refused outright rather than gated as an admin operation.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BjfMkJyKzodxkobqGZ6Lqb
* fix(datatables): refuse a save that drops a data table's roles through an undeclared rename
A data table's roles follow its entry only through a declared rename. A
settings sync sends the whole map and never declares one, so renaming a
data table under roles there read as a delete and a new entry on the same
database: the new entry carried no roles, and every caller connected as
admin. Such a save is now refused, naming both entries.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BjfMkJyKzodxkobqGZ6Lqb
* fix(datatables): no entry without roles may newly reach a database under roles
The previous guard only caught a new name replacing an entry under roles.
A whole-map save could also repoint an existing entry without roles at
that database, or another workspace could point one there, and every
caller of that entry would connect as admin. The rule is now stated on
the saved entries: one that carries no roles and newly points at an
instance database any entry under roles uses, in this workspace or
another, is refused. A declared rename carries its roles and passes.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BjfMkJyKzodxkobqGZ6Lqb
* feat(datatables): move data table role catalog and resolution to the enterprise edition
Roles are an Enterprise Edition feature. The catalog, the Postgres logins,
CONNECT convergence, tenant evaluation and the role half of connection
resolution move to windmill-ee-private. Every public function keeps its path
and signature and forwards through datatable_roles_oss, which re-exports the
enterprise implementation or, without it, refuses.
Without the enterprise edition a data table under roles, or a caller naming a
role, is refused a connection rather than resolved as admin, and the reach and
admin-access checks refuse one under roles. A data table not under roles
resolves as before in every edition, and an instance database keeps the
CONNECT grants it was created with. The catalog lock, the stream lock, the
tenant cascades and the permissions stripping stay in OSS: they only restrict.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BjfMkJyKzodxkobqGZ6Lqb
* feat(datatables): move the data table permissions endpoints to the enterprise edition
The permissions read, save and usable-roles handlers move to
windmill-ee-private; the routes stay registered and, without the enterprise
edition, answer that data table roles are an Enterprise Edition feature.
ensure_governs_datatable and ensure_reaches_datatable keep their paths: the
first refuses, the second passes a data table not under roles.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BjfMkJyKzodxkobqGZ6Lqb
* feat(datatables): move the data table role catalog endpoints to the enterprise edition
The superadmin list, create, update and delete handlers move to
windmill-ee-private. The routes stay registered and, without the enterprise
edition, refuse after authentication.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BjfMkJyKzodxkobqGZ6Lqb
* test(datatables): run the roles tests on the enterprise edition, refusals without it
Each test that exercises roles runs with private and enterprise. Two tests run
without them: every roles route answers the Enterprise refusal, and a data
table saved under roles, or a named role, is refused a connection while one
not under roles resolves as before.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BjfMkJyKzodxkobqGZ6Lqb
* feat(datatables): gate the roles UI mount sites on an enterprise license
Both mount sites are still commented out; the gate travels with them.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BjfMkJyKzodxkobqGZ6Lqb
* test(datatables): run the tenant matcher test on the enterprise edition
The matcher it covers is enterprise code now, so without the enterprise
edition the test hit the stub and failed the default windmill-common run. It
runs with private and enterprise, and a counterpart without them asserts that
no tenant list covers anyone, the wildcard and a workspace admin included.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BjfMkJyKzodxkobqGZ6Lqb
* chore: update ee-repo-ref to a1873dbb67f2302b85ff5362f8387b48eccdb607
This commit updates the EE repository reference after PR #783 was merged in windmill-ee-private.
Previous ee-repo-ref: 5c853e2c20eca6b748415fc0d6862a6ebfb5fec4
New ee-repo-ref: a1873dbb67f2302b85ff5362f8387b48eccdb607
Automated by sync-ee-ref workflow.
* fix(datatables): refuse roles while a same-workspace alias reaches the database
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* fix(datatables): let CE migrations connect as an explicitly named admin
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* fix(datatables): serialize roles going on with aliases saved from other workspaces
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* docs(datatables): note that legacy names with ? cannot be migrated
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* feat(datatables): add an ACL editor for data table roles
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* fix: take every pooled connection before the ACL apply locks
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BRoYE5ZeAVvrDYdfhDAYXb
* fix: refresh grant options only after the ACL apply validates its plan
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BRoYE5ZeAVvrDYdfhDAYXb
* fix: add only missing grant options before an ACL apply, never default privileges
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BRoYE5ZeAVvrDYdfhDAYXb
* fix: run one data table ACL apply at a time per server before it connects
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BRoYE5ZeAVvrDYdfhDAYXb
* fix: hold the ACL connection to the database that was authorized
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BRoYE5ZeAVvrDYdfhDAYXb
* fix: build the ACL connection from the authorized data table entry
Resolving the settings again could land on a resource with the same
database name on another server, which the later entry checks never see.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BRoYE5ZeAVvrDYdfhDAYXb
* fix: check ACL read reach against the entry it connects from
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BRoYE5ZeAVvrDYdfhDAYXb
* feat(datatables): Ducklake catalogs on the external instance cluster
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* fix(datatables): never grant CREATEROLE to custom_instance_user on the external cluster
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* docs(datatables): state the authorization contract of external database usage lookups
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* fix(datatables): drop a DuckDB data table secret once its ATTACH has used it
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* perf(datatables): resolve a workspace's data tables per pointer hop, not per entry
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* fix(datatables): hold the parent's settings while a fork points at its data tables
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* fix(datatables): refuse repointing the external cluster while it is in use, and keep verify-ca working for pg_dump
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* fix(datatables): protect external databases pending fork cleanup, and describe Ducklake usage in the API
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* feat(datatables): per-cluster data table role catalogs, with roles on the external instance cluster
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* fix(datatables): write the external cluster setting under the lifecycle lock, and check fork targets are registered
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* fix(datatables): register external fork catalogs under the lifecycle lock, and keep certificate verification in DuckDB attaches
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* fix(datatables): keep certificate verification when DuckDB attaches an external data table
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* fix(datatables): keep certificate verification when DuckDB attaches an external data table
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* fix(datatables): refuse fork cleanup of an external database another workspace uses
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* fix(datatables): refuse rolling back while external data tables are under roles, and type external_instance in the CLI
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* fix(datatables): stop counting storage-only fork cleanup rows as uses of an external database
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* fix(datatables): bind fork database copies to their workspace, and count every use before dropping one
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* fix(datatables): authenticate instance database setup before writing its status, and keep a fork reservation across it
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* fix(datatables): create external databases only on a cluster setup succeeded on, and document the registry reader
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* fix(datatables): keep only the most recently used DuckDB root certificate files
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* fix(datatables): migrate fork reservations on workspace rename, and lock the parent's data tables for the whole fork
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* fix(datatables): take the fork data table lock once, before the external cluster's
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* docs(datatables): describe the external instance cluster and how to run one locally
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* fix(datatables): keep fork reservations private, drop a cleaned-up entry with its database, and serialize cleanup with settings saves
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* fix(datatables): hold the fork lock across a fork import, and carry the reservation inside the setup write
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* fix(datatables): check the external cluster setting on its own transaction, and gate the registry probe
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* feat(datatables): add an ACL editor for data table roles
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* fix: take every pooled connection before the ACL apply locks
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BRoYE5ZeAVvrDYdfhDAYXb
* fix: refresh grant options only after the ACL apply validates its plan
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BRoYE5ZeAVvrDYdfhDAYXb
* fix: add only missing grant options before an ACL apply, never default privileges
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BRoYE5ZeAVvrDYdfhDAYXb
* fix: run one data table ACL apply at a time per server before it connects
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BRoYE5ZeAVvrDYdfhDAYXb
* fix: hold the ACL connection to the database that was authorized
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BRoYE5ZeAVvrDYdfhDAYXb
* fix: build the ACL connection from the authorized data table entry
Resolving the settings again could land on a resource with the same
database name on another server, which the later entry checks never see.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BRoYE5ZeAVvrDYdfhDAYXb
* fix: check ACL read reach against the entry it connects from
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BRoYE5ZeAVvrDYdfhDAYXb
* chore: update ee-repo-ref to 7e338e4dabf91689bfd7fb0333c6534040b17b59
This commit updates the EE repository reference after PR #787 was merged in windmill-ee-private.
Previous ee-repo-ref: 0edd40979cf36bfba59323f3f6a0811ae1369cf5
New ee-repo-ref: 7e338e4dabf91689bfd7fb0333c6534040b17b59
Automated by sync-ee-ref workflow.
* fix(datatables): keep DuckDB root certificate files in the job directory
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* chore: drop the unused json import from the settings crate
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* fix(datatables): drop the serde_json::json import left unused by the fork setup write
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* fix(datatables): accept external_instance data tables in the settings form type
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* fix(datatables): serialize fork database reservations, cleanup, setup and Ducklake saves on the database
- A fork copy is created and registered under its workspace's fork lock, and refused once the
workspace is archived; a rename re-migrates reservations under that lock after archiving.
- Ducklake saves lock every instance database they newly name, as data table saves do.
- Instance database setup holds the database's lock until its entry is written.
- Fork import also holds the database's lock across the restore.
- Cleanup re-reads the entry under its locks before dropping anything.
- Non-superadmins no longer see fork copies reserved for workspaces they are not in.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* fix(datatables): state the authorization contract of the external cluster status and write check
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* fix(datatables): lock the fork copy at finalization, and order cleanup's registry write after its settings row
- Fork finalization holds the copy's database lock from its availability check to the commit.
- Cleanup removes the registry entry in its own transaction, in a task of its own, instead of on a
second connection; a rename migrates reservations after its settings rewrites, so both take the
settings rows before the registry.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* fix(datatables): hold the fork lock across a rename's settings copy
Fork cleanup of the old id could otherwise drop a copy the renamed workspace goes on using. Also
states the authorization contract of the instance database drop helpers.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* fix(datatables): refuse moving the external cluster to another host or port while it holds databases
Every settings writer goes through the same check as removal: the login, TLS and maintenance
database may still change, the cluster may not.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* feat(datatables): clone a data table under roles with its owners and grants (#11120)
Claude-Session: https://claude.ai/code/session_01UbrtwiYNfayrmqouBJHwGV
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
* fix: open the raw app data table drawer when the workspace has none
Selecting the first data table of an empty list passed undefined to the name
check, which threw instead of opening the drawer on no data table.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* fix(datatables): offer cloning a data table under roles where its grants can be replayed
The server clones such a data table and replays the source's owners and grants,
which only the Enterprise Edition does, so the fork wizard hid both clone
options everywhere instead of on a build that cannot replay them.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* docs: name the placeholder the empty raw app data drawer renders
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* fix(datatables): replace a job directory file at the DuckDB root certificate path instead of trusting it
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* test: pin the enterprise refusal the role pickers read as 'not under roles'
The server's sentence and the frontend's copy of it were coupled by nothing,
so rewording either one turned every role picker on a community build into a
failed lookup.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* fix(datatables): type Ducklake catalogs on the external instance cluster in the settings form
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* fix(datatables): read a roles answer only for the workspace it was asked in
A fork and its parent each have their own roles on a data table of the same
name, so an answer stamped with the name alone settled the role from the
workspace the editor was acting on before.
Also derive the AI table creation flag from the data replaced into the editor:
data naming no data table left the flag on from before.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* fix(datatables): keep an instance database a settings save is waiting to name
Cleanup for a database whose setup failed took the lock first, read no user,
and dropped it while a save blocked on that same lock was about to commit a
reference to it.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* fix(datatables): check the workspace stamp in the default database selector too
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* fix(datatables): read what a save racing instance-database cleanup committed
A transaction blocked on the lock may still roll back, so keeping the database
for it stranded one whose name then blocks every retry: it is let through and
its outcome read instead. The waiter query also matches this database's locks
only, since pg_locks spans the cluster.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* fix(datatables): tell a waiting request apart from the workspaces using a database
Both callers render what cleanup returns as the workspaces that keep the
database, so a waiting request's pid read as one of them. Each now words that
case itself, and the give-up comment names where the kept name actually goes.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* fix(datatables): take the cleanup lock on a connection the pool cannot reclaim
A session lock outlives the future holding it, so a cancellation between
taking it and releasing it handed a locked session back to the pool, where
every later settings save waits on it. Detached, the connection closes when it
is dropped and the server releases the lock with the session.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* fix(datatables): close the cleanup connection on drop instead of detaching it
Detaching released the pool permit while the session stayed alive, so
concurrent cleanups waiting on their locks could open as many connections as
they liked. Closing on drop covers the same cancellation and keeps them
counted.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* chore: update ee-repo-ref to 179d2454a063ee818a9387a1eafdd354a217a16c
This commit updates the EE repository reference after PR #802 was merged in windmill-ee-private.
Previous ee-repo-ref: c2e43f5b5ff753d339b70e232fd17b6ffecb054d
New ee-repo-ref: 179d2454a063ee818a9387a1eafdd354a217a16c
Automated by sync-ee-ref workflow.
* chore: update ee-repo-ref to fd5b8af748f2c985b13e18d9ea30894f3bd7e9a3
This commit updates the EE repository reference after PR #798 was merged in windmill-ee-private.
Previous ee-repo-ref: 3145e422d61d580f0a82804f075285c112879da0
New ee-repo-ref: fd5b8af748f2c985b13e18d9ea30894f3bd7e9a3
Automated by sync-ee-ref workflow.
* fix(datatables): classify a fork import's target from the resolution that built its connection
A second read could see the entry flipped to a resource and skip the reservation check while the
connection already built still reached the instance cluster.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* fix(datatables): run external database lifecycle on the caller's transaction, one lock order for fork cleanup
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* fix(datatables): check a fork copy's reservation on the locked transaction's connection
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* fix(datatables): make and drop fork copies of external data tables on the external cluster
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* fix(datatables): refuse a fork whose external copy something already names
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* fix(datatables): check an external fork copy's uses before removing its entry, so child fork pointers still count
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* fix(datatables): migrate external cluster fork reservations on workspace rename
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* feat(datatables): configure the external instance cluster and pick its databases from the UI
Instance settings gets an External Postgres tab: the cluster's connection, the setup run
with its report, and the databases Windmill created there. Data table and Ducklake settings
offer the external_instance kind, with a picker over those databases.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* fix(datatables): keep the external cluster form out of the instance settings the page bulk-saves
The form seeded its key into the settings store after the page snapshotted them, so merely
opening the tab made the page send a setting the validator refuses, failing an admin's
unrelated save. The form is local state now, and only the setup writes the key.
The database picker also treats a superadmin-only listing as authoritative: a workspace
admin, who cannot list, no longer sees a saved database as missing.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* fix(tests): app paths are validated, so a guest app cannot be renamed to one with a space
Path validation on apps landed after this test, which still expected a space to be accepted
and created its fixture on a ':' path. The scopable-path guard it exists for is kept by
planting that path on the row, which is now the only way an app can hold one.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* feat(datatables): one Managed Postgres tab, and a switch for Windmill's own database
The instance settings tab covers both substrates Windmill administers: the external cluster,
which can now be disabled once nothing sits on it, and Windmill's own database, which an
operator can turn off so the cluster is the only one a workspace may newly name. A save that
names it then refuses, as the create endpoint does.
Workspace settings name them the way an admin meets them: Postgres Resource, and Managed
instance, qualified as Internal or External only while both are on offer.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* feat(datatables): offer the external cluster in the add-data-table wizard, and give Ducklake its room back
The wizard gets the external cluster as a fourth substrate: pick a database it already holds
or name a new one, which the run creates before writing the entry, so Try again does not trip
on a database the last attempt made.
Ducklake's maintenance column moves into the settings popover next to the extra args, which
the wider catalog select had squeezed the name box out of.
A setting a component saves itself now also moves the page's baseline, or Discard would
restore what it replaced and a later save would send that stale value back.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* fix(datatables): only a database this wizard run made makes its retry idempotent
A retry skipped the create whenever the name was registered, which let a run adopt a database
another superadmin had made and share their data without the warning the existing-database
branch shows. The run tracks what it created instead, and reports it so Discard can say the
database is still there. Creating one now refreshes the registry the next run's default name
and its validation read, and that default counts the external cluster's databases rather than
the instance's.
Row ownership carries the substrate: "instance" and "external_instance" can hold the same name,
and a row repointed between them while a run probes must not read as that run's own.
Also fixes the merge leftovers in the DuckDB executor's tests, which cargo check never compiles:
the new PgDatabase field and the attach helper's job directory.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* fix(datatables): shorten the picked managed kind, and say what an empty database list means
The closed select has room for the qualifier, not the whole name, so a picked managed kind
reads "Managed (Internal)". An external entry gets the tooltip its internal counterpart has,
and a database picker with nothing in it says to type a name rather than "No items found".
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* fix(datatables): keep the wizard's external-database bookkeeping optional and parked
The run deps gained a required field, which every existing caller of runSetup -- the sibling
test suite included -- does not pass. It defaults to empty instead, the parked payload carries
it across a Supabase redirect as it claims to, and a resumed run counts it as something left
behind. Two regressions pin the create: made on a first attempt, skipped only for a database
this run made.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* fix(datatables): review an external-cluster database as one, not as a Postgres resource
The review step fell through to the resource branch for the new provider, so it announced a
connection already in the workspace for a database that does not exist yet.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* fix(datatables): show the server's refusal instead of an object in the toast
Saving data table settings passed the whole error to the toast, which rendered the request
and response as JSON, and the instance-database wizard replaced it with "check console".
Both surface the message the API sent, through the helper that exists for it.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* chore: point at the EE commit dropping the unused import
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* fix(datatables): define a role on the cluster its data table sits on
The permissions drawer created and listed roles without naming a cluster, so a role defined
from an external data table became a login on Windmill's own, and the refresh then replaced
the correct catalog with the internal one. The drawer passes the data table's cluster to both.
The wizard also read a database it had just created as a name collision, which blocked
returning to the failed attempt and finishing under the same name.
And the merge had left the CONNECT-grant pass and its log in both the caller and the callee.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* feat(datatables): manage the external cluster's role catalog, and name the cluster in the copy
The roles drawer offered one catalog, so the external cluster's roles could only be reached
through a data table sitting on it — and became unreachable once the last one was gone, while
the cluster could not be unset until they were dropped. Opened from the page it now offers the
clusters the instance has; a data table's own drawer still pins its cluster, and its copy names
that cluster rather than "the instance".
A DuckDB attach also decides whether to verify certificates the way every other Postgres
connection does, so a resource carrying a root certificate is no longer downgraded to require
here alone.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* fix(datatables): bind the roles list to the cluster it was read for
Switching catalogs left the previous cluster's rows on screen and applied whichever response
landed last, so a slower read could seat one cluster's logins under the other's heading while
every control acted on the wrong id — invisible where a name exists on both. The switch clears
the rows, a token discards a response the selection has moved past, and the controls stay inert
while a catalog is being read.
The drop confirmation also names the cluster whose databases it reaches.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* chore: update ee-repo-ref to dfe7b8b9204d21e0264fbea1c6f6eedf9e738d56
This commit updates the EE repository reference after PR #810 was merged in windmill-ee-private.
Previous ee-repo-ref: 30db1b33bda0446f5fc5dfe353fbb226d57a26d4
New ee-repo-ref: dfe7b8b9204d21e0264fbea1c6f6eedf9e738d56
Automated by sync-ee-ref workflow.
---------
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
Co-authored-by: windmill-internal-app[bot] <windmill-internal-app[bot]@users.noreply.github.com>
Co-authored-by: Ruben Fiszel <ruben@windmill.dev>
* feat: add an options field to the postgresql resource
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
* fix: keep a literal plus in postgres connection string parameters
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
* fix: pass postgres options to trigger connections
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
* fix: read DATABASE_URL options the way sqlx does
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
* feat: include postgres options in databaseUrlFromResource
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
* feat(frontend): guided setup wizard for data tables
On Cloud a data table cannot use the Windmill instance database, so a new
workspace hit a dead end: an alert telling the user to go find a PostgreSQL
resource somewhere else. Setting one up meant three disconnected places, and the
connection could only be tested after the config had already been saved.
Adds a three-step wizard (choose a database -> set it up -> name it) reached from
the data tables settings page:
- Supabase: signs in via the existing supabase_wizard OAuth client and creates
the project from inside Windmill. Because db_pass is an input to project
creation, Windmill sets the password and the user never visits a dashboard.
- Your own database: picks an existing postgresql resource, or adds one with a
connection string through the form that already supports it.
- Windmill database: hands back to the inline row editor, since instance
databases are provisioned by a superadmin.
Verifying access is no longer a step the user takes: Continue runs the check and
passing it is what advances the wizard, so a database that cannot create tables
never reaches the workspace config.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* chore: pin ee-repo-ref to the Supabase provisioning endpoints
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* fix(frontend): do not claim the database is ready when its check failed
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* fix: address review findings on the data table wizard
- The Supabase create branch advanced on `provisioning === 4` without consulting
the check it had just run, so a role that cannot create tables could reach
Finish. It now blocks and offers Try again.
- Retrying no longer mints a fresh secret variable + resource each time: the
credentials are only re-created when the password actually changed.
- The generated password is captured before the create call rather than after,
since a throw there can still leave a project behind.
- On a failed provision the project list is refreshed, so the just-created
project can be picked up from the other tab instead of provisioning a second.
- Finish refuses a name that already belongs to another data table, which
previously repointed it at the new database.
- Secrets go to the acting user's namespace instead of a literal `u/admin/`.
- The progress list no longer ticks "Created on Supabase" before the request is
sent, and does not claim the database is ready when its check failed.
- The wizard's resume state is cleared when it closes, so reopening after an
abandoned OAuth round trip is not stuck on step 2.
- The OAuth callback shares the session-storage key rather than repeating it.
- SupabaseConnect uses the shared provisioning helpers instead of a fork.
- Restores the doc comment displaced onto TestDataTableResourceQuery.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* fix(frontend): simplify Alert layout and balance its vertical padding
The body was rendered by two near-duplicate branches, each wrapping the text in an
extra div only to hang a margin on it, and the margins disagreed: the collapsible
branch spaced above with mt-2, the static one below with mb-2. Since isCollapsed
defaults to true, every non-collapsible alert took the static branch, so titled
alerts read as 24px of space below the text against 16px above -- visibly
off-centre -- with the title and body flush against each other.
Collapse both branches into one and drop the margins; the container's own padding
now sets top and bottom equally, with a small gap under the title row.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* fix(frontend): only offer Supabase when its OAuth client is configured
The wizard offered the Supabase card unconditionally, so on an instance whose
superadmin never configured a supabase_wizard client -- or whose backend is built
without the oauth2 feature, which compiles the whole /api/oauth router out -- the
card dead-ended at a 404. Gate it on listOauthConnects, the same check
ApiConnectForm already makes, fetched on open so configuring the client mid-session
does not require a reload.
Also drop the Supabase project ref from the existing-project cards: it is an opaque
identifier that means nothing outside Supabase's own dashboard URLs. Show the region
instead, plus a status word when the project is not healthy, since a paused project
is the one case where the connection check fails for a reason unrelated to the
password.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* fix(frontend): run the Supabase OAuth leg in a popup
A full-page redirect unmounts the wizard, so anything the user does on Supabase's
side -- signing in, confirming an email, browsing their dashboard -- leaves them
with nothing pointing back at Windmill, and the wizard had to park its state in
sessionStorage to survive the trip.
Open the connect endpoint in a popup instead. The modal stays on screen throughout
and the callback hands the token back through postMessage rather than navigating.
The parked-state path stays as the fallback for browsers that block the popup.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* fix(frontend): scope the connection check to the choice that produced it
A failed check stayed on screen when the user switched Supabase mode or picked a
different provider, so a fresh tab opened showing an error about a database it had
nothing to do with. Clear the report and the error on both switches; re-clicking the
tab already selected leaves an error the user is reading in place.
Also polish the Supabase step: project cards get the provider-card treatment (icon,
p-3, flex column) instead of a hand-rolled variant whose block layout left more
padding above the name than below; form labels settle on text-emphasis; and the
signup link sits under the primary button for anyone who does not have an account
yet.
Drop the "free" badge and the "Free on Supabase" line -- every option in the wizard
is free, so neither told the user anything -- and say what the Supabase card
actually does now that connecting an existing project is the default.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* refactor(frontend): one setup checklist and one Supabase step for every host
The data table wizard, the instance database modal and the resource drawer each had
their own version of the same two interactions, and they had already begun to drift:
the wizard's Supabase resource shape was rebuilt by hand in the drawer, and the
instance checks rendered with no notion of a step being in flight.
SetupChecklist replaces LoggedWizardResult, whose only consumer was the instance
modal. It adds the running state that component lacked, so a list driven by an
endpoint that reports nothing until it returns still shows where it is. Both the
instance checks and the Supabase provisioning stages render through it.
SupabaseProjectStep owns picking or creating a project, and useSupabaseOauth owns
the popup leg. Each host keeps only what is genuinely its own: the wizard saves a
variable and resource then verifies the connection, the resource drawer fills in its
own form. Both trigger authorization themselves, so a host can offer it a screen
earlier than the step does.
The lists load behind a spinner because which mode to open on depends on whether the
account has projects; deciding that after rendering flipped the toggle under the user.
Adds a kitchen_sink playground for the checklist so the animation and every failure
position can be exercised without a backend, a superadmin, or a Supabase account.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* fix(frontend): tidy the resource drawer around the Supabase entry point
Connect Supabase was a hand-styled anchor carrying Supabase's brand hex values
rather than a Button, and it sat in a row whose other controls had settled on
unifiedSize md. Making it a Button meant SupabaseIcon had to satisfy IconType, so it
now takes `size` (deriving height/width from it) alongside the string props its other
callers pass.
The manual resource form spaced every field 32px apart and WhitelistIp added another
16px of its own, which read as a gap rather than a rhythm. One gap of 16px, with the
form itself given a little more separation from the description above it.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* fix(frontend): stop Supabase resources coming up modified when first opened
Resource forms fill in every unset property from the schema as soon as they render,
so a postgresql resource saved without region, root_certificate_pem and use_iam_auth
was dirty -- and had saved a draft -- the first time anyone looked at it. Write them
with the rest of the value.
SupabaseConnect also rebuilt the resource shape by hand instead of using the shared
helper, which is how the pooler host format ended up in two places.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* feat(backend): record where a data table came from and whether setup finished
edit_datatable_config replaces the whole datatables map and DataTable does not deny
unknown fields, so anything the request omits is dropped without a word. origin and
setup_incomplete would have been erased by any unrelated save;
preserve_unmanaged_datatable_fields carries them -- and migrations_enabled, which had
the same problem inline -- forward for entries that already exist, following renames.
setup_incomplete is what lets a row be recorded before the resource it points at
exists, so the wizard can write nothing until the user finishes. There is deliberately
no intermediate state: the setup runs entirely in the browser, so nothing server-side
could advance one.
datatable_health probes every data table at once for the settings page and skips the
incomplete ones, whose resource_path resolves to nothing yet. set_datatable_setup
patches a single entry instead of resending the map. test_datatable_connection_value
checks a connection the caller has not saved anywhere, which the wizard needs before
it has written a resource.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* fix(frontend): make destructive default and subtle buttons read red
Both variants were neutral until the pointer arrived, then filled solid red: nothing
marked the button as destructive until you were already on it. They now carry red text
at rest, with a faded red border on default and a light red wash on hover, which is
what the legacy red border style in the same file had always done.
Three call sites passed color="red" alongside a design-system variant. getStyleClass
returns before colour is read for accent, accent-secondary, default and subtle, so the
delete-migration control, its modal confirm and the import-database button had all been
rendering neutral. They pass destructive now.
The dropdown variant strips the button's own border, and matched border-border-light
literally -- a class the destructive style no longer contains.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* feat(frontend): rebuild data table setup around a read-only row
The wizard gathers intent over two steps, reviews it on a third and writes nothing
until Finish, so a billable Supabase project is created only once the user has seen
what will happen. runSetup is also the retry: every step probes for its own result
before doing anything, so running it again on a half-finished data table resumes
instead of duplicating. Its steps are keyed rather than dispatched on their titles,
where rewording one changed what it did.
The settings row stops being an editable form with a dirty/save cycle. It carries the
name, where the database came from, a health dot and two actions; everything rare
moved into the gear panel, which also offers Finish setup for a data table whose
wizard never completed. Manage is ExploreAssetButton, the control the ducklake list
already uses, and the row and panel both link out to the underlying resource.
supabaseResourceValue no longer assembles the pooler host from the region.
aws-0-<region>.pooler.supabase.com is wrong for any project Supabase allocated
elsewhere, so the host, user and port come from the pooler config endpoint.
Two data tables sharing one database also share _wm_migrations, which is probed
unqualified, so the review step warns when the database being connected is already
behind another data table.
SupabaseConnect is deleted. The resource drawer uses the shared project step
restricted to existing projects: creating one is a billed action and belongs in the
wizard, which has somewhere to report what it did. The kitchen_sink checklist
playground goes with it.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* fix(frontend): fall back to a direct Supabase connection when the pooler cannot be read
Reading a project's Supavisor config needs the database_pooling_config_read scope, which
an instance's Supabase OAuth app may never have been granted. No retry recovers from
that, and the wizard treated it as fatal: the user was left with an error and no way to
finish connecting a project that was otherwise fine.
resolveSupabaseConnection replaces the bare pooler read everywhere it happened. Asking
for session pooling and failing now yields a direct connection plus the reason, which
supabaseResourceValue already knew how to write. Nothing about the fallback is silent --
direct is IPv6-only, which is the whole reason session pooling is the default -- so the
wizard warns on its review step and the resource drawer says so in its toast.
The row is recorded before credentials are saved, so an origin claiming session pooling
has to be corrected once a direct host is what gets written; the run patches it through
set_datatable_setup rather than leaving the panel to report a mode nothing uses.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* feat(frontend): open the database behind a data table, and say when it cannot write
Every database in the list now opens the surface that owns its credentials. A postgres
one opens its resource in the editor drawer; a Windmill instance one opens the instance
modal, which is where its setup checks, password rotation and drop already lived. Both
are reachable from the row and from the panel's provenance list, and the provider icon
moved inside the button so the whole thing is one target.
CustomInstanceDbWizardModal targeted #content unconditionally, which put it underneath
the panel drawer that now opens it. It takes a target, and the panel portals it to the
body.
The status column gains a third state. The probe reports privileges but nothing gated
the dot on them, so a data table whose role cannot create tables showed as Connected and
only failed when someone ran a migration. It reads "Limited permissions" instead, and
opens the panel on the report carrying the GRANTs that fix it -- the settings page has
already probed, so the panel takes that report rather than asking the user to run Test
connection over work already done. fullyPrivileged is exported from the report component
so the dot and the report cannot disagree about what counts as healthy.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* revert(frontend): keep the data tables settings table as it was
The settings table and the setup wizard are two changes that only shared a file. Splitting
them makes each reviewable: this branch keeps the wizard, and the read-only row, gear
panel, health probe and clickable databases move to their own branch.
The rows go back to the editable form with its pickers and save footer, still opening the
wizard from Add a database. DataTableSettingsPanel, dataTableHealth and dataTableOrigin
had no other consumers and go with them; the connection report stays, because the wizard
shows it too.
DataTableSettingsType keeps `origin`: the wizard writes it, and the review step reads it
back to warn when two data tables would share one database and therefore one
_wm_migrations table.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* fix(frontend): confirm before dismissing the data table wizard mid-setup
Closing was guarded while a run was in flight and unguarded before one, which is backwards:
a run leaves a row to resume from, whereas a backdrop click on the review step threw away
the project, the pasted password and the folder with nothing to recover them from.
Backdrop, Escape and the close button now go through one path that asks first. It only asks
when there is something to lose -- no provider chosen yet, or a run that already produced a
result, closes immediately -- so the dialog does not become something to click through.
Continue in the background still leaves in one click; that exit was always the deliberate
one.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* fix(frontend): stop the wizard claiming the resource folder controls who can use a data table
"Who can use this database" was wrong. Every path that resolves a datatable:// reference --
both executors and the agent-worker endpoint -- reads the resource unchecked, by workspace
and name. A resource in u/admin is usable by everyone's scripts. The folder governs who can
see and edit the connection, and who can reference the resource directly in a SQL step;
neither is who can use the data table. The wizard was contradicting the tab's own
description two screens later.
The folder select and name field become one Path picker, the same one the resource,
variable and script forms use, so the review step reads as a resource path rather than a
permission choice. Its initialPath is snapshotted when the step opens: Path seeds itself
from it, and a live value fights the typing. Finish now also gates on Path's error, so a
taken or malformed path stops the run before it writes anything.
The button that opens all this says "Add a data table" -- the data table is what you get;
the database is a detail chosen along the way.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* revert(frontend): move the destructive button restyle out of the wizard PR
This reverts 3881e4d8ea. Making default and subtle destructive buttons red at rest changes
every existing caller of the prop -- the workspace integrations, AI skills, workspace
creation and the instance database drop -- so it is a design-system change, and the call
sites it fixed are the migrations list and the database manager. None of that is the setup
wizard.
Nothing on this branch passes destructive any more, so it leaves with no loose ends.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* fix(frontend): make the wizard stepper navigate the steps it already offers
Stepper dispatches a click and paints cursor-pointer on every reached step, but the wizard
never listened, so the breadcrumbs invited a click and did nothing.
They now reach any step already passed, in either direction: going back to check something
should not cost the progress, which means tracking the furthest step reached rather than
the current one. Forward movement still only happens through the primary action, so a step
is never reachable without having been validated -- and changing the intent revokes the
steps ahead of it, or Finish could run against a review built from something the user has
since edited. The five places that cleared the probe on an edit now do both through one
call.
During a run nothing is reachable, and the stepper says so rather than showing a pointer
over steps that will not respond.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* fix(frontend): restore the data tables description lost in the branch split
The rewritten description went into DataTableSettings.svelte shortly before that file was
restored wholesale to its pre-rebuild state, so it left with the row rework it had nothing
to do with. The tab went back to describing the plumbing -- a fully managed PostgreSQL
database, reachable from the SDK -- which never answered the question a new user actually
has: why this rather than a Postgres resource.
It leads with what a data table is, then the two things a resource cannot do -- nobody
needs the credentials to query it, and the name can be pointed at another database without
editing anything that uses it -- and closes with what Windmill runs on top. Both middle
claims are the ones every resolution path backs up: datatable:// resolves by workspace and
name, unchecked.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* fix(backend): say what is missing when a $res: or $var: reference does not resolve
Both interpolations fetched with fetch_one and mapped the error through to_anyhow, so a
reference to something deleted surfaced as "no rows returned by a query that expected to
return at least one row @workspaces.rs:2169". It names neither the kind of thing that was
missing nor its path, and it is what a data table pointing at a deleted resource reports.
They now fetch_optional and return NotFound naming the path, and datatable resolution adds
the data table on the way out: the caller asked for one by name, and a bare "resource
f/x/y does not exist" leaves them to work out which of them points at it. The health probe
is new, so this string had only just become something users read.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* feat(frontend): gate the data table wizard behind a dev flag
The wizard only appears with `dataTableWizard` set in localStorage; without it the
settings page keeps the inline-row flow it had before this branch, down to the empty-state
copy and the "New Data Table" button, and the wizard component is not mounted at all. The
existing e2e suite drives that button, so the default-off flag is also what keeps it green.
Step 2 of "your own database" becomes one list rather than a segmented control: the
workspace's Postgres resources, then a New resource card that expands in place. A
connection string is not an alternative to a resource, it is how one is written, and the
old layout taught otherwise. The card holds the same connection as a string or as fields
and carries values across when you switch, so `parse` and `compose` have to be inverses --
hence the percent-encoding on both sides, which also fixes a password containing `@`
silently corrupting in the resource form. The Supabase step now uses the same shape.
Names and paths are checked as they are typed rather than at the end of a run that may
have created a billed project first: the data table name against the charset
`edit_datatable_config` enforces, the instance database name against what
`setup_custom_instance_db` will accept, and the resource path against both the resource
and variable namespaces, since the run writes to both and both writes upsert.
`test_datatable_connection_value` refuses `$var:`/`$res:` in its body. It feeds
`transform_json_value_unchecked`, which resolves references with no permission check of its
own, so an admin could otherwise have had the API server decrypt any workspace secret and
hand it to a host the same request chose -- without the audit trail a variable read leaves.
Callers testing something unsaved hold the literal value already.
Alert, SetupChecklist and postgresConnectionString change for everyone, not just behind the
flag: body-only alerts no longer reserve an empty title row, the checklist can nest the
checks a step is made of, and the connection-string parser is shared with the resource form.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* chore: pin ee-repo-ref to the EE branch merged with EE main
The Supabase proxies the wizard calls are still unmerged, so the ref cannot be an EE
main commit yet; it now names that branch merged with EE main rather than the branch
alone, which was nine commits behind and would have been built against a CE main it
never saw.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* feat(frontend): gate the supabase resource path behind the dev flag
* test(frontend): pin connection string parsing to libpq behaviour
* fix(frontend): keep the supabase resource link off the popup callback path
* refactor(frontend): load the supabase resource dialog only behind the flag
* fix(frontend): refuse a resource path the wizard run does not own
* fix(frontend): let a failed data table setup be corrected without losing what it made
* fix(frontend): let a failed setup reuse the resource path it claimed
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* fix(backend): record the two data table connection tests in the audit log
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* fix(frontend): use Section for the data table wizard advanced group
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* fix(frontend): read connection strings the way libpq does
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* fix(backend): pin the ee ref back to a commit this branch can build
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* fix(frontend): keep a failed setup's claims across the redirect and rollback
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* fix(backend): probe a data table with the auth mode the worker will use
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* fix(frontend): keep every part of a connection string through the round trip
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* refactor(frontend): give a setup run one record of what it created
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* fix(frontend): mark a resource claim by edited_at, not its creator
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* fix(frontend): mark every claim by revision, and keep an unconfirmed project's secret
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* fix(frontend): refuse to test or save behind a connection string that will not parse
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* fix(frontend): refuse a connection string carrying options the resource cannot hold
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* fix(frontend): allowlist the connection-string parameters a resource can honour
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* fix(frontend): guard every created Supabase project, not just the last one
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* feat: three-step wizard for importing a hub project
Importing used to be a single page that inherited whatever workspace happened to
be active, with no way to say where the project should go — the hub cannot know,
since it only ever links to *an* instance. `/projects/import` now asks: which
kind of destination, which workspace, then imports.
Nothing is created, switched or written until the last step runs. The wizard's
state is a plain value in the URL (`importWizard/plan.ts`), so the back button,
the stepper and the Back control are the same operation, and none of them can
strand a half-created workspace — there is no state anywhere else to unwind.
`importWizard/execution.svelte.ts` is the only code that acts on a plan: it runs
create → fetch → import as an observable task list, reuses what already
succeeded when retried, and offers to delete the workspace it created if the run
stops early. Its UI needs — the data table migration review — are injected, so
it holds no components.
The old `/projects/install` becomes a redirect: hubs upgrade on their own
schedule and a self-hosted one may keep pointing at it for a long time.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* fix: let the import wizard survive sign-in and a missing workspace
Signing in with `rd=/projects/import?hub=...` dropped the destination: the login
redirect only honours `rd` verbatim for `/user/workspaces`, so anyone with more
than one workspace landed on the workspace picker instead — the page the wizard
exists to replace, asking the question it was about to ask. Both copies of that
logic now allow the wizard through.
The root layout's "no workspace selected" redirect skips the wizard too. It
picks the destination itself and may end in a workspace that does not exist yet,
so bouncing it to the picker forces the very choice it is there to make.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* chore: bench page for the import project card
/kitchen_sink/import_project_card renders the card against fixtures — a real
project, an oversized one, a minimal one — so its layout can be judged without a
hub running or an import in flight.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* fix(frontend): do not warn about renaming an item that does not exist yet
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* fix(frontend): make the review step read as one list of what will exist
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* fix(frontend): keep the picked Supabase project across the redirect, reject connect_timeout
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* fix: check the data table connection from a worker, not the API server
The wizard's connection check ran on the API server through two endpoints added
for it. That server is a different machine with a different identity, so the
answer was about the API server rather than about the worker that will run the
queries: a host reachable from one is not necessarily reachable from the other,
and IAM RDS and Azure workload identity authenticate as whichever process opens
the connection.
Run the privilege query as a preview job instead. A job goes through the
worker's Postgres executor, which is where `PgAuthMode::of` already picks the
authentication mode, and it takes either a resource value or a `$res:` path
exactly as a Postgres step does. Postgres composes the suggested GRANT
statements through `format('%I')`, so identifier quoting stays where it is
already implemented.
Removes `test_datatable_resource_connection` and
`test_datatable_connection_value`, and `connect_as_the_worker_would` with them.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* refactor: fold check_datatable_connection back into its only caller
The helper was split out so the two connection-test endpoints could share a
body. Those endpoints are gone, leaving one caller.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* revert: keep the data table connection check schema inline
It was lifted into components so three endpoints could share it. Two of those
are gone, so it is back to one user and the extraction changes nothing.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* fix: restore openapi.yaml to the branch point
The previous commit restored main's tip rather than the merge base, which
carried three unrelated main-only changes into this branch: the resource
mcp_tools truncation fields, the execution_mode description, and a version bump.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* refactor(frontend): drop four effects from the data table wizard
Each was doing work a derived, a load callback or a real entry point does
better.
- The name conflict is kept with the name it was raised for and derived from
it. As an effect it was correct only because it never read what it wrote:
the pre-flight sets the message and the effect does not re-trigger, so adding
a read would have cleared it the instant it appeared. The message now also
comes back if the taken name is retyped, which is what the server will say.
- The default resource selection is seeded inside the fetcher that loads the
list, where "has the fetch settled" cannot be asked wrong.
- Reset-on-open becomes an exported open(), called by the settings page, so a
fresh run is set up by the act of opening rather than by a flag emulating
mount.
- The OAuth connects and the folder list become resources; supabaseAvailable
and folders are derived from them. defaultFolder takes the list rather than
reading it, so the fetch can seed off its own result.
Leaves the debounced path check, which is async with an out-of-order guard.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* refactor(frontend): drop three effects from the Supabase branch
- useSupabaseOauth reports success as onAuthed, alongside the failures it
already reported. SupabaseResourceConnect was watching `authed` to find out;
it takes the callback instead, keeping the guard that stops an authorization
started elsewhere on the page from opening its dialog.
- SupabaseProjectStep loads its orgs and projects through a resource keyed on
the token, so the `loaded` latch goes and re-authorizing reloads rather than
keeping the lists from the expired session.
- SetupChecklist records what the user toggled and derives the open state from
it, a failed step defaulting to open. Recording the open state instead needed
an effect to force it, and that effect re-ran on every progress update, so a
description closed while anything was still ticking reopened. A close now
holds for the life of the checklist, including across Try again.
Leaves the message listener, which subscribes to another window.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* fix(frontend): confine the modal restyle to the wizard, and trim the comments
The wider side padding and lighter dialog heading were changing all 17 Modal2
dialogs to suit this one flow. They move behind an opt-in `formStyling`, taken
by the three dialogs this branch owns; every other Modal2 renders as it did.
Also drops two comments that cited a design approval rather than a constraint,
and shortens the blocks that had grown past the four lines AGENTS.md asks for.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* fix(frontend): use the accent token for the wizard's links
`text-blue-500` is the marketing blue `#3B82F6`, which brand-guidelines.md
rules out in the app interface.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* chore: point ee-repo-ref at the EE branch head
Picks up EE main, which the branch now needs, and the Supabase proxy auth fix.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* fix(frontend): read sslmode by name, and stop decrypting a secret to date it
- `sslmode` was found by searching the query text, so it also matched inside
another parameter's value: `?application_name=sslmode=disable` passed the
allowlist on the parameter name and then parsed as a request to turn TLS off,
which both the wizard and the resource form saved and probed. Parsed with
`URLSearchParams` by exact name, with a test.
- `secretMark` read the variable with `decryptSecret` defaulted to true, so
every write decrypted a secret nothing reads and recorded the decryption --
including someone else's on the retry about to refuse it. It wants only
`edited_at`, which is returned either way.
- The probe gave up at 15s while the worker allows its Postgres connect 20s, so
a host that accepts the connection and never answers was cancelled and
reported as a missing worker rather than a failed connection.
- The create-mode region and project name did not report an intent change, so
renaming a project after a name collision left the failure naming the old one.
- Two comments described the code as it was before the claim mark became a
revision, and a doc comment outlived the field it documented.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* fix(frontend): read connection parameters the way libpq does
One reader for both the parser and the allowlist, since they disagreed about
what a string says in two ways that both ended in a weaker connection than was
pasted:
- `URLSearchParams.get` takes the first of a repeated parameter and libpq takes
the last, so `?sslmode=disable&sslmode=require` was read as `disable`.
- The allowlist folded the parameter name and the parser did not, so
`?SslMode=verify-full` was refused by neither and honoured by neither, and
saved as the `require` default.
The parked Supabase run is now handed to `open()` rather than read back off the
`resume` prop it was just assigned to, so restoring it does not depend on when
that prop reaches the component.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* fix(frontend): keep connection parameter names case-sensitive
libpq does not fold them: `?SslMode=disable` is rejected as an invalid URI
query parameter rather than read as `sslmode`, which a local server confirms.
Folding made Windmill accept and honour a string Postgres itself refuses;
naming the parameter instead tells the user why it cannot be stored.
The last-value-wins rule for a repeated parameter is unchanged, and matches
what the same server does.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* fix(frontend): seed the Supabase organization from the project it selects
The loader took `orgs[0]` independently of the project it seeded, so an account
whose first project sits outside its first organization had the review step name
an organization the database does not belong to. Picking a project by hand
already derives it; the seeding now does the same.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* fix(frontend): let the probe report an empty search_path instead of failing on it
`format('%I', NULL)` raises rather than returning NULL, so a role whose
search_path names no valid schema failed the whole privilege query and was
reported as an unreachable database. That is the one case `fix_search_path`
exists to name, and it never reached the user. Verified against a local server
with `SET search_path = ''`.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* fix(frontend): say which of the two refusals a connection string hit
Making parameter names case-sensitive gave `unsupportedConnectionParam` two
reasons to refuse, and the single message explained only one. `?SslMode=` was
answered with "Windmill cannot store SslMode on a Postgres resource", which is
false twice over: sslmode is exactly what the resource stores, and the string
asks for nothing because Postgres rejects the URI. It now names the spelling
when the parameter is one we keep, and the storage limit otherwise.
The folder-list guard also still read the `resume` prop that `open(parked)` was
changed to stop trusting, so the resumed path now comes from whatever `reset`
was handed.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* fix(frontend): leave the Supabase organization unset when the lookup misses
Falling back to the first organization named one the seeded project is not in,
since `supabaseSummary` prefers `intent.org` over the project's own. Unset, it
falls through to the project's organization identifier — the right one, spelled
as a slug rather than a name.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* fix: harden the import wizard and put it on the design system
Review fixes, then the parts of the wizard that were hand-built where the
design system already had an answer.
Correctness:
- Hub SVGs are sanitised with DOMPurify before `{@html}`. The earlier comment
claimed the markup came from the hub's own icon package rather than user
input, which the custom-URL feature makes false: the hub is whatever address
the user typed.
- The run owns navigation while it is in flight. The stepper refuses to move,
`beforeNavigate` cancels browser back/forward, and unmounting resolves a
pending migration review so the executor cannot hang waiting on a component
that is gone.
- The folder edited on the last step reaches the executor, so a retry after
changing it imports where the field now says.
- `validateWorkspaceId` and the workspace-entry pair (`listUserWorkspaces` then
`switchWorkspace`) are extracted, so the wizard and the real create form
cannot drift on what an id is or on what entering a workspace means.
Design system:
- The destination tiles are `RadioCard`, which gains `showRadio` and a snippet
`description`; the wizard turns the glyph off because the border and tint
already say which one is picked. `RadioCard` now also carries `role="radio"`
and `aria-checked`, which it had neither of, and marks its selection with
`surface-accent-selected` — the token `FileExplorer`, `TriggersTable` and
`RunnableRow` all use for the chosen row.
- Form labels follow `brand-guidelines.md` — sentence case, real `<label>`
elements so the text focuses the field, Caption-styled errors — rather than
one-off 11px uppercase tertiary text. They use the lighter secondary weight,
since the fields arrive prefilled and the value carries the meaning.
Folder choice, restored and merged:
- Picking an existing folder came back for an existing-workspace destination.
`FolderPicker` takes a `workspace` prop so it can list a workspace without
switching to it, and resolves `whoami` there — its write flags came from
`$userStore`, i.e. the wrong workspace, which rendered every real folder
read-only and unselectable. A new workspace has no folders to choose between,
so it is not asked.
- The progress list and the imported paths are one component: the paths hang
off the import task that produces them instead of forming a second list.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* fix: address review — history, workspace restore, SVG styles
Two blockers and the findings around them.
- The wizard pushed history entries for navigations the user did not ask for.
`folder` initialises to the project slug while the plan holds none, so the
mirroring effect fired on mount and pushed a URL differing only by `&folder=`;
pressing Back returned to the entry without it, which re-fired and re-pushed.
Back could never leave step 3. `go` now takes `{ replace: true }`, used by that
effect and by the step guard — the two navigations the page decides on its own.
The comment claiming `go` replaced was written without checking that `goto`
forwards to SvelteKit, which defaults `replaceState` to false.
- Undoing a run left the app pointing at the workspace it had just deleted:
`#ensureWorkspace` switches in, `deleteCreatedWorkspace` deleted without
switching out. The dead id was persisted on the next navigation, `getUserExt`
then returned undefined, and the following reload logged the user out. The
executor now remembers where the app pointed before it started and puts it back.
- `FORBID_TAGS: ['style', 'image']` on the hub SVGs. The profile allows both; an
inline `<svg><style>` is document-scoped, so a hostile hub could restyle this
page — including moving the wizard's own Import and Delete controls — and
`<image href>` is a beacon. The doc comment asserted a guarantee the config did
not deliver.
- The existing-workspace id is validated like the new one and encoded where it is
interpolated into `/api/w/<ws>/...`; it arrives from the URL exactly as the new
one does and ends up in `workspaceStore`.
- `AppConnectInner`'s two RadioCards get a `role="radiogroup"` wrapper, since they
now carry `role="radio"` and a screen reader cannot place a radio without one.
- `FolderPicker` records a created folder against the membership it is reading, and
before reloading, so a non-admin can re-pick the folder they just made in another
workspace instead of finding it `(read-only)`.
- Step 3 shows trigger and data table migration counts once the export is fetched.
The page this replaced showed them, and the warning underneath talks about
triggers the user was never told about.
- First tests for the two pure modules: the workspace-id contract the wizard and
the create form must not drift on, and the plan/URL round trip the whole wizard
rests on.
- Doc fixes: the retry claim (the granularity is the task, not the item), the bench
header, a fractional `?step=`, an empty name in the destination card, and the
three copies of one rationale AGENTS.md asks to state once.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* test(frontend): pin which refusal a connection string gets
The two messages differ in what they ask the user to do, and the condition
choosing between them — whether the lowercased name is one the resource keeps —
is not visible from either call site. `Connect_Timeout` is the case that keeps
them honest: miscased *and* unstorable, so respelling it would not help and the
message must not suggest it.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* fix(frontend): hand a failed Supabase leg back to the page holding its run
Denial, a token error and a malformed callback all sent the user to
/resources whether or not a run was parked. Nothing else consumes the park, so
the run stayed in sessionStorage and sprang the wizard open on an unrelated
later visit instead. A parked run now lands on the data tables tab, where the
wizard resumes on the setup step and can authorize again.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* fix(frontend): let a run reuse the name of a row it could not take back out
`removeRow` reports `kept` when the undo cannot reach the server, so the row
this run wrote stays in the workspace config and comes back in `existingNames`.
The client-side name check then refused the retry on the run's own name, with
no way forward but a rename. The instance database name has carried the same
exemption since it was written; this is the data table name catching up.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* fix(frontend): discard a variable check the wizard has moved on from
The post-await guard compared only the path, and the path is built from the
review step's fields -- so picking an existing resource stops the wizard minting
one without changing it. A check already in flight then answered for a branch
nobody was on, and a `true` disabled Finish over a path the run no longer
writes. The cleanup cannot help: it cancels a pending timer, not a live request.
Both sides of the await now ask the same question.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* chore: update ee-repo-ref to 483513b70979aa9497cab869837108d948449984
This commit updates the EE repository reference after PR #715 was merged in windmill-ee-private.
Previous ee-repo-ref: 8604b30a740c5620069208801a7ae50937b61977
New ee-repo-ref: 483513b70979aa9497cab869837108d948449984
Automated by sync-ee-ref workflow.
* feat: a setup step for what the import cannot bring with it
A project's data tables and credentials cannot travel with it: a data table is a
named database connection the workspace owns, and resource values are secrets the
hub never publishes — `importResourceStub` creates every one of them empty. The
wizard used to state that as a dead end. Mid-import it asked the user to cancel,
create the data tables by hand and start over, which for a *new* workspace was
every single time, since a new workspace has no data tables at all.
Step 4 replaces that. It appears only when the run leaves something undone, lists
what that is, and does it in place: a Postgres resource per missing data table
(one merged `editDataTableConfig` write, then the migrations), and the existing
resource editor for each credential. Skipping is allowed and says plainly which
parts of the project will not run.
It is self-sufficient from `workspace` + `slug` — it re-fetches the export rather
than reading the executor — so reloading on it works and the plan in the URL stays
the whole state. Rows are marked done rather than removed, with SaveButton's
confirmation flash, because a checklist line that vanishes when completed reads as
something going wrong.
Two things the step needed from elsewhere:
- `ResourceEditorDrawer` gained `onSaved`. `onRestored` fires only when an old
version is restored, so a caller showing state derived from the resource had no
way to know a save had happened — the row kept saying "missing token" after the
token was filled in.
- The run now loads the destination's membership into `userStore`. The wizard's
page is reparented out of `(logged)` and never gets that layout's `getUserExt`,
so anything asking what the user may do reads "no user" and refuses.
`applyOneMigration` is exported for the same reason the step exists: the import
skips a migration whose data table is not configured, and this is where it is not
skipped any more.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* feat: set up data tables through the wizard, not a hand-rolled form
The setup step drove `editDataTableConfig` itself, which meant it could
name a table and record migrations but could not create the database
behind it — the case a brand-new workspace is always in. It now opens
`AddDataTableWizard`, which owns that whole path.
Four additive props carry what the import flow needs and nothing else,
so `DataTableSettings` is unchanged:
- `initialName` — the migrations only apply to a table of the name they
target, so the wizard opens on it. Still editable.
- `modalTarget` — `#content` is the `(logged)` shell's scroll container,
and the import page reparents out of it, so the portal would find
nothing and the dialog never appear.
- `finishAlso` / `onFinishAlso` — running the migrations was invisible
until it had already happened. It is now named on the final button
("Create data table and run migrations") and reported as the last row
of the wizard's own checklist, failing there rather than silently.
Rows are marked done rather than removed, so the list still says what
was set up. Resources keep their card and swap "Fill in" for "Saved".
`Finish` is the primary and stays disabled until nothing is outstanding;
`Skip for now` sits beside it, and the info alert explaining the skip
turns into a success one when everything is configured.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* fix: show each credential's own integration icon, and cut the data table blurb
The credentials list marked every row with the same key glyph, so the only
thing distinguishing them was the path. `IconedResourceType` renders the
provider's own mark from the resource type already on the row, falling back
to a generic box for types with no icon.
The data table explanation said "a data table is a database this workspace
owns" directly under a label reading "Data tables to set up", and "this
project ships with one it expects to find" directly next to the count that
says so. Both halves went; what a data table is *for* and what to do next
are what a first-time reader needs.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* fix: build the Google sign-in button from the design system
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01HtYCxXEn2WujwVvh5aZRCa
* fix: qualify a data table FK target with its schema
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01HtYCxXEn2WujwVvh5aZRCa
* fix: confirm before skipping an unconfigured data table
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01HtYCxXEn2WujwVvh5aZRCa
* fix: show a loader while the wizard hands off to the workspace
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01HtYCxXEn2WujwVvh5aZRCa
* fix: resume an import whose workspace was already created
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01HtYCxXEn2WujwVvh5aZRCa
* refactor: draw the import run with SetupChecklist and ask before leaving it
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01HtYCxXEn2WujwVvh5aZRCa
* fix: portal the setup step's confirmation above the data table wizard
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01HtYCxXEn2WujwVvh5aZRCa
* fix: supply the three APIs the import wizard already calls
`AppConnectDrawer`, `ImportProjectStep` and `execution.svelte.ts` landed
calling into props and exports that were never committed alongside them,
so the branch did not type-check. Each half is here now:
- `AppConnectInner.fillPath` — connect into a resource that already
exists instead of refusing the path. The import creates every resource
as an empty stub, so without it the connect flow can only ever say
"already exists, delete it or pick another path". Opt-in: unset, the
flow still refuses to write over anything, which is what `ResourcePicker`
and the resources page rely on.
- `ProjectContentBadges.contentSummary` — the badge counts as one line of
text, for the import step's task row. Shares `kinds()` with the badges
so a project cannot be counted two ways.
- `installProject.onMigrationsStart` — fires before the reviewed
migrations run, which is the only signal that phase has begun; the
import step draws them as their own checklist row off the back of it.
Also fixes the wizard wedging itself shut: `requestClose` set `dismissing`
and cleared it after awaiting the confirmation, so an `ask` that threw left
the flag set — and the backdrop, Escape and the close button all return
early on it, leaving a reload as the only way out. Now `finally`, plus a
reset on open, since a promise that never settles never reaches `finally`.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* fix: offer Connect wherever the connect dialog would actually work
The setup step decided a resource was connectable by looking only at the
instance's configured OAuth clients, while the dialog it opens also accepts
a provider the registry marks client-credentials-capable — those carry their
credentials per resource, so no superadmin has to configure anything. The
two disagreed for bitbucket, coupa, linkedin, servicenow, spotify, visma,
xero and zoho: the step showed "Fill in" where the dialog would have
connected.
Rather than copy the predicate, `oauthRegistry.ts` now owns it, and
`AppConnectInner` reads it from there. That folds in three lookups of the
same registry that had drifted apart inside the component — `registryEntry`,
`isCcCapable`, and a raw index at the connect-template site — so the sandbox
suffix rule is written once instead of twice.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* refactor: draw the project card's icons from the ones we already ship
The card fetched each integration icon from the hub as SVG markup, sanitized
it and injected it with `{@html}`. The hub renders those icons out of
`@windmill-labs/components` — this frontend's own package — so it was a
cross-origin round trip to get our own assets back, and it made the card
depend on a read that a hub with `API_SECRET` set refuses outright.
`hubAppIcon` resolves them through `appIconComponent` instead, so they are
components again: no fetch, no DOMPurify, no `{@html}`, and they paint on
first render rather than after a round trip. Integration icons now show even
against a gated hub; only the summary and the uploaded logo still need it.
The one thing the hub was doing for us was resolving `postgres` to the
`postgresql` mark, which its `aliasApp` bridges and our icon map does not —
so that single alias comes along, next to a note pointing at its counterpart.
`ImportProjectSummary.hub` goes with it: it existed to build icon URLs and
nothing read it afterwards.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* fix: two regressions this branch introduced into shared drawers
Found auditing the files here that are used elsewhere in the app.
`AppConnectDrawer`: the guard added to stop the inner component being opened
twice compared the last resource type against the current one, and reset it
to `undefined` on close. The resources page opens the drawer with no resource
type, so both sides were `undefined`, the guard matched, and the second
opening never handed off — the type list came up empty. The drawer destroys
its content on close, so this hit every reopen. Now a flag armed per `open()`
call, which cannot collide with a resource type.
`ResourceEditorDrawer`: adding `onSaved` had turned the Save handler into
`await save(); closeDrawer()`, so the drawer stopped closing immediately and
waited for the write. `save()` catches its own errors and never rejects, so
that was pure added latency for all ten callers. It now starts the save,
closes as it always did, and awaits only to fire `onSaved`.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* fix: keep the destination through a first-time signup
Someone who follows a shared hub project without an account signs up, and the
OAuth callback sends a first-time user to onboarding — dropping the `rd` it
had already read out of localStorage. They finish onboarding in an empty
workspace with no sign of what they came to import, and have to go back to the
hub and click again. That is the path this feature exists for.
The callback now passes `rd` on, and onboarding's two exits honour it instead
of hardcoding `/user/workspaces`. Same-origin relative paths only: `//host` is
a valid URL that leaves the origin while still starting with `/`, so the guard
rejects it rather than bouncing a fresh account off-site.
Nothing changes for a signup without `rd`, which is every existing one.
Gets the user to the wizard with the project in hand; they still pick a
destination on step 1. Having onboarding create the workspace and hand into
step 3 is the larger version, not done here.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* fix: address review — username, name length, leaving mid-run
**The new-workspace username was never validated.** Step 2 shows the field when
the instance does not derive one, but neither the Continue gate nor
`planProblem` looked at it. `create_workspace` does not close that hole:
`nw.username.ok_or(...)` accepts `Some("")` and never runs the `VALID_USERNAME`
check `join_workspace` does, so a cleared field created a workspace whose owner
has an empty username, and a digit-first one was stored verbatim. Both now
refuse, using the same `validateUsername` the sibling creator has always run.
**The name length was unchecked**, so a >50-char name walked through two more
steps and failed at create. `WORKSPACE_NAME_MAX_LENGTH` sits next to the id
limit and `planProblem` checks it.
**Leaving mid-run did not stop the run.** The dialog promised "The import stops
where it is. Coming back to this link picks it up again", but navigating away
only unmounted the UI: the executor kept going, reached `done`, and called
`clearParkedImport()` — so returning to the link tried to create the workspace
again and failed with "already exists". Worse, the review drawer's teardown
resolved the pending review to `false`, meaning "skip the migrations", and the
orphan imported every item without the tables they need.
Nothing can abort a request already in flight — `installProject` takes no
signal — so `abandon()` stops the run at the next phase boundary and leaves the
workspace parked, and the teardown now resolves `'abort'`, which stops the
import rather than silently dropping the migrations.
Also drops a stale JSDoc above `hubAppIcon` still describing the fetch-and-
sanitize implementation that `ea31f73ed3` replaced.
Adds the coverage the review asked for: the parking decision at the end of a
run, and the two validation gates.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* fix: address review round 2 — XSS, retargeting, abandonment, redirects
**Hub data-table names were inlined as raw HTML.** `skip()` built the
confirmation body as an HTML string, and `createAsyncConfirmationModal` renders
`children` through `createRawSnippet`. A `datatable_name` comes straight from
the hub export, and a hub is not necessarily ours — `hub_base_url` is an
instance setting — so one carrying an event-bearing element ran script in this
authenticated origin. Escaped. Same class as the round-1 SVG finding, in a
different sink.
**The setup step read unretargeted resource paths.** `installProject` rewrites
every resource into `f/<folder>/`, but the step re-fetched the raw export and
used its paths verbatim. Importing into a folder other than the slug made
`getResource` throw for every stub, the catch skipped them, and the step
reported "You're all set" over credentials nobody had filled. It now retargets
the same way the import did, and filters to the import folder — the containment
guard the installer applies, so a crafted export cannot name a path outside it
and get offered for editing.
**Abandoning only stopped between phases.** `installProject` takes a `stopped`
callback now, checked before every write loop, so leaving mid-run stops the
remaining items instead of just the remaining phases.
**A failed setup migration reported success.** `runMigrationsFor` swallowed the
error, so the wizard marked its "Run migrations" step done and closed over a
failure — leaving the data table name taken and no way back to retry. Rethrown,
which is what the wizard's checklist reads.
**`onboardingDestination` used a weaker redirect check.** `/\evil.com` passes
`startsWith('/') && !startsWith('//')` but WHATWG URL parsing resolves it to
another origin. Replaced with `toSameOriginRelativePath`, which already rejects
that, control characters and oversized values.
**Two workspace ids reached step 3 that the backend refuses:** a blank one (the
Continue gate never required `id.trim()`) and `global`, which
`check_w_id_conflict` rejects outright while `existsWorkspace` reports it free.
Also: `size="xs2"` → `unifiedSize="2xs"`, and two doc comments reattached to the
functions they describe.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* fix: address review round 3 — both regressions from round 2
**`resume` never rejected a run from a different plan.** The seed computed its
tag from the plan being rendered, so `run.key === planKey` was true by
construction and the guard could not fire — the comment claimed the opposite.
Finishing an import into workspace X, stepping back to pick workspace Y, then
returning showed X's finished checklist against Y's plan, with a Continue
button, over an import into Y that never happened. `ImportExecution.planTag`
now carries the plan the run was made for, and the seed uses that.
**Abandoning mid-import still reported `done`.** `installProject` returns early
when `stopped` goes true, and it returns exactly as it does on success, so the
tail of `#import` could not tell the two apart: a run stopped after 3 of 10
items wrote `import: done — 3 items`, no error, `done = true`. Since the page
hands that run back on return, the primary button became Continue rather than
Retry and the seven skipped items were silently lost — breaking the promise the
leave dialog makes. The tail now checks the flag and leaves the run failed and
retryable.
`abandon.test.ts` was a hand-written copy of the parking decision, which is why
it guarded neither. It now drives a real `ImportExecution` with the install seam
mocked, abandons from inside the write loop (the only way it happens — `run()`
clears the flag on entry so a retry can proceed), and asserts `done`, the error,
and both parking outcomes. `planTag` is covered too: different destination,
different project, and that the editable folder does not change it.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* fix: address review round 4 — unreviewed SQL, premature finish, stale Back
**Setup ran hub SQL nobody had seen.** Step 3 reviews the migrations it can run
there, but the ones deferred to setup went straight to `applyOneMigration`
against whatever database the wizard was pointed at — which can be an existing
resource holding unrelated objects. Each unconfigured row now carries a
disclosure showing exactly what will run, before "Set up" runs it.
**Finish was live while the setup decision was still outstanding.** For a project
with migrations but no resources, `execution.done` exposed the button while
`listDataTables` was still in flight and `setupNeeded` was still false — clicking
in that window left for the workspace and skipped a step the answer, a moment
later, said was needed. It now reads "Checking…" and is disabled until the check
settles.
**A reload on step 4 turned Back into a re-import.** `resume` only carries the
page's in-memory execution, so after a reload Back mounted a fresh step 3
offering Import over a bundle already in — and on a new workspace, a create that
now fails because the finished run cleared its parking. Back exists only while
the page still holds the run, which excludes exactly that case.
**`validateWorkspaceId` over-rejected a fork named `global`.** It reaches the
backend as `wm-fork-global`, which is accepted; only the effective id is checked
now, so a plain `global` is still refused. Covered by a test.
**An abandoned run left the migrate row spinning.** It is appended once the
review settles and set running by `onMigrationsStart`; stopping before its loop
left it on `running` forever, reading as work still in progress on a run that
had stopped.
Also moves the `run()` contract back onto `run()`, and gives `ImportSetupRow` an
optional `extra` snippet for detail that does not fit on one line.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* fix: close the other two routes back into a run-less import step
Round 4 gated the setup step's own Back button when the completed run was no
longer in memory, but that is the least used of three ways back into step 3, and
both reviewers landed on the same gap.
The stepper renders every earlier step as reachable, and `importIsRunning()` is
false after a reload, so its "Import" tab walked straight there. And `onFinish`
pushed step 4 over step 3, leaving the browser's own Back pointing at the same
place.
After a reload there is nothing to hand back: the executor was in memory, and a
clean finish clears the parking, so step 3 mounted with `resume` undefined and
offered a fresh run — re-importing a bundle already in (a wall of path
conflicts), or on a new workspace re-running a create that now fails as already
existing, with no Delete offered because that execution never made it.
`ImportWizardSteps` takes a `lowestStep`, which the page raises to 4 exactly
when the run is gone, and the step-3 → 4 transition replaces rather than pushes.
Verified against a real reload: the stepper stays on step 4 and says why, and
browser Back lands on step 2 with no runnable import.
Also adds the migration-phase abandonment assertion the review asked for — that
no task is left on `running` when a run stops.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* test: make the migration-phase abandonment test actually reach it
It asserted over a branch it never ran. The mock `installProject` never called
`onMigrationsStart`, and with `migrations: []` in the export and
`reviewMigrations` returning nothing, `#import` never appended the `migrate` row
at all — so "no task is left running" was true because no task existed. The
comment was wrong too: the real `onMigrationsStart` fires at the head of the
migration loop, past every item loop, not at the start of the writes.
The mock now mirrors that order — item loops, then `onMigrationsStart`, then the
migrations, with `stopped` checked before each write — and a second hook lets a
test abandon after the row is running. The export ships a migration and
`reviewMigrations` returns it, so the row exists to be pinned, and the test
asserts it exists before asserting its status.
Checked by removing the fix: it fails with `expected 'running' not to be
'running'`, and passes with it restored.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* refactor: ask the instance what exists instead of remembering it
The wizard kept a note in `sessionStorage` — "this run created workspace X" —
so a reload could tell that a create had already happened. A note is a second
copy of a fact the instance already holds, and it could outlive the workspace
it named: the comment on `createdWorkspace` said a parked id might point at a
workspace someone else made at that id afterwards, and that there was no way to
tell, because a workspace carries no discriminator.
It carries `owner`. It is set to the creator's email at `INSERT INTO workspace`,
`listWorkspaces` already selects it, and the generated `Workspace` type already
has it. So the question the note was answering can simply be asked:
`probeWorkspace` returns whether a workspace with the plan's id exists among
the caller's, and whether they own it. Ownership is what makes adopting one
safe — an id that exists but belongs to someone else is not this run's work.
`parking.ts` and its test are gone. Nothing in the wizard writes storage now:
the plan is in the URL, what exists is in the instance, and what is in flight is
in memory, which is where in-flight things belong.
`probe.ts` also carries the two reads the follow-up needs — which of the paths
an import would write are already there, and whether a migration's tables exist.
The second is the ground truth for "did this migration run", covering both paths
`applyOneMigration` takes: it records a migration when the data table has them
enabled, and otherwise runs the SQL as a job nothing remembers. The tables
outlive both. It returns `undefined` rather than `false` when it cannot tell,
since "not there" invites a caller to run the migration and "cannot tell" does
not.
Verified against a real reload mid-run: the second attempt makes no
`createWorkspace` call, one `workspaces/list` call, and carries on to the fetch.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* feat: import only what the destination does not already have
A retry resent the whole bundle. Everything that had already landed came back
as "already exists" — nine failures over work that had succeeded, with no way
to tell those from the ones that genuinely failed. The same thing happened
importing into a workspace that already held some of the project.
`installProject` now takes `alreadyPresent`, checked after retargeting because
that is what the items will actually be called, and `probeImportedPaths` fills
it from the destination on every run. On a workspace the run just created the
answer is empty and nothing is skipped, so this costs four scoped reads and
changes nothing about a first import.
Skipping is not replacing. An item that is there is left exactly as it is —
the same promise `updateIfExists: false` already makes for a resource whose
value someone has since filled in.
`InstallResult` gains `skipped`, because "already there" is neither an import
nor a failure and reporting it as either is a lie. The checklist still lists
every item the project ships; a skipped one shows as skipped and says why. The
import row now counts the three outcomes separately — `8 already there` rather
than a green tick over `2 apps, 4 scripts, 2 resources` it did not write. That
last part needed the pre-run breakdown to stand down once the run has an
outcome of its own, or it went on claiming the import had happened.
Checked by removing the gate: two of the four new tests fail. Verified against
a real backend by re-importing Calendly into a workspace that already had it —
0 failures, 0 create requests, and the row reads "8 already there", where the
same run previously produced 9 conflicts.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* fix: skip triggers that are already in the destination
probeImportedPaths asked about scripts, flows, apps and resources but not
triggers, so a retry replayed every trigger create into an API that rejects
an existing path — reporting a failure for something already there, which
is the wall the presence probe exists to remove.
Triggers have no prefix-filtered list endpoint, so they cost one call per
kind; the probe only asks when the project actually ships triggers.
The presence set is now keyed by kind as well as path. The five kinds share
one f/<folder>/ namespace, so a trigger and a script may both be called
f/cal/sync, and a flat path set would let either one mask the other.
Also drops expectedPaths, which was exported and tested but never called.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Xg8vXUuHCH3aRkf91sfxjx
* fix: don't let a failed migration read as a finished setup
Three linked gaps around the step-4 data table setup:
Finish setup was clickable while the step was still loading. Empty rows and
blanks made outstanding === 0, which reads the same as having nothing to do,
so a quick click left the wizard before the missing data table was even
discovered. Skip already guarded on loading; Finish now does too.
When the data table wizard's appended migration step failed, run.result kept
runSetup's successful verdict, so the primary action offered Done over a
failed row and closing raised no warning. The failure is now tracked apart
from run.result, and Try again re-runs only the appended step — re-running
the setup would ask for the table name it just took and be refused.
A failed row in the import step reopened the full wizard, which rejected the
name it had itself created, leaving no way back to the migration that
actually failed. Such a row now offers "Run migrations again" instead.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Xg8vXUuHCH3aRkf91sfxjx
* fix: read the destination's real state instead of inferring it
Three ways step 4 could report work that had not happened:
AddDataTableWizard wrote through $workspaceStore while ImportSetupStep used
the workspace from the URL plan. The import page is reparented out of
(logged), so nothing re-runs the layout's workspace persistence; after a
reload the store still named the workspace the user came from. "Set up"
would then create the data table there and run the migrations in the
destination. The workspace is now a prop, defaulting to the store so every
other call site is unchanged.
load() marked a row done whenever the data table name existed. The wizard
creates the table and the migrations run after it, so a table can be there
with none of the project's tables inside it — and a reload rebuilds rows
from scratch, hiding the failure. It now asks probeMigrationApplied, which
already existed for exactly this question. An undefined answer ("cannot
tell") keeps whatever the row said rather than inventing an outstanding row.
A reviewed migration could fail in step 3 while the run still reported a
clean finish: the migrate row said failed, but `error` was set only from
item failures, and `error` is what offers Retry.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Xg8vXUuHCH3aRkf91sfxjx
* fix: keep the migration retry reachable after a reload
The retry-only action needed two things the reload did not have. load()
read the destination's data tables into a local set and dropped them, so
configuredNames was empty and the branch could not fire; it now seeds
configuredNames from the call it already makes.
And the branch keyed on the row saying `failed`, which only holds while the
failure is still in memory. A reload rebuilds every row from scratch, so the
same situation reads as `unconfigured`. It now keys on the data table
existing while its tables do not, which is the same state either way.
Without both, a reloaded failure sent the user back into the wizard, which
refuses the name it created — no way to reach the migration that failed.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Xg8vXUuHCH3aRkf91sfxjx
* docs: state the constraint, not how the code got here
AGENTS.md: "Describe the code as it is, never its drafting history". Nine
comments across the wizard narrated what an earlier iteration did — "used
to remember", "The regression:", "would otherwise warn" — which says
nothing to a reader who never saw it. Each now states the durable reason
directly: what the code must hold to, and what breaks without it.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Xg8vXUuHCH3aRkf91sfxjx
* fix: page the presence probe, and keep a failed run retryable
probeImportedPaths called each list endpoint once. They paginate at 30 rows
by default, so it answered correctly for a small project and silently
under-reported a large one — every item past the first page went back
through a create call that rejects an existing path. It now pages at 100
until a short page, with a 100-page stop so an endpoint that never returns
one cannot loop.
And a run that finished with failures offered only Finish. `done` is what
the step reads as terminal, not `error`, so a failed migration left no way
to run the SQL again. Retry now sits beside Finish whenever the run reports
an error — beside rather than instead, so a migration that fails every time
cannot trap the user short of step 4.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Xg8vXUuHCH3aRkf91sfxjx
* fix: don't offer to discard a data table that was created
Closing the wizard after a failed appended step asked "Leave without adding
a data table?" and warned that what ran had left things behind. Every part
of that is false when the setup itself succeeded: the data table exists and
works, and only its migrations did not run.
hasUnfinishedIntent() now asks only whether the setup succeeded. The import
step is the only caller that passes onFinishAlso, and it shows that failure
on its own row with a way to run it again, and will not let Finish through
while it stands — so closing loses nothing.
The in-dialog "Try again" is unchanged; it is still the direct retry.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Xg8vXUuHCH3aRkf91sfxjx
* fix: address round 8 — trigger kinds, migration retry target, unknown state
Four findings, two of them real defects in this branch's own work.
The presence key flattened every trigger kind into `trigger`. Each kind is a
separate table keyed on (path, workspace_id), so a workspace can hold a
schedule and an HTTP trigger both called f/cal/sync; whichever existed
answered for the other and the second was reported "already there" without
being imported. The key now carries the kind, which both sides already had.
projectInstall's own doc makes this argument for the five top-level kinds —
it just stopped one level short.
The wizard's in-dialog "Try again" ran runMigrationsFor(wizardFor), but
afterWizard() clears wizardFor as soon as the failed run reports, while the
dialog stays up. It resolved against no row and the step was marked done
over SQL that never ran. The target is now held separately, and an unknown
name throws rather than resolving — a resolved promise is what the appended
step reads as success.
settle() resolved "cannot tell" to done exactly on the reload it was written
for. A data table whose database is unreachable read as Configured and the
step said "You're all set" over a project whose apps fail on open. There is
now an `unknown` state that says so and still counts as outstanding. It also
asked for one full schema per migration; migrations for one data table all
target the same schema, so probeMigrationsApplied reads it once.
run()'s doc still described the pre-probe retry.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Xg8vXUuHCH3aRkf91sfxjx
* fix: address round 9 — abandon during the probe, and copy that outlived it
Abandoning while probeImportedPaths was in flight returned without settling
anything. `import` goes running before the probe is asked, so the checklist
kept a spinner on a run that had stopped, beside an enabled Retry and with
no explanation. The settling the post-installProject path already did is now
a helper both paths call.
Three pieces of copy still described the behaviour this branch replaced:
the resource alert said an existing path is "reported as failed" when the
probe now leaves it alone and reports it as already there; and the step-4
footer and skip confirmation both told the user to set up a data table that
the new `unknown` state means they already set up — only its schema could
not be read. Those two now branch, so the strong warning stays strong for a
data table that genuinely does not exist.
The presence-key doc named `trigger:http_trigger`; WorkspaceTriggerKind has
no such value. It is `http`, in the comment and in the two test mocks.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Xg8vXUuHCH3aRkf91sfxjx
* fix: never rerun SQL whose applied state is unknown
`unknown` covers two different unknowns, and this treated them as one. The
schema could not be read, or the SQL names no table `expectedTables` can
resolve — and the second is arbitrary published SQL, which may carry a
non-idempotent INSERT or ALTER. The row offered "Run migrations" and the
footer claimed rerunning was safe; both were claims this code cannot make.
An unknown row now offers "Check again", which re-reads and executes
nothing. That settles the case which actually recovers — a database briefly
unreachable — and leaves Skip, which states the uncertainty, as the way past
one that does not.
The partitions behind the copy also missed `failed` rows entirely: the
footer rendered a title with no body, and Skip described them as unreadable.
Both now group by what it costs the project — tables that are missing
(never created, or a migration that failed) against tables that could not be
verified — which is also what makes the sentences true: a failed row is
configured, so "this data table does not exist yet" was wrong about it.
Skip and the footer now read the same partition instead of each computing
one, so they cannot disagree again.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Xg8vXUuHCH3aRkf91sfxjx
* docs: the SQL disclosure should not promise a run that cannot happen
An unknown row's only action re-reads the schema; nothing executes its SQL.
The summary still said "Show the SQL this will run", which is the sentence
the previous commit removed from the footer for the same reason. On those
rows it now says what the SQL is.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Xg8vXUuHCH3aRkf91sfxjx
* fix: a row whose migrations are running does not offer "Set up"
Found by walking every branch on row.status rather than the ones I
remembered: `running` falls through to the catch-all action, which labelled
itself "Set up" in accent. Disabled, so nothing could come of it, but it is
the same label-outruns-state mistake the last rounds were spent on.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Xg8vXUuHCH3aRkf91sfxjx
* fix: never fill a path a resource of another type already holds
The presence probe matches on path, and a path says nothing about type. A
workspace resource of another kind sitting where the project wanted one of
ours was skipped as "already there", then read for missing fields against
the *project's* expected schema — so it looked like an empty stub, offered
Connect, and had its value replaced with credentials for a different
provider while keeping its own type. A working resource unrelated to the
import, destroyed.
Guarded at both ends. AppConnectInner checks the occupant's type before
updating, because `fillPath` only says "write into this path" and a caller
cannot be trusted to have checked. And the setup step records the conflict,
so the row explains that the project did not get the resource it shipped and
offers no action at all — every action there writes to that path.
Such a row is always listed, however full the occupant's value looks: it is
the only thing that tells the user something is missing.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Xg8vXUuHCH3aRkf91sfxjx
* fix: fail closed everywhere the import cannot tell
Codex was right that the occupant-type guard failed open: a getResource
that threw became `undefined`, which passed the mismatch test and left
filling enabled — so a transient read failure still overwrote the resource
the guard exists to protect. Only a read that succeeds and answers with
exactly this type now permits the write; a failed read, a missing type and
any other type all refuse.
That was the same "cannot tell, so proceed" this branch already fixed once
in settle(), so the rest of the wizard was swept for it. Two more:
findBlankResources dropped a row whenever getResource threw, on the
assumption that meant absent. Only a 404 means absent — and that failure the
import already reported. Any other error is a read that did not complete,
which says nothing about whether the credential needs filling; dropping the
row reports "all set" over one nobody filled. The row now stays and offers
no action, since none of them can be safe about a path this cannot read.
A resource type whose schema would not load left `required` empty, which
reads as "nothing missing" — so a half-filled resource passed as done. It
stays on the checklist; it just cannot name which fields are short.
The other four catches were checked and are already closed in the right
direction: probeWorkspace reports absent so the caller creates rather than
adopts, probeMigrationsApplied answers undefined which settles to a
non-actionable row, and afterWizard keeps whatever the run last said.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Xg8vXUuHCH3aRkf91sfxjx
* fix: a refresh takes every field the fresh read decides
`refreshBlanks` merged only `missing` out of the new scan, so the two fields
added alongside it were left at whatever the row said before. Both reviewers
found the same seam from opposite ends: a resource that had just become
unreadable kept its old readable-looking row, and one that had come back
stayed blocked until a reload.
These fields describe what is at the path now, so the fresh read owns all of
them — and the branch that marks a row done clears them, because a row that
has left the blank list was read and is filled.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Xg8vXUuHCH3aRkf91sfxjx
* fix: resolve the username in the destination, and lock a targeted name
Two P1s, both from the same earlier fix being half-done. Routing this
wizard's API calls through an explicit workspace left `$userStore` behind,
and that store describes the workspace the app is in. After a reload on
step 4 it names the workspace the user came from, so a resource path built
from it lands on `u/<someone-else>` inside the destination — failing an
ownership check, or for an admin, quietly putting database credentials in
another member's namespace. The membership is now resolved for the target
workspace, the way FolderPicker already did it.
And `initialName` was documented as "a starting point, not a lock" while
`onFinishAlso` targets that exact name. Renaming `main` to `other` created
`other`, ran the migrations against `main`, failed, and left a data table
nobody asked for. The field is locked when a caller passes follow-up work
bound to the name, and says why; without one it stays editable.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Xg8vXUuHCH3aRkf91sfxjx
* fix: resolve membership before seeding, and hold the name lock for the dialog
Two follow-ons to the previous commit, both where a value is read live that
should have been settled once.
The username was fetched in an effect while `open()` reset the wizard
immediately, so `defaultFolder()` ran against an empty username and seeded
`u/admin`. It was corrected only if `whoami` happened to win a race against
the folder list, and never if `whoami` failed — which is the case that
matters, since an admin would then save database credentials in another
member's namespace. `open()` now awaits the membership before reset, and a
destination whose membership cannot be read blocks setup outright rather
than guessing a path.
And the name lock read the live `initialName`, which is the caller's
`wizardFor` — cleared from `onDone`, which fires after a *failed* run too,
while the dialog stays up offering Back. The lock released exactly when the
user was most likely to edit the name, so the rename-then-retry path still
diverged from the migration target. It is captured at reset, for the life of
the dialog.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Xg8vXUuHCH3aRkf91sfxjx
* fix: do not show the data table dialog before it knows the destination
`open()` became async so it could resolve the destination's membership
before seeding a resource path from it. But `openWizard` still set
`wizardOpen` first, and that is bound to the dialog's `opened` — so the
dialog was mounted, visible and clickable for the whole lookup, with the
username unresolved and `membershipFailed` not yet set. Setup reached in
that window writes exactly the wrong-namespace path the await was added to
prevent, and a late response could reset a dialog the user had already
touched or closed.
`open()` sets `opened` itself, once it has an answer. `wizardFor` alone
mounts the component, which is all `wizard?.open()` needs to exist.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Xg8vXUuHCH3aRkf91sfxjx
* fix: honour the configured base path, and guard a second Set up click
Windmill can be served under a prefix (`paths.base`, from VITE_BASE_URL),
and four import entrypoints compared or emitted `/projects/import` without
it. Under a base of `/windmill` the real pathname is
`/windmill/projects/import`, so the layout's picker exemption and both login
redirect checks stopped matching and sent people through the workspace
picker — and the compatibility redirect emitted a path outside the base
entirely, which is a 404. All four are now built from `base`.
`Login.svelte` takes it from `$lib/base` rather than `$app/paths` because it
already did; both read VITE_BASE_URL, and importing the second name into
that file collides with the first.
And the previous commit left Set up clickable while `open()` resolves the
destination membership, deliberately — but with no guard, a second click
starts a second lookup whose `reset()` lands on the dialog the first one
opened, wiping fields already filled. The action is disabled while a dialog
is opening.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Xg8vXUuHCH3aRkf91sfxjx
---------
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
Co-authored-by: windmill-internal-app[bot] <windmill-internal-app[bot]@users.noreply.github.com>
* feat(frontend): guided setup wizard for data tables
On Cloud a data table cannot use the Windmill instance database, so a new
workspace hit a dead end: an alert telling the user to go find a PostgreSQL
resource somewhere else. Setting one up meant three disconnected places, and the
connection could only be tested after the config had already been saved.
Adds a three-step wizard (choose a database -> set it up -> name it) reached from
the data tables settings page:
- Supabase: signs in via the existing supabase_wizard OAuth client and creates
the project from inside Windmill. Because db_pass is an input to project
creation, Windmill sets the password and the user never visits a dashboard.
- Your own database: picks an existing postgresql resource, or adds one with a
connection string through the form that already supports it.
- Windmill database: hands back to the inline row editor, since instance
databases are provisioned by a superadmin.
Verifying access is no longer a step the user takes: Continue runs the check and
passing it is what advances the wizard, so a database that cannot create tables
never reaches the workspace config.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* chore: pin ee-repo-ref to the Supabase provisioning endpoints
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* fix(frontend): do not claim the database is ready when its check failed
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* fix: address review findings on the data table wizard
- The Supabase create branch advanced on `provisioning === 4` without consulting
the check it had just run, so a role that cannot create tables could reach
Finish. It now blocks and offers Try again.
- Retrying no longer mints a fresh secret variable + resource each time: the
credentials are only re-created when the password actually changed.
- The generated password is captured before the create call rather than after,
since a throw there can still leave a project behind.
- On a failed provision the project list is refreshed, so the just-created
project can be picked up from the other tab instead of provisioning a second.
- Finish refuses a name that already belongs to another data table, which
previously repointed it at the new database.
- Secrets go to the acting user's namespace instead of a literal `u/admin/`.
- The progress list no longer ticks "Created on Supabase" before the request is
sent, and does not claim the database is ready when its check failed.
- The wizard's resume state is cleared when it closes, so reopening after an
abandoned OAuth round trip is not stuck on step 2.
- The OAuth callback shares the session-storage key rather than repeating it.
- SupabaseConnect uses the shared provisioning helpers instead of a fork.
- Restores the doc comment displaced onto TestDataTableResourceQuery.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* fix(frontend): simplify Alert layout and balance its vertical padding
The body was rendered by two near-duplicate branches, each wrapping the text in an
extra div only to hang a margin on it, and the margins disagreed: the collapsible
branch spaced above with mt-2, the static one below with mb-2. Since isCollapsed
defaults to true, every non-collapsible alert took the static branch, so titled
alerts read as 24px of space below the text against 16px above -- visibly
off-centre -- with the title and body flush against each other.
Collapse both branches into one and drop the margins; the container's own padding
now sets top and bottom equally, with a small gap under the title row.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* fix(frontend): only offer Supabase when its OAuth client is configured
The wizard offered the Supabase card unconditionally, so on an instance whose
superadmin never configured a supabase_wizard client -- or whose backend is built
without the oauth2 feature, which compiles the whole /api/oauth router out -- the
card dead-ended at a 404. Gate it on listOauthConnects, the same check
ApiConnectForm already makes, fetched on open so configuring the client mid-session
does not require a reload.
Also drop the Supabase project ref from the existing-project cards: it is an opaque
identifier that means nothing outside Supabase's own dashboard URLs. Show the region
instead, plus a status word when the project is not healthy, since a paused project
is the one case where the connection check fails for a reason unrelated to the
password.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* fix(frontend): run the Supabase OAuth leg in a popup
A full-page redirect unmounts the wizard, so anything the user does on Supabase's
side -- signing in, confirming an email, browsing their dashboard -- leaves them
with nothing pointing back at Windmill, and the wizard had to park its state in
sessionStorage to survive the trip.
Open the connect endpoint in a popup instead. The modal stays on screen throughout
and the callback hands the token back through postMessage rather than navigating.
The parked-state path stays as the fallback for browsers that block the popup.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* fix(frontend): scope the connection check to the choice that produced it
A failed check stayed on screen when the user switched Supabase mode or picked a
different provider, so a fresh tab opened showing an error about a database it had
nothing to do with. Clear the report and the error on both switches; re-clicking the
tab already selected leaves an error the user is reading in place.
Also polish the Supabase step: project cards get the provider-card treatment (icon,
p-3, flex column) instead of a hand-rolled variant whose block layout left more
padding above the name than below; form labels settle on text-emphasis; and the
signup link sits under the primary button for anyone who does not have an account
yet.
Drop the "free" badge and the "Free on Supabase" line -- every option in the wizard
is free, so neither told the user anything -- and say what the Supabase card
actually does now that connecting an existing project is the default.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* refactor(frontend): one setup checklist and one Supabase step for every host
The data table wizard, the instance database modal and the resource drawer each had
their own version of the same two interactions, and they had already begun to drift:
the wizard's Supabase resource shape was rebuilt by hand in the drawer, and the
instance checks rendered with no notion of a step being in flight.
SetupChecklist replaces LoggedWizardResult, whose only consumer was the instance
modal. It adds the running state that component lacked, so a list driven by an
endpoint that reports nothing until it returns still shows where it is. Both the
instance checks and the Supabase provisioning stages render through it.
SupabaseProjectStep owns picking or creating a project, and useSupabaseOauth owns
the popup leg. Each host keeps only what is genuinely its own: the wizard saves a
variable and resource then verifies the connection, the resource drawer fills in its
own form. Both trigger authorization themselves, so a host can offer it a screen
earlier than the step does.
The lists load behind a spinner because which mode to open on depends on whether the
account has projects; deciding that after rendering flipped the toggle under the user.
Adds a kitchen_sink playground for the checklist so the animation and every failure
position can be exercised without a backend, a superadmin, or a Supabase account.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* fix(frontend): tidy the resource drawer around the Supabase entry point
Connect Supabase was a hand-styled anchor carrying Supabase's brand hex values
rather than a Button, and it sat in a row whose other controls had settled on
unifiedSize md. Making it a Button meant SupabaseIcon had to satisfy IconType, so it
now takes `size` (deriving height/width from it) alongside the string props its other
callers pass.
The manual resource form spaced every field 32px apart and WhitelistIp added another
16px of its own, which read as a gap rather than a rhythm. One gap of 16px, with the
form itself given a little more separation from the description above it.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* fix(frontend): stop Supabase resources coming up modified when first opened
Resource forms fill in every unset property from the schema as soon as they render,
so a postgresql resource saved without region, root_certificate_pem and use_iam_auth
was dirty -- and had saved a draft -- the first time anyone looked at it. Write them
with the rest of the value.
SupabaseConnect also rebuilt the resource shape by hand instead of using the shared
helper, which is how the pooler host format ended up in two places.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* feat(backend): record where a data table came from and whether setup finished
edit_datatable_config replaces the whole datatables map and DataTable does not deny
unknown fields, so anything the request omits is dropped without a word. origin and
setup_incomplete would have been erased by any unrelated save;
preserve_unmanaged_datatable_fields carries them -- and migrations_enabled, which had
the same problem inline -- forward for entries that already exist, following renames.
setup_incomplete is what lets a row be recorded before the resource it points at
exists, so the wizard can write nothing until the user finishes. There is deliberately
no intermediate state: the setup runs entirely in the browser, so nothing server-side
could advance one.
datatable_health probes every data table at once for the settings page and skips the
incomplete ones, whose resource_path resolves to nothing yet. set_datatable_setup
patches a single entry instead of resending the map. test_datatable_connection_value
checks a connection the caller has not saved anywhere, which the wizard needs before
it has written a resource.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* fix(frontend): make destructive default and subtle buttons read red
Both variants were neutral until the pointer arrived, then filled solid red: nothing
marked the button as destructive until you were already on it. They now carry red text
at rest, with a faded red border on default and a light red wash on hover, which is
what the legacy red border style in the same file had always done.
Three call sites passed color="red" alongside a design-system variant. getStyleClass
returns before colour is read for accent, accent-secondary, default and subtle, so the
delete-migration control, its modal confirm and the import-database button had all been
rendering neutral. They pass destructive now.
The dropdown variant strips the button's own border, and matched border-border-light
literally -- a class the destructive style no longer contains.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* feat(frontend): rebuild data table setup around a read-only row
The wizard gathers intent over two steps, reviews it on a third and writes nothing
until Finish, so a billable Supabase project is created only once the user has seen
what will happen. runSetup is also the retry: every step probes for its own result
before doing anything, so running it again on a half-finished data table resumes
instead of duplicating. Its steps are keyed rather than dispatched on their titles,
where rewording one changed what it did.
The settings row stops being an editable form with a dirty/save cycle. It carries the
name, where the database came from, a health dot and two actions; everything rare
moved into the gear panel, which also offers Finish setup for a data table whose
wizard never completed. Manage is ExploreAssetButton, the control the ducklake list
already uses, and the row and panel both link out to the underlying resource.
supabaseResourceValue no longer assembles the pooler host from the region.
aws-0-<region>.pooler.supabase.com is wrong for any project Supabase allocated
elsewhere, so the host, user and port come from the pooler config endpoint.
Two data tables sharing one database also share _wm_migrations, which is probed
unqualified, so the review step warns when the database being connected is already
behind another data table.
SupabaseConnect is deleted. The resource drawer uses the shared project step
restricted to existing projects: creating one is a billed action and belongs in the
wizard, which has somewhere to report what it did. The kitchen_sink checklist
playground goes with it.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* fix(frontend): fall back to a direct Supabase connection when the pooler cannot be read
Reading a project's Supavisor config needs the database_pooling_config_read scope, which
an instance's Supabase OAuth app may never have been granted. No retry recovers from
that, and the wizard treated it as fatal: the user was left with an error and no way to
finish connecting a project that was otherwise fine.
resolveSupabaseConnection replaces the bare pooler read everywhere it happened. Asking
for session pooling and failing now yields a direct connection plus the reason, which
supabaseResourceValue already knew how to write. Nothing about the fallback is silent --
direct is IPv6-only, which is the whole reason session pooling is the default -- so the
wizard warns on its review step and the resource drawer says so in its toast.
The row is recorded before credentials are saved, so an origin claiming session pooling
has to be corrected once a direct host is what gets written; the run patches it through
set_datatable_setup rather than leaving the panel to report a mode nothing uses.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* feat(frontend): open the database behind a data table, and say when it cannot write
Every database in the list now opens the surface that owns its credentials. A postgres
one opens its resource in the editor drawer; a Windmill instance one opens the instance
modal, which is where its setup checks, password rotation and drop already lived. Both
are reachable from the row and from the panel's provenance list, and the provider icon
moved inside the button so the whole thing is one target.
CustomInstanceDbWizardModal targeted #content unconditionally, which put it underneath
the panel drawer that now opens it. It takes a target, and the panel portals it to the
body.
The status column gains a third state. The probe reports privileges but nothing gated
the dot on them, so a data table whose role cannot create tables showed as Connected and
only failed when someone ran a migration. It reads "Limited permissions" instead, and
opens the panel on the report carrying the GRANTs that fix it -- the settings page has
already probed, so the panel takes that report rather than asking the user to run Test
connection over work already done. fullyPrivileged is exported from the report component
so the dot and the report cannot disagree about what counts as healthy.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* revert(frontend): keep the data tables settings table as it was
The settings table and the setup wizard are two changes that only shared a file. Splitting
them makes each reviewable: this branch keeps the wizard, and the read-only row, gear
panel, health probe and clickable databases move to their own branch.
The rows go back to the editable form with its pickers and save footer, still opening the
wizard from Add a database. DataTableSettingsPanel, dataTableHealth and dataTableOrigin
had no other consumers and go with them; the connection report stays, because the wizard
shows it too.
DataTableSettingsType keeps `origin`: the wizard writes it, and the review step reads it
back to warn when two data tables would share one database and therefore one
_wm_migrations table.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* fix(frontend): confirm before dismissing the data table wizard mid-setup
Closing was guarded while a run was in flight and unguarded before one, which is backwards:
a run leaves a row to resume from, whereas a backdrop click on the review step threw away
the project, the pasted password and the folder with nothing to recover them from.
Backdrop, Escape and the close button now go through one path that asks first. It only asks
when there is something to lose -- no provider chosen yet, or a run that already produced a
result, closes immediately -- so the dialog does not become something to click through.
Continue in the background still leaves in one click; that exit was always the deliberate
one.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* fix(frontend): stop the wizard claiming the resource folder controls who can use a data table
"Who can use this database" was wrong. Every path that resolves a datatable:// reference --
both executors and the agent-worker endpoint -- reads the resource unchecked, by workspace
and name. A resource in u/admin is usable by everyone's scripts. The folder governs who can
see and edit the connection, and who can reference the resource directly in a SQL step;
neither is who can use the data table. The wizard was contradicting the tab's own
description two screens later.
The folder select and name field become one Path picker, the same one the resource,
variable and script forms use, so the review step reads as a resource path rather than a
permission choice. Its initialPath is snapshotted when the step opens: Path seeds itself
from it, and a live value fights the typing. Finish now also gates on Path's error, so a
taken or malformed path stops the run before it writes anything.
The button that opens all this says "Add a data table" -- the data table is what you get;
the database is a detail chosen along the way.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* revert(frontend): move the destructive button restyle out of the wizard PR
This reverts 3881e4d8ea. Making default and subtle destructive buttons red at rest changes
every existing caller of the prop -- the workspace integrations, AI skills, workspace
creation and the instance database drop -- so it is a design-system change, and the call
sites it fixed are the migrations list and the database manager. None of that is the setup
wizard.
Nothing on this branch passes destructive any more, so it leaves with no loose ends.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* fix(frontend): make the wizard stepper navigate the steps it already offers
Stepper dispatches a click and paints cursor-pointer on every reached step, but the wizard
never listened, so the breadcrumbs invited a click and did nothing.
They now reach any step already passed, in either direction: going back to check something
should not cost the progress, which means tracking the furthest step reached rather than
the current one. Forward movement still only happens through the primary action, so a step
is never reachable without having been validated -- and changing the intent revokes the
steps ahead of it, or Finish could run against a review built from something the user has
since edited. The five places that cleared the probe on an edit now do both through one
call.
During a run nothing is reachable, and the stepper says so rather than showing a pointer
over steps that will not respond.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* fix(frontend): restore the data tables description lost in the branch split
The rewritten description went into DataTableSettings.svelte shortly before that file was
restored wholesale to its pre-rebuild state, so it left with the row rework it had nothing
to do with. The tab went back to describing the plumbing -- a fully managed PostgreSQL
database, reachable from the SDK -- which never answered the question a new user actually
has: why this rather than a Postgres resource.
It leads with what a data table is, then the two things a resource cannot do -- nobody
needs the credentials to query it, and the name can be pointed at another database without
editing anything that uses it -- and closes with what Windmill runs on top. Both middle
claims are the ones every resolution path backs up: datatable:// resolves by workspace and
name, unchecked.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* fix(backend): say what is missing when a $res: or $var: reference does not resolve
Both interpolations fetched with fetch_one and mapped the error through to_anyhow, so a
reference to something deleted surfaced as "no rows returned by a query that expected to
return at least one row @workspaces.rs:2169". It names neither the kind of thing that was
missing nor its path, and it is what a data table pointing at a deleted resource reports.
They now fetch_optional and return NotFound naming the path, and datatable resolution adds
the data table on the way out: the caller asked for one by name, and a bare "resource
f/x/y does not exist" leaves them to work out which of them points at it. The health probe
is new, so this string had only just become something users read.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* feat(frontend): gate the data table wizard behind a dev flag
The wizard only appears with `dataTableWizard` set in localStorage; without it the
settings page keeps the inline-row flow it had before this branch, down to the empty-state
copy and the "New Data Table" button, and the wizard component is not mounted at all. The
existing e2e suite drives that button, so the default-off flag is also what keeps it green.
Step 2 of "your own database" becomes one list rather than a segmented control: the
workspace's Postgres resources, then a New resource card that expands in place. A
connection string is not an alternative to a resource, it is how one is written, and the
old layout taught otherwise. The card holds the same connection as a string or as fields
and carries values across when you switch, so `parse` and `compose` have to be inverses --
hence the percent-encoding on both sides, which also fixes a password containing `@`
silently corrupting in the resource form. The Supabase step now uses the same shape.
Names and paths are checked as they are typed rather than at the end of a run that may
have created a billed project first: the data table name against the charset
`edit_datatable_config` enforces, the instance database name against what
`setup_custom_instance_db` will accept, and the resource path against both the resource
and variable namespaces, since the run writes to both and both writes upsert.
`test_datatable_connection_value` refuses `$var:`/`$res:` in its body. It feeds
`transform_json_value_unchecked`, which resolves references with no permission check of its
own, so an admin could otherwise have had the API server decrypt any workspace secret and
hand it to a host the same request chose -- without the audit trail a variable read leaves.
Callers testing something unsaved hold the literal value already.
Alert, SetupChecklist and postgresConnectionString change for everyone, not just behind the
flag: body-only alerts no longer reserve an empty title row, the checklist can nest the
checks a step is made of, and the connection-string parser is shared with the resource form.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* chore: pin ee-repo-ref to the EE branch merged with EE main
The Supabase proxies the wizard calls are still unmerged, so the ref cannot be an EE
main commit yet; it now names that branch merged with EE main rather than the branch
alone, which was nine commits behind and would have been built against a CE main it
never saw.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* feat(frontend): gate the supabase resource path behind the dev flag
* test(frontend): pin connection string parsing to libpq behaviour
* fix(frontend): keep the supabase resource link off the popup callback path
* refactor(frontend): load the supabase resource dialog only behind the flag
* fix(frontend): refuse a resource path the wizard run does not own
* fix(frontend): let a failed data table setup be corrected without losing what it made
* fix(frontend): let a failed setup reuse the resource path it claimed
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* fix(backend): record the two data table connection tests in the audit log
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* fix(frontend): use Section for the data table wizard advanced group
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* fix(frontend): read connection strings the way libpq does
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* fix(backend): pin the ee ref back to a commit this branch can build
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* fix(frontend): keep a failed setup's claims across the redirect and rollback
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* fix(backend): probe a data table with the auth mode the worker will use
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* fix(frontend): keep every part of a connection string through the round trip
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* refactor(frontend): give a setup run one record of what it created
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* fix(frontend): mark a resource claim by edited_at, not its creator
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* fix(frontend): mark every claim by revision, and keep an unconfirmed project's secret
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* fix(frontend): refuse to test or save behind a connection string that will not parse
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* fix(frontend): refuse a connection string carrying options the resource cannot hold
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* fix(frontend): allowlist the connection-string parameters a resource can honour
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* fix(frontend): guard every created Supabase project, not just the last one
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* fix(frontend): do not warn about renaming an item that does not exist yet
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* fix(frontend): make the review step read as one list of what will exist
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* fix(frontend): keep the picked Supabase project across the redirect, reject connect_timeout
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* fix: check the data table connection from a worker, not the API server
The wizard's connection check ran on the API server through two endpoints added
for it. That server is a different machine with a different identity, so the
answer was about the API server rather than about the worker that will run the
queries: a host reachable from one is not necessarily reachable from the other,
and IAM RDS and Azure workload identity authenticate as whichever process opens
the connection.
Run the privilege query as a preview job instead. A job goes through the
worker's Postgres executor, which is where `PgAuthMode::of` already picks the
authentication mode, and it takes either a resource value or a `$res:` path
exactly as a Postgres step does. Postgres composes the suggested GRANT
statements through `format('%I')`, so identifier quoting stays where it is
already implemented.
Removes `test_datatable_resource_connection` and
`test_datatable_connection_value`, and `connect_as_the_worker_would` with them.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* refactor: fold check_datatable_connection back into its only caller
The helper was split out so the two connection-test endpoints could share a
body. Those endpoints are gone, leaving one caller.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* revert: keep the data table connection check schema inline
It was lifted into components so three endpoints could share it. Two of those
are gone, so it is back to one user and the extraction changes nothing.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* fix: restore openapi.yaml to the branch point
The previous commit restored main's tip rather than the merge base, which
carried three unrelated main-only changes into this branch: the resource
mcp_tools truncation fields, the execution_mode description, and a version bump.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* refactor(frontend): drop four effects from the data table wizard
Each was doing work a derived, a load callback or a real entry point does
better.
- The name conflict is kept with the name it was raised for and derived from
it. As an effect it was correct only because it never read what it wrote:
the pre-flight sets the message and the effect does not re-trigger, so adding
a read would have cleared it the instant it appeared. The message now also
comes back if the taken name is retyped, which is what the server will say.
- The default resource selection is seeded inside the fetcher that loads the
list, where "has the fetch settled" cannot be asked wrong.
- Reset-on-open becomes an exported open(), called by the settings page, so a
fresh run is set up by the act of opening rather than by a flag emulating
mount.
- The OAuth connects and the folder list become resources; supabaseAvailable
and folders are derived from them. defaultFolder takes the list rather than
reading it, so the fetch can seed off its own result.
Leaves the debounced path check, which is async with an out-of-order guard.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* refactor(frontend): drop three effects from the Supabase branch
- useSupabaseOauth reports success as onAuthed, alongside the failures it
already reported. SupabaseResourceConnect was watching `authed` to find out;
it takes the callback instead, keeping the guard that stops an authorization
started elsewhere on the page from opening its dialog.
- SupabaseProjectStep loads its orgs and projects through a resource keyed on
the token, so the `loaded` latch goes and re-authorizing reloads rather than
keeping the lists from the expired session.
- SetupChecklist records what the user toggled and derives the open state from
it, a failed step defaulting to open. Recording the open state instead needed
an effect to force it, and that effect re-ran on every progress update, so a
description closed while anything was still ticking reopened. A close now
holds for the life of the checklist, including across Try again.
Leaves the message listener, which subscribes to another window.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* fix(frontend): confine the modal restyle to the wizard, and trim the comments
The wider side padding and lighter dialog heading were changing all 17 Modal2
dialogs to suit this one flow. They move behind an opt-in `formStyling`, taken
by the three dialogs this branch owns; every other Modal2 renders as it did.
Also drops two comments that cited a design approval rather than a constraint,
and shortens the blocks that had grown past the four lines AGENTS.md asks for.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* fix(frontend): use the accent token for the wizard's links
`text-blue-500` is the marketing blue `#3B82F6`, which brand-guidelines.md
rules out in the app interface.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* chore: point ee-repo-ref at the EE branch head
Picks up EE main, which the branch now needs, and the Supabase proxy auth fix.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* fix(frontend): read sslmode by name, and stop decrypting a secret to date it
- `sslmode` was found by searching the query text, so it also matched inside
another parameter's value: `?application_name=sslmode=disable` passed the
allowlist on the parameter name and then parsed as a request to turn TLS off,
which both the wizard and the resource form saved and probed. Parsed with
`URLSearchParams` by exact name, with a test.
- `secretMark` read the variable with `decryptSecret` defaulted to true, so
every write decrypted a secret nothing reads and recorded the decryption --
including someone else's on the retry about to refuse it. It wants only
`edited_at`, which is returned either way.
- The probe gave up at 15s while the worker allows its Postgres connect 20s, so
a host that accepts the connection and never answers was cancelled and
reported as a missing worker rather than a failed connection.
- The create-mode region and project name did not report an intent change, so
renaming a project after a name collision left the failure naming the old one.
- Two comments described the code as it was before the claim mark became a
revision, and a doc comment outlived the field it documented.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* fix(frontend): read connection parameters the way libpq does
One reader for both the parser and the allowlist, since they disagreed about
what a string says in two ways that both ended in a weaker connection than was
pasted:
- `URLSearchParams.get` takes the first of a repeated parameter and libpq takes
the last, so `?sslmode=disable&sslmode=require` was read as `disable`.
- The allowlist folded the parameter name and the parser did not, so
`?SslMode=verify-full` was refused by neither and honoured by neither, and
saved as the `require` default.
The parked Supabase run is now handed to `open()` rather than read back off the
`resume` prop it was just assigned to, so restoring it does not depend on when
that prop reaches the component.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* fix(frontend): keep connection parameter names case-sensitive
libpq does not fold them: `?SslMode=disable` is rejected as an invalid URI
query parameter rather than read as `sslmode`, which a local server confirms.
Folding made Windmill accept and honour a string Postgres itself refuses;
naming the parameter instead tells the user why it cannot be stored.
The last-value-wins rule for a repeated parameter is unchanged, and matches
what the same server does.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* fix(frontend): seed the Supabase organization from the project it selects
The loader took `orgs[0]` independently of the project it seeded, so an account
whose first project sits outside its first organization had the review step name
an organization the database does not belong to. Picking a project by hand
already derives it; the seeding now does the same.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* fix(frontend): let the probe report an empty search_path instead of failing on it
`format('%I', NULL)` raises rather than returning NULL, so a role whose
search_path names no valid schema failed the whole privilege query and was
reported as an unreachable database. That is the one case `fix_search_path`
exists to name, and it never reached the user. Verified against a local server
with `SET search_path = ''`.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* fix(frontend): say which of the two refusals a connection string hit
Making parameter names case-sensitive gave `unsupportedConnectionParam` two
reasons to refuse, and the single message explained only one. `?SslMode=` was
answered with "Windmill cannot store SslMode on a Postgres resource", which is
false twice over: sslmode is exactly what the resource stores, and the string
asks for nothing because Postgres rejects the URI. It now names the spelling
when the parameter is one we keep, and the storage limit otherwise.
The folder-list guard also still read the `resume` prop that `open(parked)` was
changed to stop trusting, so the resumed path now comes from whatever `reset`
was handed.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* fix(frontend): leave the Supabase organization unset when the lookup misses
Falling back to the first organization named one the seeded project is not in,
since `supabaseSummary` prefers `intent.org` over the project's own. Unset, it
falls through to the project's organization identifier — the right one, spelled
as a slug rather than a name.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* test(frontend): pin which refusal a connection string gets
The two messages differ in what they ask the user to do, and the condition
choosing between them — whether the lowercased name is one the resource keeps —
is not visible from either call site. `Connect_Timeout` is the case that keeps
them honest: miscased *and* unstorable, so respelling it would not help and the
message must not suggest it.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* fix(frontend): hand a failed Supabase leg back to the page holding its run
Denial, a token error and a malformed callback all sent the user to
/resources whether or not a run was parked. Nothing else consumes the park, so
the run stayed in sessionStorage and sprang the wizard open on an unrelated
later visit instead. A parked run now lands on the data tables tab, where the
wizard resumes on the setup step and can authorize again.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* fix(frontend): let a run reuse the name of a row it could not take back out
`removeRow` reports `kept` when the undo cannot reach the server, so the row
this run wrote stays in the workspace config and comes back in `existingNames`.
The client-side name check then refused the retry on the run's own name, with
no way forward but a rename. The instance database name has carried the same
exemption since it was written; this is the data table name catching up.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* fix(frontend): discard a variable check the wizard has moved on from
The post-await guard compared only the path, and the path is built from the
review step's fields -- so picking an existing resource stops the wizard minting
one without changing it. A check already in flight then answered for a branch
nobody was on, and a `true` disabled Finish over a path the run no longer
writes. The cleanup cannot help: it cancels a pending timer, not a live request.
Both sides of the await now ask the same question.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* chore: update ee-repo-ref to 483513b70979aa9497cab869837108d948449984
This commit updates the EE repository reference after PR #715 was merged in windmill-ee-private.
Previous ee-repo-ref: 8604b30a740c5620069208801a7ae50937b61977
New ee-repo-ref: 483513b70979aa9497cab869837108d948449984
Automated by sync-ee-ref workflow.
---------
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
Co-authored-by: windmill-internal-app[bot] <windmill-internal-app[bot]@users.noreply.github.com>
hugocasaandClaude Opus 5
