starred/windmill
1
0
Fork 0
mirror of https://github.com/windmill-labs/windmill.git synced 2026-10-29 16:15:51 +00:00
Code Issues Packages Projects Releases Wiki Activity
14,688 Commits 862 Branches 1,582 Tags
43d2c299884851213fb0f73f22b72d42ae9899a2
Commit Graph
190 Commits
Author SHA1 Message Date
Diego ImbertandClaude Opus 5 43d2c29988 fix(datatables): a rename reaches only the workspace it is renaming in 
`update_username_in_workpsace` locks one workspace's settings row but wrote
`usr` and `usr_to_group` across every workspace holding that name, so it waited
on principals another workspace's settings row guards — the deadlock the last
commit closed, one workspace over. Both writes are scoped now, and the rename
walks memberships in workspace order.

The deferred role drop holds the settings row while it runs, so its session
carries a statement timeout: the statements run on a database the workspace does
not control, and a lock held there would stall every save behind it.

`PgRoleInventory` is read by the enterprise planner alone, which a community
build compiles without.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01S5arH3G2Sa1Qqm32veJQ1n
 2026-09-04 10:09:48 +02:00
Diego Imbert 42271dcf7f Merge remote-tracking branch 'origin/main' into datatable-perms-3 
# Conflicts:
#	backend/ee-repo-ref.txt
 2026-09-04 09:32:27 +02:00
Diego ImbertandClaude Opus 5 f3e3281336 fix(datatables): a deferred role drop stops at the data table that holds those roles now 
Role names are generated from the data table's name, so one recreated under a
deleted table's name generates the same ones, and its save adopts them. The drop
planned for the deleted table would then take the live one's logins and grants.
It now runs under the settings row it deletes them for, and stands down when
that name is back.

The rename writes `usr` and `usr_to_group` after the settings row, like every
other path that frees or renames a principal.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01S5arH3G2Sa1Qqm32veJQ1n
 2026-09-04 08:47:14 +02:00
Diego ImbertandClaude Opus 5 1f872c5edb fix(datatables): a role only works where it was created, and only under its own name 
Postgres roles are cluster-wide, so two data tables that generate one name share
a login. The name's discriminator was four bytes, which is searchable rather
than merely unlucky: two data table names that collide were found by hand. It
is now sixteen, and a name that is already taken is refused unless this data
table owns the role (enterprise repo).

The database a data table's roles were created in is recorded with them and
checked where the connection is finally resolved. The guards that refuse a
resource edit read the config separately from the save that enables permissions,
and never expand a `$res:` or `$var:` that can repoint the resource just as
well — so they stay as early refusals while the check that holds is the one at
the point of use.

Dropping a deleted data table's roles now runs after the settings transaction
commits, planned before it: `DROP OWNED` discards their grants for good, and a
save that rolls back after that point would leave a data table naming logins
that no longer exist.

Leaving a workspace takes the tenant of the row it deletes rather than of the
cached identity, which a rename leaves stale, and group deletion takes the
settings row before the membership rows like every other path that frees a
principal.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01S5arH3G2Sa1Qqm32veJQ1n
 2026-09-03 23:59:04 +02:00
Diego ImbertandClaude Opus 5 23c1fc470f fix(datatables): one lock order for every path that frees a username, one badge 
Removing a member takes both the workspace settings row (to drop the tenants
naming them) and their `usr` row. Two of those paths running at once took them
in opposite orders and deadlocked; every path now takes the settings row first.

The role picker in the data table explorer is the design system's badge rather
than a hand-rolled button.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01S5arH3G2Sa1Qqm32veJQ1n
 2026-09-03 21:37:11 +02:00
Diego Imbert 2c55c3918c fix(datatables): every route that frees a username takes its tenants with it 
Three deletions bypassed the workspace-user cleanup and left `u/<username>` on
the roles it named: the superadmin's global delete, which frees the name in
every workspace at once, and both leave routes. A member invited into one of
those names afterwards inherited the roles.

The settings lock says what it hands back — the config as stored, generated
role passwords included — so it carries `_unchecked` and the contract that goes
with it: callers authorize the read, and redact before passing the value on.
 2026-09-03 19:43:39 +02:00
Diego Imbert 625753c2d4 fix(datatables): one lock for everything that reads the config and writes it back 
The role save took an advisory lock of its own while principal cleanup took the
settings row, so the two never excluded each other: a save could persist a
permissions block it had computed before a group's deletion took that group off
it. The settings form had the same shape with no lock at all — it carries the
old permissions forward by construction.

They all take the settings row now, before reading and until they have written,
which is the lock the cleanups already used. One mechanism, so there is no
ordering to get wrong, and the advisory lock goes away with its cached query.

The tenant removal says what it does not do: it authorizes nothing, and the
rules differ per caller — a workspace admin for a user, the owner for a group
or folder, no identity for the system paths — so the name carries `_unchecked`
the way the resolution helpers next to it do.
 2026-09-03 12:17:26 +02:00
hugocasaandClaude Opus 5 17ba521c35 fix: record supplied script lock hashes so importers can skip relocking (#10915) 
* fix: record supplied script lock hashes so importers can skip relocking

Creating a script with a caller-supplied lock — a CLI push, a git-sync deploy,
any create carrying a lockfile — stored the lock on `script` but never wrote the
matching `lock_hash(workspace_id, path, hash_script(lock))` row. Only
worker-generated locks did.

`try_skip_relock` treats a missing hash for an imported script as changed, so no
importer of such a script could ever satisfy the skip predicate: every deploy of
it relocked every importer, forever.

The create transaction now records the hash for any lock it accepts, including
the empty one a codebase or a language with no lock generation carries — the
worker writes `hash_script("")` there, and a path going from a real lock to an
empty one has to stop matching what its importers recorded. Only a lock left to
a dependency job is skipped, because that job writes it.

A workspace clone now carries `lock_hash` too, without which every
dependency-map snapshot the clone later recorded held NULL and nothing in it
could ever skip. `dependency_map.imported_lockfile_hash` is deliberately not
copied: it records what an importer resolved against when it was last locked,
the clone runs READ COMMITTED, and a relock landing in the source between the
scripts being cloned and that statement would attach a hash the cloned
importer's lock was never resolved against — a hash older than the cloned
scripts costs one relock, a newer one skips a relock that was needed.

Lock generation is untouched, as is everything a relock does once it runs. The
only behavior that moves is which relocks are skipped.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0138oct9a6SLEZvFyCQgHRBx

* fix: narrow to the create-path lock hash

Drop the workspace-clone copy of lock_hash. It sits outside the reported
bug, and its double join over `script` can emit a path twice where two
versions are live, which the unique key on (workspace_id, path) then
rejects, failing the whole fork.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0138oct9a6SLEZvFyCQgHRBx

* fix: restore the workspace-clone lock hash copy, guarded against fanout

A path can hold two live versions, and both joins match on path alone, so
the select can emit it four times against a primary key that admits one.
Every such row carries the single hash the path has, so ON CONFLICT DO
NOTHING settles it rather than aborting the fork.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0138oct9a6SLEZvFyCQgHRBx

* fix: hash a clone's own locks rather than copying the source's rows

A source row is only as current as the last write to it, and a supplied
lock deployed before this was recorded leaves one naming a lock the path
no longer holds. Copying that into a fork hands an importer a hash it
never resolved against; hashing what the clone holds cannot.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0138oct9a6SLEZvFyCQgHRBx

* test: pin the lock hash written on a no-op push

Removing that write leaves the assertion with no row, which is the state
a script deployed before this shipped would stay in.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0138oct9a6SLEZvFyCQgHRBx

* refactor: share one lock hash writer between the create and clone paths

Both wrote the same upsert with different SQL. The existing writers fold
theirs into the statement that writes the lock itself, which is what keeps
the two consistent; these two have nothing to fold it into, so they take a
shared one instead. The clone walks its pages by path rather than listing
them first, dropping a query with it.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0138oct9a6SLEZvFyCQgHRBx

* fix: stream a clone's locks rather than reading them in pages

script.lock is unbounded, so a page of them is bounded only by how many
it holds. Hashing each as it arrives keeps one in memory at a time and
lets the clone site collapse to a single call.

Also states on both writers that they check no access to the workspace
they write, which their callers are the ones to have established.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0138oct9a6SLEZvFyCQgHRBx

* fix: make the lock hash writer safe to repeat and free when unchanged

A path given twice in one call would have Postgres reject the whole
statement, so the last hash for each wins. And recording a hash a path
already has cut a row version for nothing on every unchanged sync, which
is the mode the no-op push runs in.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0138oct9a6SLEZvFyCQgHRBx

---------

Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
 2026-09-02 19:20:49 +02:00
Diego Imbert 0fc9ba39a7 fix(datatables): a tenant and a resource path follow the principal they name 
A data table role names its tenants — `u/alice`, `g/devs`, `f/team` — and a
data table names its database by resource path. Both are strings that outlive
what they point at, and the flows that free those names left them behind:

- Deleting a workspace user, a group or a folder left its tenant on every role
  it could run as. The name is free afterwards, so whoever takes it next
  inherits the role without an admin granting it. All three now drop it in the
  same transaction that removes the principal.
- Offboarding reassigns a user's resources to a folder without the data table
  following, which both stops it resolving and frees the path it named — a new
  resource there points it at another database with its roles intact. The
  username rename had the same gap. Both move the config with the resource.

The walk both need is one function each now, with the table test that pins
which tenant forms it touches.

Saving permissions also takes a lock per data table, held from the read that
plans to the write that persists: two saves interleaved each planned against
what the other was leaving, and the one that persisted last stored roles the
other had already dropped. The ACL apply takes the same lock, since it reads
the same config and catalog.
 2026-09-02 10:27:28 +02:00
Diego Imbert c289a40e5b fix(datatables): keep role passwords out of the settings every member reads 
`get_public_settings` is the one the logged-in layout calls for every user, and
it served the data table config as stored — generated role logins included. A
member who is a tenant of no role could read every role's password and connect
as it directly, which is the whole tenant model. It goes through the same
redaction as the admin settings and the tarball now.

Also, on the way there:

- The permissions drawer opened from a data table's row sent the role the
  manager is connected as, which belongs to another data table: roles are
  per data table, so a row under another one is read as its own default role.
- The grant repair on an instance database ran before `apply_datatable_acl`
  authorized anything, so any member could drive that privileged connection
  with a request about to be refused.
- That repair also assumed schema `public` exists, and failed whole where it
  had been dropped.
 2026-09-01 23:52:05 +02:00
Ruben FiszelClaude Opus 5windmill-internal-app[bot]
 		94af8d0fb5 fix: let a principal without a login account own a draft (#10925) 
* fix: let a principal without a login account own a draft

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_018Lu3hExEDPZu2dAEhZDVAi

* fix: keep an accountless draft owner from colliding or reading as legacy

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_018Lu3hExEDPZu2dAEhZDVAi

* fix: drop the unnameable draft owner everywhere and guard the no-op rename

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_018Lu3hExEDPZu2dAEhZDVAi

* fix: drop the unused Acquire import in the draft rename test

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_018Lu3hExEDPZu2dAEhZDVAi

* docs: drop the stale draft_users claim from the fork-clone rationale

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_018Lu3hExEDPZu2dAEhZDVAi

* chore: update ee-repo-ref to f5b783d2f7608e1ff3a817caa8b719e06f8b8981

This commit updates the EE repository reference after PR #768 was merged in windmill-ee-private.

Previous ee-repo-ref: f3dba016e9274ee9bbe46b4f070d3ed29843e5fd

New ee-repo-ref: f5b783d2f7608e1ff3a817caa8b719e06f8b8981

Automated by sync-ee-ref workflow.

---------

Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
Co-authored-by: windmill-internal-app[bot] <windmill-internal-app[bot]@users.noreply.github.com>
 2026-09-01 23:42:08 +02:00
Diego Imbert 68e7f72560 fix(datatables): the database a permissioned data table points at cannot move 
The path in the config staying the same said nothing: a postgres resource is
editable in place, so its host, database or user could change underneath roles
whose logins and grants live in the database it used to name. The identity a
connection resolves to is what has to hold still while those roles exist; a
password rotation is not an identity change and stays allowed.

Also generalizes the admin guard: what 'admin' holds is what every role here
connects through, on the database and on schema public alike, so a revoke
naming it is refused wherever it is aimed.
 2026-09-01 23:37:05 +02:00
Diego Imbert 88d6e11ff0 fix(datatables): a permissioned data table cannot be pointed at another database 
Its roles live in the database it points at: the logins were created there and
every grant they hold is recorded there. Carried onto another database they
authenticate against a cluster that never heard of those grants. Opting out
first is what drops them from the database they belong to.

Also: the ATTACH test now calls the parser the executor runs, rather than a
byte-identical copy of its regex that no regression could reach.
 2026-09-01 23:19:42 +02:00
Diego Imbert 6e79d9636f fix(datatables): keep the database's own grants to admin out of reach 
admin is the login the data table reaches Postgres through, so revoking on
the database itself takes away what every role here connects with — and what
the role that would grant it back connects with. Refuse it server-side, and
stop the drawer from offering a row it cannot act on.
 2026-09-01 23:15:43 +02:00
Diego Imbert be186e6fdd fix(datatables): the objects a revoke names do not answer for its scope 
The ownership guard took the named-objects branch whatever the scope, and
returned. A revoke naming one table the caller owns, scoped to all tables,
therefore passed a check over that one table and planned a statement that
names the whole schema — the objects never reach the SQL there.
 2026-09-01 23:10:53 +02:00
Diego Imbert d6f36d3135 test(datatables): pin the object classes a schema-wide change reaches 2026-09-01 23:05:00 +02:00
Diego Imbert 9a63c6884d fix(datatables): count materialized views among the tables a scope reaches 2026-09-01 23:02:54 +02:00
Diego Imbert 6768e2a9e9 fix(datatables): check what a change reaches, not only what it names 
Owning the target is not owning what a change through it covers: a scope that
reads IN SCHEMA names every object in the schema, and handing a schema over
takes them all with it. Postgres would have skipped the ones the caller does
not own — these statements run as the data table's admin, so it will not.
Refuse, naming the object that is not theirs, and let a workspace admin
through as before.

A default-privilege rule speaks for the role that creates the objects, so a
non-admin now only writes them for the roles they may run as.

Also: the revoke button follows can_manage like the grant builder already did,
the copy path resolves a data table as admin for an admin (dumping as a
restricted role silently omits what it cannot read), and two doc comments now
sit on the function they describe.
 2026-09-01 23:00:51 +02:00
Diego Imbert fa4c06658f fix(datatables): resolve ACL objects against the catalog, and gate on the enterprise edition 
A revoke's objects came from the request, argument types included, and those go
into the statement unquoted — so a schema owner could close a routine signature
and append SQL that ran as the data table's administrative login. The request
now only names an object: what reaches the statement is read back from the
catalog, and an object that resolves to nothing is refused.

The planners were behind `private`, which community builds carry, so the
permissions API answered on a CE binary. They take `enterprise` as well, with
a test that pins the refusal in every other edition.

Default privileges are read back scoped to one data table's own roles: two data
tables can share a database, and a new role of one was inheriting the other's
rules.
 2026-09-01 11:58:58 +02:00
Diego Imbert 45d5eeda0e fix(datatables): close the fail-open in role resolution, and gate the ACL endpoints 
A role with a stored pg_rolename but no password resolved to the data table's
own connection, which owns everything — so a caller authorized as one role got
the admin one instead. Exports and git-synced settings redact that password, so
a restored config is exactly the shape that produced it. Refuse instead, and
name the fix.

The ACL endpoints took any workspace member: on a data table without roles
every member resolves to that same admin connection, so ownership and grants
there are the workspace admins' to change, as the roles themselves are.

Also: check a migration batch's roles before applying any of it, keep the role
picker for a single non-default role, drop the revoke button from a default
privilege on types (which no scope can express), and say what
get_datatable_resource_as_default_role actually resolves as.
 2026-09-01 04:48:44 +02:00
Diego Imbert 28b31a437e feat(datatables): gate role and ACL changes on the enterprise edition 
Both planners now live in windmill-ee-private, reached through a dispatcher
that refuses in the open-source build, and an enterprise binary additionally
checks for an active license before planning anything. Reading a data table's
roles, its usable roles and a schema's owner and grants stays open, so an
instance that lapses can still see what it has; every mutation is refused.

The UI stops offering what the server would refuse: no Roles entry on a data
table, and no Permissions entry on a schema or table, without a license.
 2026-09-01 04:32:52 +02:00
Diego Imbert 5f59b199b1 test(datatables): pin the audit parameter's redaction 2026-09-01 04:14:29 +02:00
Diego Imbert 2bc39481da fix(datatables): cover routines in ownership, grants and default privileges 
Functions live in pg_proc, so a schema changing hands left them behind and a
grant on ALL FUNCTIONS disappeared from the drawer on the next read, with no
way to revoke it. Read them alongside pg_class, and name them by their
identity arguments, which is what tells two of one name apart.

ALTER DEFAULT PRIVILEGES binds only the roles it names, so a role created
after a 'created later' grant produced tables no one else could read.
Replay the rules already in force for each new role.
 2026-09-01 04:09:19 +02:00
Diego Imbert 7619a7a51e fix(datatables): keep role passwords out of audit, and role names out of SQL 
The audit parameter of a data table config save carried the whole settings
blob, generated role passwords included. Redact it the way every other
export of that blob already is.

Both SDKs pasted the caller's role straight into the `-- role` annotation,
where a newline ends the comment and leaves the rest running as whatever the
first line named. Check the value against the role-name grammar the server
enforces.
 2026-09-01 04:09:12 +02:00
Diego Imbert a4e6eb5e4d fix(datatables): never hide access management from a workspace admin 2026-09-01 03:11:23 +02:00
Diego Imbert e3f0a22810 feat(datatables): manage access on what you own, not only as an admin 2026-08-31 23:39:49 +02:00
Diego Imbert 22f99f99f1 fix(datatables): grant created roles to admin with the SET option 2026-08-31 23:39:49 +02:00
Diego Imbert 709cf8aecb Merge remote-tracking branch 'origin/main' into datatable-perms-3 
# Conflicts:
#	backend/ee-repo-ref.txt
 2026-08-31 22:01:47 +02:00
Diego Imbert eb95a95f48 feat(datatables): a permissions drawer on a table 2026-08-31 06:52:03 +02:00
Diego Imbert a8a8812839 feat(datatables): group grants that repeat over objects into one row 2026-08-31 06:45:37 +02:00
Diego Imbert e989a4d9ae fix(datatables): report create privileges for the role the manager is on 2026-08-31 06:01:13 +02:00
Diego Imbert fc3732fb7f fix(datatables): repair an instance database's grant options where they are used 
Handing a privilege to a role means granting it, and a privilege held without
the grant option cannot be passed on — Postgres answers such a statement with
a warning and no effect, so the roles silently receive nothing. Databases
provisioned before those options were part of the instance grants still hold
them plain.

The repair now runs on the two paths that need it, saving permissions and
applying an ACL change, rather than only on opting in to migrations: a data
table can use roles without ever touching a migration.
 2026-08-31 05:55:50 +02:00
Diego Imbert 38023de582 feat(datatables): only offer create where the role may create 2026-08-31 05:52:42 +02:00
Diego Imbert e1556004f6 feat(datatables): grant database privileges from the roles drawer 2026-08-31 05:19:51 +02:00
Diego Imbert 395eac68c7 fix(datatables): give custom_instance_user the grant option on instance databases 2026-08-31 05:07:01 +02:00
Diego Imbert 924389a0ac fix(datatables): test the connection as the data table's own admin role 2026-08-31 04:39:56 +02:00
Diego Imbert 8429d909d4 fix(datatables): accept postgres 17's MAINTAIN table privilege 2026-08-31 04:33:22 +02:00
Diego Imbert bac89b0790 fix(datatables): allow deleting a migration that has already been applied 2026-08-31 04:33:22 +02:00
Diego Imbert 7effde8bbe fix(datatables): hide schemas the connection's role cannot reach 2026-08-31 04:33:22 +02:00
Diego Imbert 126204a11e fix(datatables): resolve pg_database_owner to the database's real owner 2026-08-31 04:15:21 +02:00
Diego Imbert 803e89319e feat(datatables): own and grant on a data table's schemas 
A permissions drawer on a schema row: who owns it, and what each role may do
in it. Ownership moves the schema and everything already in it, and writes
the default privileges that keep the owner in reach of what the other roles
create later — an object belongs to whoever creates it, so that is the only
way to cover what does not exist yet.

Grants are built as statements — privileges, scope, role — and read back from
the catalog, per object and including default privileges, so what the page
shows is what the database has. Every change is confirmed against its own SQL
and runs in one transaction.

The editor takes a target rather than a schema: a table is the same call with
one more identifier, for when the table and role sections want it.
 2026-08-31 04:03:42 +02:00
hugocasawindmill-internal-app[bot]Ruben Fiszel
 		7dd88c470c fix: unify billable seat counting and prevent fork subscriptions (#10818) 
* fix: unify billable seat counting and prevent fork subscriptions

* fix: authorize candidate before reading its plan, scope seat breakdown

* chore: pin ee ref for the stripe checkout fork guard

* fix: grant the billable_member view and widen the paid-plan check

* refactor: keep the seat rule in rust instead of a view and function

* docs: correct the attach guard summary after widening the plan check

* revert: keep cloud out of the ci test feature set

* chore: update ee-repo-ref to 9ff97cd818e85940fec282c92161e98c1b8583e2

This commit updates the EE repository reference after PR #742 was merged in windmill-ee-private.

Previous ee-repo-ref: 0ec0b42565a41f271a45bf24a93467d110c36df3

New ee-repo-ref: 9ff97cd818e85940fec282c92161e98c1b8583e2

Automated by sync-ee-ref workflow.

---------

Co-authored-by: windmill-internal-app[bot] <windmill-internal-app[bot]@users.noreply.github.com>
Co-authored-by: Ruben Fiszel <ruben@windmill.dev>
 2026-08-28 17:13:02 +02:00
Diego ImbertandClaude Opus 5 3ce9bbc716 fix(datatables): stop a fork's pg_dump restore from failing silently (#10830) 
* fix(datatables): stop a fork's pg_dump restore from failing silently

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_018t5LRAAT6ixHkc955ifmg6

* fix(datatables): keep source ACLs when importing into a resource database

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_018t5LRAAT6ixHkc955ifmg6

* fix(datatables): drop dump ownership on every import, ACLs only for instance targets

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_018t5LRAAT6ixHkc955ifmg6

* fix(datatables): probe the target through psql and drop an instance source's grants

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_018t5LRAAT6ixHkc955ifmg6

* fix(datatables): make a generated initial migration replayable elsewhere

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_018t5LRAAT6ixHkc955ifmg6

* fix(datatables): keep a resource data table's own ACLs in its initial migration

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_018t5LRAAT6ixHkc955ifmg6

---------

Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
 2026-08-28 16:56:05 +02:00
Ruben FiszelClaude Opus 5windmill-internal-app[bot]
 		8b80b09f33 fix: restrict filesystem workspace storage to debug builds (#10864) 
* fix: restrict filesystem workspace storage to debug builds

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Q7p2VbtYqaXHGaAskgwVk5

* chore: update ee-repo-ref to b58ad414b098d3d7787001a352bfbb13e43a335f

This commit updates the EE repository reference after PR #747 was merged in windmill-ee-private.

Previous ee-repo-ref: 1b4dada77a8fe2224579c643550c63b1ac2616de

New ee-repo-ref: b58ad414b098d3d7787001a352bfbb13e43a335f

Automated by sync-ee-ref workflow.

---------

Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
Co-authored-by: windmill-internal-app[bot] <windmill-internal-app[bot]@users.noreply.github.com>
 2026-08-26 23:35:44 +02:00
Diego Imbert ae162b6d3e Merge branch 'main' into datatable-perms-3 2026-08-26 10:24:13 +02:00
Diego Imbert 7e5ac78447 feat(datatables): badge each data table row with the role it is reached through 
Serve the caller's usable roles and the default one alongside the tree, so
the badge costs no extra round trip per data table. Shown only where naming
the role says something: a data table without permissions, or one whose only
usable role is the implicit admin, gets none.

The left-pane filter also matches schema names now.
 2026-08-25 21:00:26 +02:00
Diego ImbertandClaude Opus 5 3b2a6d7604 feat(datatables): add a down migration from the migration viewer (#10812) 
* feat(datatables): add a down migration from the migration viewer

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Ha889ogajX9jbqmwTaF8kD

* fix(datatables): refuse an empty down migration and keep the saved one visible

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Ha889ogajX9jbqmwTaF8kD

* refactor(datatables): use unifiedSize on the new buttons and fix the lock comment

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Ha889ogajX9jbqmwTaF8kD

* fix(datatables): make the add-down exemption atomic against concurrent additions

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Ha889ogajX9jbqmwTaF8kD

* fix(datatables): re-test the whole observed row when an upsert skips the lock

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Ha889ogajX9jbqmwTaF8kD

* fix(datatables): re-test the observed row even when none was read, and sync the down draft on the leading change

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Ha889ogajX9jbqmwTaF8kD

---------

Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
 2026-08-24 22:36:32 +02:00
Diego Imbert 6228e266a1 feat(datatables): run migrations as the role their SQL declares 2026-08-24 14:44:29 +02:00
hugocasaandClaude Fable 5 9c557859c5 feat: AI agent evals: datasets, scored runs and comparison (#10633) 
* feat: eval datasets and standalone runs for reusable AI agents

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* feat: agent eval drawer with case editor, runs and capture entry points

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* docs: document AI agent eval datasets and standalone runs

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* fix: say how many eval cases the list is not showing

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* fix: address review findings on eval datasets

- keep an edited case's conversation and tool inputs: serde(flatten) silently
  drops Box<RawValue> fields, so the update payload is spelled out
- remount the case editor per case so one case's turns cannot leak into another
- require jobs:read / flow_conversations:read on the capture endpoints, which
  UserDB does not gate by token scope
- take the dataset lock in create and update so a delete cannot be undone by a
  concurrent metadata write, and delete cases before metadata
- load more cases beyond the first page, and stop capping the agent picker
- record that the version stamp is taken at enqueue, not at resolution

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* fix: address round-2 review findings on eval datasets

- block operators from dataset and case writes
- pass the editor's operating workspace through the drawer and the capture
  request, instead of assuming the navigation workspace
- discard superseded case-list responses so switching datasets cannot land the
  previous dataset's cases
- reject a dataset without a case_id (or vice versa) rather than running an
  inline case under a dangling association
- run unsaved edits inline instead of silently running the stored case
- surface the API error body on a failed run
- fetch dataset metadata concurrently when listing
- $bindable() without a default on the optional open prop
- correct the permission and enqueue-time-version wording in the docs

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* fix: run an untouched saved case by reference again

The editor writes back keys the stored case omits, so comparing the raw objects
reported every unedited case as edited: the run went inline and lost the
dataset/case stamp its history depends on. Compare a normalized form, and pin it
with a test. Also scope the history query to the drawer's workspace and drop
superseded responses.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* feat: show a dataset's cases as a table, and fix round-4 review findings

The case list showed one case at a time with no overview. It is now a table with
the case, where it was captured from, and its last run — the last-run column is a
single jobs query on the path stamp rather than a request per row.

Review fixes in the same file:
- keep the edit baseline on the selected case rather than looking it up in the
  loaded page, so a case beyond page 1 is not treated as unedited and run stale
- release the loading state when a superseded case load returns early
- reload every loaded page after a write instead of collapsing to page 1
- last remaining 'resolved to' wording in the version tooltip

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* feat: run a dataset as an experiment, with scorers as runnables

An experiment runs every case of a dataset against one subject and records the
exact case set it executed, so a result set stays reproducible while the dataset
keeps changing.

Each case runs as its own small flow — the agent, then a step per scorer — so a
case keeps the run stamp, history query and trajectory view a single run already
has, and scorers need no orchestration of their own. Results are read back per
step by node id rather than by walking a nested loop's status.

A scorer is any runnable taking (input, output, expected): a script, a flow, or a
reusable agent used as a judge. A judge is prompted with the case and the answer
as one JSON message; a script or flow receives them as named arguments. Scores
accept a bare number, a boolean or {score}.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* feat: results table for an experiment, with scorer columns

One row per case: status, the agent's answer, and a column per scorer, with the
mean per scorer above the table and a link into each case's run for its
trajectory. Averages skip cases a scorer produced no number for — counting a
missing score as zero would read as a regression.

The drawer's left pane becomes Cases / Results, and Results carries the scorer
picker and Run dataset.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* feat: compare an experiment against a baseline

Per-scorer deltas on each row and on the mean, and a filter down to the rows that
regressed. Rows join by case id, so a case added after the baseline ran has no
delta instead of counting as a change.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* fix: address round-5 review findings on experiments

- match scorers by label when diffing two experiments; joining by array position
  subtracted one scorer from another whenever the scorer sets differed
- report a row's status from the case job, not the agent step, so a case whose
  scorer failed no longer reads as a success
- delete a dataset's experiments with it: they hold copies of its cases, and a
  recreated dataset of the same path would have exposed them
- select the experiment that Run dataset just started instead of leaving the
  table on the previous one
- expected is scored now, so stop describing it as having no consumer

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* fix: address round-6 review findings on experiments

- hold the dataset lock across an experiment launch, so a delete landing between
  reading the cases and writing the experiment cannot recreate the deleted
  dataset's inputs
- match scorers between experiments on kind and path, not on label: labels
  default to a path's last segment, so f/a/quality and f/b/quality compared
  against each other
- average mean deltas over the cases both runs scored; comparing each run's own
  average reported a regression from a case the baseline never ran, with no
  regressed row to point at
- openapi: the row status is the job's, which is also canceled/skipped; runEval
  takes scorers; the update-case body no longer advertises source, which the
  handler deliberately ignores
- record why the experiment prefix cannot reach a sibling dataset

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* fix: address round-7 review findings on experiments

- release the dataset lock for the push loop and retake it for the write,
  re-checking the dataset still exists: holding it across the whole launch made
  every capture and case edit on that dataset 409 until the last job queued
- assemble experiment results with bounded concurrency; a 100-case, 3-scorer
  experiment was 400 sequential lookups, each itself several queries
- clear the baseline when it becomes the selected experiment, which was
  comparing a run against itself and reporting zero deltas
- take the header mean over the same cases as its delta while comparing, so the
  two numbers beside each other describe the same set
- a canceled or skipped case is no longer the same grey dot as a running one

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* fix: address round-8 review findings on experiments

- verify the dataset's identity, not just its existence, before recording an
  experiment: the path can be deleted and recreated during the push loop, and
  the experiment holds copies of the old dataset's cases
- give the recording lock a longer budget than a case edit, since its jobs are
  already queued and giving up strands them, and say so when it fails
- keep score lookups sequential within a case: nesting two bounded streams
  multiplied into 32 in-flight queries against a 50-connection pool
- clear a baseline that no longer belongs to the loaded experiments, so
  switching datasets does not leave comparison mode on with nothing to compare
- keep a scorer's own mean when the baseline never ran it, instead of blanking a
  column full of numbers
- EvalCaseDraft.expected no longer claims nothing scores it

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* fix: do not trust an experiment's job ids, and require write to record one

Experiment objects live in workspace object storage, which a script can write
directly, and results are read on the unrestricted pool — so a forged experiment
naming another flow job returned output the jobs API would have refused. Only
jobs this server stamped with that experiment's id are read now.

Also from round 9:
- recording an experiment requires write on the dataset, not read: it persists
  into the dataset's namespace and its shared list
- clear the results table when the selection changes and surface a failed load,
  instead of labelling the previous experiment's numbers as the new one's
- a storage fault is no longer reported as a deleted dataset
- the lock-timeout message at the recording site no longer says to retry, which
  would run the whole dataset again on top of the jobs already queued
- ExperimentRow.status documents canceled and skipped

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* fix: bind the experiment trust check to the requested dataset

The previous check matched jobs on the experiment id alone, which the stored
object supplies — so copying another dataset's experiment JSON under a readable
key carried its jobs' output along with it. A job is now only read if it was
stamped for this experiment *and* for the dataset the caller's read access was
checked against, and an experiment that names a different dataset is not served
from this key at all.

Also from round 10:
- add the .sqlx entry for that query; without it every SQLX_OFFLINE build failed
- serve results over GET: as POST the route-scope middleware classified a read
  as ai_evals:write, locking read-only tokens out of their own results
- clear the selected and baseline experiments synchronously when the dataset
  changes, so the previous dataset's id is not requested under the new one

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* fix: address round-11 review findings on experiments and scorers

- give scorers the whole case input, not just the message: an answer that came
  from attachments or a replayed conversation could not be judged on it
- accept a judge's boolean and structured {score} answers, including stringified
  ones, and pin every documented scorer shape with a test
- record an experiment for the cases that did launch when a later push fails,
  instead of leaving those jobs running with nothing to attribute them to
- do not capture a preview parent's synthetic runnable_path as a host flow; the
  saved case could not be rerun
- clear the case table before loading a dataset and surface a failed load, so a
  failure cannot leave the previous dataset's cases under the new name
- keep the results table through a refresh of the same experiment
- exclude flow-step jobs from the per-case last-run lookup
- drop case sets from the experiment list, which is only used to pick a run
- report a database failure at the recording lock as itself, not as contention

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* fix: address round-12 review findings on capture and run history

- load flow_node.flow for flownode parents: an agent inside a deployed branch or
  loop captured without its agent, host flow or tool bindings
- decide host_flow_path by whether the path resolves to a flow, not by job kind:
  excluding previews wholesale also dropped the flow editor's step test, whose
  path is real
- page the per-case last-run lookup by created_before until the loaded cases are
  covered; one page of 200 reported older cases as never run
- do not record an experiment when nothing launched
- only attach the case input to a job when a scorer will read it
- keep the case table through a save; only a different dataset clears it
- drop the superseded duplicate comment on the score parser

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* fix: stop refetching run history on every case write

Reading the case list before the first await made the whole job-history query a
dependency of it, so every save, delete and Load more refetched up to 1000 job
rows and blanked the column. Read untracked instead.

- an empty Last run cell now distinguishes never-ran from not-found-within the
  page bound, which the comment already claimed and the cell did not
- reloading a dataset no longer replaces a populated table with a skeleton
- keep the score-parser comment that describes every shape it handles

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* refactor: keep eval datasets in Postgres instead of object storage

Datasets, cases and experiments become rows (`eval_dataset`, `eval_case`,
`eval_experiment`, `eval_experiment_case`) rather than objects under a
`wmill_eval_datasets/` prefix. What a run produced is still the job's:
only case inputs and an experiment's case snapshot are stored.

This removes the machinery the object store needed:

- The advisory lock and the read-modify-write of a per-dataset JSONL. A
  case is a row, so there is nothing to serialize.
- The launch-time identity check on the dataset. The foreign key makes a
  concurrent delete fail the transaction instead.
- The trust guard on an experiment's job ids, which existed because a
  script can write workspace object storage directly and could forge an
  experiment naming somebody else's job.

An experiment now chooses every job id and records itself before pushing
anything, so a launch that dies partway leaves a recorded case whose job
is missing rather than a running job nothing accounts for; cases that
never reached the queue are removed again.

Row-level security on `eval_dataset` is the authority on who may read or
write a dataset, so `extra_perms` grants work and the rule is not
mirrored in Rust. Cases and experiments carry a read policy derived from
their dataset and no write policy: they are written on the unrestricted
pool after the dataset row itself has been asked, with
`SELECT ... FOR UPDATE`, whether the caller may write it.

Cases are capped at 256 KiB each and 10 000 per dataset, refused rather
than truncated. Attachments are S3 references, not inline bytes, so a
case that approaches either cap is a mistake rather than a use case.

Evals no longer need the `parquet` feature or a configured workspace
object storage.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* style: align the eval drawer with the design system

- Scorer chips are `Badge`s rather than a hand-rolled bordered span, and
  the section header is a `Label` with its tooltip, as are the case
  editor's fields (which also gets the label colour right).
- The results table showed status as a coloured bullet, which says
  nothing to a colour-blind reader. It now carries the same icons the
  runs table uses, with the status as its accessible name.
- Feedback colours move to the `-500` shades the brand guidelines name.
- The conversation JSON error uses `TextInput`'s `error` prop for the
  border and the caption style for the message, as elsewhere.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* feat: author an expected answer, tags and attachments on a case

Every scorer is handed `(input, output, expected)`, but nothing could
produce an `expected` except a conversation capture: the case editor had
no field for it and a captured run left it empty. So:

- The editor gains Expected, Tags and a read-only list of the
  attachments a captured case carries. Expected is plain text, or JSON
  when the answer has structure.
- Capturing from an AI agent run keeps what that run answered, which is
  the only moment a reference answer exists for free.

The results table also laid itself out by content, so a long answer
pushed the scores — the numbers the table exists for — off the edge of
the pane. It is fixed-layout now, with the text columns bounded.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* docs: expected is captured from a run and can be authored

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* feat: link a saved agent when inserting an ai agent step

"AI Agent" in the step picker was a leaf that always created a blank
step, so reusing a saved agent meant inserting a blank one, opening its
step input and linking it there. It is a category now, like Flow and AI
Sandbox, listing the workspace's `ai_agent` resources next to a blank
option, filtered by the picker's own search.

A picked agent produces a step that is already linked rather than one
linked afterwards: `agent` set, no tools, and only the flow-local
`user_message`/`user_attachments` transforms. Seeding the brain keys
there would leave transforms a linked step never reads and that
`AgentResourceBar` strips on its next link change.

Each `on:new` forwarder rebuilds the insert detail field by field
instead of spreading it, so a new field is dropped unless the forwarder
names it. `agentPath` is typed on both `GraphEventHandlers.insert` and
`FlowGraphV2`'s `onInsert` so the next one to forget it fails the check.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* feat: restore the link on cancel and simplify the agent bar

Cancel on an agent edit forked the step into a standalone copy, which is
the opposite of what the word means and needed a paragraph under the
card to explain. It discards the edits and re-links the step now,
leaving the agent untouched; diverging from an agent is Unlink's job, on
the linked card. This flow's `tool_inputs` survive the round trip as
overrides, so Cancel no longer folds them into the tools the way Unlink
does.

Linking a step to a saved agent happens in the step picker at insert
time, so the bar's own resource picker is gone and "Save as agent" is
the one action left. Its `+` button was a trap besides: it opened the
generic resource form, where an agent would have to be written as raw
JSON.

The card itself was `surface-secondary`, the sections token, so in dark
mode it was darker than the pane and read as a sunken well rather than
an elevated card. It uses `surface-tertiary` as the brand table
prescribes, its tool chips are `Badge`s, and the editing card no longer
overflows the pane and clips its own buttons. The remaining tooltip
follows the inline `Label` convention rather than sitting in a flex row
whose gap stacked on the trigger's own margin.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* feat: rework the AI agent evals surface into one table

Evals become a single pane: a dataset of cases, one column per scorer, one
row per case, with the run being looked at chosen from the toolbar.

Runs are permanent. Running the whole dataset opens one; running a single
case records nothing at all — it is a job, and looking at what it did is
not a claim that it belongs in the history. Its result and its scores sit
over the row until they are saved as a run, which carries the cases that
were not rerun and the scoring jobs themselves, so the number that is
saved is the number that was looked at.

A scorer is a runnable: a judge agent or a script, created in one click and
edited in place. Scores carry a reason and per-assertion checks, shown on
hover with a rescore button.

What ran is always named. A run records the agent version, or — for a
configuration that is not deployed — a hash of it, so a table can say that
its numbers describe an agent that no longer exists: those rows dim and the
table offers to rerun. An agent's draft can be run directly instead of the
deployed value, and once those edits are deployed the runs that made them
are recognised as that version. A step with no agent of its own is
evaluable too, and saving it as an agent moves its history onto it.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* feat: keep an agent's in-progress edits on the agent

Editing a linked agent forks it into the step, which is what makes the
edits runnable there — but the agent is what is being edited, so that is
where the unsaved state belongs. The edit is mirrored into the agent's own
resource draft as it is made.

It then survives leaving the flow, shows the agent as drafted wherever it
appears, and is what evals run when asked to run the draft rather than what
is deployed. Deploying or cancelling clears it; opening Edit without
changing anything does not mark the agent as drafted.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* feat: shape the evals surface around a saved agent

Evals hang off an `ai_agent` resource, so the surface is now only ever about
one: the `draft` subject kind, the standalone-step subject and the move that
carried a step's history onto a newly saved agent are gone.

- A run is permanent and numbered per agent. Running a single case is a trial:
  it answers in the panel and never touches the table.
- "Run scorers only" opens a run of its own that reuses the answers of the run
  you are looking at, so a scorer added later measures what already ran without
  calling the agent again.
- A draft run whose configuration is later deployed is stamped, once, to the
  version it became, so its label stops reading `v23 + edits` forever.
- A scorer can carry a pass threshold, read off the scores already recorded.
- The table is the case, its answer and one number per scorer; datasets are
  created and edited in a drawer; a run that executed an earlier state of the
  current draft says so above the table, in one line.
- Which agent a step is, whether it is being edited, and which version it is on
  is a strip above the step's tabs, because it is true of every tab.
- Capturing a case from a step test or a conversation is dropped, and with it
  the `memory` override on a linked step that nothing set.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* feat: run past versions of an agent, and number versions per resource

The evals home becomes one table of every run of the agent, whichever dataset
each is of, with one badge per scorer. A list spanning datasets cannot hold every
dataset's scorers to look a name up, so a score carries its name and kind with
its number, and thresholds are joined in per run and column.

Run now asks what to run: the latest agent, resolved when the run executes as a
flow step does, any past version, or the unsaved edits. Pinning is a subject kind
of its own, since a linked step resolves the resource live and inlining is the
only way to run a version that is no longer current.

Scorers move into the edit-dataset drawer. The column header over a run reports
and nothing else: a run is permanent, and a control there that changed the
columns would edit the past from the one place that must not. Adding one offers
four ways rather than two, writing and reusing being different jobs, and both new
kinds open with a summary filled in.

Versions are numbered per resource. `resource_version.id` is one identity
sequence for the whole table, so an agent saved nine times read v4 ... v24, and
the gaps counted writes in workspaces the reader cannot see. The id stays how a
version is addressed; the new number is what it is called, in the resource
history drawer as well as here. It is assigned on write rather than counted on
read because trimming past the cap and clearing a history both take the oldest
rows, and counting the survivors would renumber a version a run already names.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* fix: read the dataset a remembered selection names

Reopening the evals modal restored the last dataset from storage as a bare path,
without reading the row it names. Every "is this already the one?" test compared
against that selection, so all of them short-circuited and the dataset was never
loaded: editing it opened a drawer with no summary, no scorers and no cases.

The remembered path is now brought into context the same way any other choice is,
and the tests compare against the dataset that is loaded rather than the one that
is selected, so a selection can no longer stand for a read that did not happen.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* feat: give dialogs a trail in their header

A dialog deep enough to navigate had nowhere to say where you were: the header
held a fixed title, and the way back was a control each body placed for itself,
somewhere in a toolbar that moves with everything else the toolbar holds. The
header is the one part of the surface that does not move, which is where the
trail belongs.

`Modal` takes an optional `trail` of levels below its title, rendered as a
breadcrumb whose ancestors are the way back. Declarative on purpose: callers of
this depth already hold the state that says where they are, so the dialog reads
it rather than owning a stack they would have to push and pop in step with it.

Escape follows the trail. Leaving a level is what someone deep in a dialog means
by it, and closing the whole surface throws away the navigating they did to get
there; at the root it closes as before. That only works if a dialog can tell it
is the surface being addressed, so `Disposable` now answers `isTopmost()` and the
dialog asks before acting: it keeps Escape for itself, so nothing else was
arbitrating between it and a drawer opened from inside it, and both were acting
on one key press.

Evals is the first caller: its runs list is the root, a run is a level in it, and
the back button that used to sit above the table is gone.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* fix: portal dialogs out of wherever they were opened from

A dialog rendered in place inherits whatever the calling component happens to sit
inside. One `transform`, `filter` or `overflow` anywhere above it makes its
`fixed` positioning resolve against that ancestor instead of the viewport, and a
surface meant to cover the app is then confined to a box it never asked for: the
nav rail paints over it and its own edges are clipped.

Drawers have always portalled for this reason. Dialogs only did so when an
enclosing pane claimed them, and rendered in place otherwise, so the same screen
could show a drawer over everything and a dialog trapped behind the nav. They now
portal the same way: to the pane when one claims it, to `body` otherwise.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* fix: make the dialog's title the first step of its trail

The trail listed levels below the title, so a dialog one level deep read
"Evals > All runs > Run 20 · v6": three steps for two places, the first two of
them the same place under different names. The title is the root, so it is the
root's own segment, and the trail a dialog is given is now the whole path with
that segment at its head.

Its height stopped moving too. A heading carries a line-height of its own, so a
header holding only an h3 stood six pixels shorter than one holding segments as
well, and the dialog's whole top edge stepped as you navigated.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* fix: sharpen the evals controls around where you are standing

Each screen now offers what belongs to it. The list starts runs; a run is a
record, so it offers only the one thing that acts on the record itself, which is
measuring the answers it already stored. Starting a fresh run from inside one
asked which agent and which dataset from the screen least about either, and
scoring an existing run was offered from the list, where there is no run to
score. Which run and what it is read against are one question asked twice, so
they sit together rather than at opposite ends of a row.

Choosing what to run is now a toggle over the two states worth naming, the draft
and the saved agent, with every earlier version one click further: running an old
version is deliberate, and a list made all three look alike. The draft is read
when the dialog opens rather than taken from the caller's polled copy, which
could be seconds behind an agent edited a moment ago and would leave the option
out exactly when it is the reason for opening the dialog.

The dataset field carries its path under it and its edit button on hover, as a
resource picker does, so the closed field says what the open list said. Edits
waiting on an agent are a "draft" here as everywhere else in Windmill, rather
than "+ edits". The dialog runs an evaluation rather than "the agent", which is
what it was already called everywhere it is recorded. An agent being edited keeps
its evals button on a line of its own, clear of the decision to save or discard.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* fix: settle the evals controls on the patterns Windmill already has

The version choice uses ToggleButtonMore, as the AI provider picker does: the two
states worth naming stay in the group, the rest are behind the overflow menu, and
the one you pick joins the group rather than appearing in a second control below
it. The deployed one says which version it resolves to.

A run offers nothing to start. Scoring an existing run again was the last thing
left there, and it was one button explaining a distinction that the run and the
dataset already make between them.

The warning that a run executed an earlier draft is about the run on screen, so
it goes when the run does rather than following you back to the list, and it sits
against the table instead of inside a frame of its own.

A dataset just created stays open for its scorers and cases: those are what a
dataset is, they can only be added to one that exists, and closing on create sent
you to find it again to add them. Scorer settings are a cog rather than a word,
now that the row holds three actions.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* fix: close the gap in the version toggle and say what naming a dataset does

The overflow trigger is not a pill, so the room it reserves showed as a gap
between it and the button before it; it is pulled in by that much. The dataset
field gets its clear button, which is also the slot the edit button is positioned
against, so the two now sit where a resource picker puts them.

Naming a new dataset said nothing about what happens next, and the drawer looked
like it was missing the rest of itself. It says so instead: a scorer and a case
both belong to a dataset, so there is nothing to attach either to until this one
exists, and creating it leaves the drawer open on them.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* feat: choose a dataset's scorers while naming it

A scorer is a reference to a runnable, not a child of the dataset, so it needs
the dataset's name but not its row. The list is collected in the drawer while the
dataset is being named and sent with the create, which already accepts one, so a
dataset arrives holding the columns that were chosen for it rather than being
made empty and then edited to hold them.

Cases stay where they were: a case *is* a row of the dataset, so there is nothing
for it to be a row of until one exists. The drawer says which of the two is which
instead of leaving the screen looking like it is missing the rest of itself.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* fix: level the version toggle and name the dataset in its own field

The overflow trigger stands a row taller than a toggle button, so the group grew
to its height and left the sunken background showing under every pill beside it.
Every child of the group is the same height now, which is why the AI provider
picker never had the band: it sizes them all alike.

The dataset field says the summary with the path after it rather than carrying
the path on a line below. The list stacks the two, which a one-line field cannot
do, so it says both the other way round.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* fix: tidy the evals forms and the run's own controls

Picking a scorer that exists chooses between two sources rather than showing
both: the ones already measuring something, and everything else in the workspace.
The first list says what each is called with its path under it and what it
already measures on the right, instead of three columns that were the same path
truncated three ways whenever a scorer had no name of its own.

A dataset's drawer says what it is for on the page rather than under an icon, and
its summary is sized like the field beneath it.

The run's own row lines up with the table under it, the warning above that table
is spaced off the rule rather than sitting on it, and adding a case is gone from
a run: a run is a record of cases that were answered, so curating them from it is
editing what it measured.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* feat: create a dataset holding the cases written for it

Creating a dataset takes the cases to create it with, so one can be assembled in
a single act instead of made empty and then filled in. The drawer holds them
while the dataset is being named, gives them ids of its own to be edited by, and
sends them with the create.

Every case is checked before the dataset is written. `eval_case` grants users no
write, so the rows cannot be inserted in the transaction that creates the dataset
under the caller's own policies; validating first is what keeps "created holding
these cases" from becoming "created, holding some of them", and the rows that do
follow go in one transaction of their own.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* fix: name the button for what it opens, and say what each version is

Starting an evaluation asks which state of the agent and which dataset, and both
cost a provider bill, so a button that read as spending one on the way past was
lying about the click. It opens something, and says so. Running one case from the
panel keeps its own name and its play icon, because that one does run on click.

The version options say what they are rather than what they are not: what a flow
step would or would not run is a fact about somewhere else, and someone choosing
what to evaluate is not standing in it.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* fix: give the editing card two rows and mark evals as beta

At the width of a step panel the card's one row wrapped: the line naming the
agent, the line saying what saving does, and the two buttons deciding the edits'
fate all fought for it. Deciding gets a row of its own, and evals sits against the
line it is about, since evals of an agent being edited run the edits.

Evals is named wherever it is offered. It read as a word in one state of the card
and as an icon in the other, which is two things to recognise for one door.

The dialog carries a beta badge against its own name, before any level below it:
every way in lands there, so it is said once and stays put as you navigate.

The version toggle spells out which is which. Both are the agent at v2 and the
difference between them is the whole choice, so it is worth the width.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* fix: name a new dataset, and lay the scorer's settings out like a step's inputs

A new dataset arrives called "Dataset 1", which the path follows as it follows
any summary: a dataset with none was one every table could only call by its path,
and the two seeds are what the summary rule already produces.

Scorer settings put each field's description between its label and its input,
where a step's inputs put theirs, and its inputs are the size the rest of the
drawer uses. The runnable behind the column is a link to it with its kind's icon,
since it is a resource of its own and the one thing about it these fields cannot
change. The line explaining that a pass line re-reads recorded scores went: the
threshold is a number to set, and how it is applied is not a decision being made
here.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* feat: curate a dataset in the drawer and save it in one act

The drawer holds the cases while they are edited and writes them when it is
saved: added, changed and dropped, whichever it is. Typing no longer writes, so a
set is never half saved while someone is still deciding what is in it, and Save
means the same thing whether the dataset exists yet or not.

A case panel offers reading rather than acting. Running one case now and editing
one from a run were the last two ways to change a record from the screen showing
it, and the machinery behind the first went with it. The answer is rendered as
the prose it is, under what it is: the case's result, whichever run is selected
above it.

The rest is what the run's table was doing to its own edges: a column name is
clipped to its column rather than running into the next, the table squares off
against an open panel, and that panel closes with the run it belonged to.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* fix: one border above a table, and a link to the run's job

The row above the table drew a bottom border and the table draws its own top
edge, so every table sat under two lines. The row keeps its spacing and the table
keeps its edge.

A column header no longer spins while its scores arrive: the cells under it are
where the numbers are missing, and they say so themselves. The beta badge is the
height of the word beside it rather than of the line it sits on.

A run is one flow and therefore one job, so the run says where that job is: what
it is doing, what it cost and what it logged are all there rather than
reconstructed from the table.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* feat: stream scores as each scorer finishes, and show them per case

A scorer runs after the agent inside the case's own iteration, so its verdict can
be read as soon as its step is done. Waiting for the iteration to end held every
column of a case back until the last of them finished, which is why answers
arrived one at a time and scores all at once.

Reading a job that is still running needs one guard: a module with nothing in it
is a step that has not run, not one that produced nothing, and recording the
second makes a failure that never goes away.

The panel beside the table shows what each column made of the case and why. The
reason a judge gave was stored and never shown, which is the half of a score that
says anything. It stops repeating the question the header already asks, and a
case still running reads as waiting rather than as an answer that says "Running".

A run is a number beside a dataset, so the list puts the two together.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* feat: score a case with every scorer at once

The scorers of a case read the answer and never each other, so they ran one after
another for no reason: measuring a case now takes as long as its slowest column
rather than as long as all of them. Each is a branch of its own, kept from
failing the others, so a judge that errors costs its own column and no more.

An iteration is three steps again — answer, payload, scores — rather than one per
scorer, and each branch is named for the column it produces, so the graph of a
run says which scorer did what instead of spelling out an id.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* fix: read a judge's score out of the JSON it nearly wrote

A judge quoting the agent inside its own reason writes those quotes unescaped,
which is invalid JSON and also the most ordinary sentence for it to produce. The
whole verdict was being thrown away over it, so a column that had a number
reported having none.

The number and the reason are now read straight out of such text. Deliberately
not a second JSON parser: it finds the two keys and takes what follows, which is
what survives a quote in the middle of a sentence.

A case still running says so with a spinner rather than with the word "Running"
sitting where its answer goes.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* feat: ask a judge for a shape instead of trusting it to write one

A new judge carries an output schema, so the provider holds it to `{score,
reason}` rather than the prompt asking it to. Windmill already delivers a schema
whichever way the model takes it, a tool for Claude and Bedrock and the native
parameter elsewhere, so there is no list of models to keep here.

An agent with no runs offers its first one where the first row would be, rather
than from a toolbar above a table that has nothing in it.

Starting a run no longer picks a dataset for you. It fell back to whichever came
first, which on an agent that has never run means offering another agent's set as
though it were the obvious one; and with no dataset at all it says so and offers
the one move there is.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* feat: report a column that failed throughout, and hold the run dialog

The runs overview dropped any column that produced no number, so a judge
that failed on every case of a run vanished from the row and read as a
column nobody had asked for. The aggregate now reports every column that
has cells, with the count of the ones it failed on, and the badge says
"failed" where there is nothing to average. A column with no cells at all
is still left out: that one was added after the run and has nothing to say
about it.

Creating a dataset closes the drawer rather than turning it into an edit
of what it just made: scorers and cases already ship with the create, so
there is nothing left to stay open for. Reached from the run dialog, it
gives the screen back with the new dataset selected, and the dialog keeps
the version you had already chosen.

Also:
- the case panel's job link moves to the panel's own header, where its
  scope is: the job is the whole iteration, not the answer it sat over
- one action in the scorer drawer's header, as its neighbours have. The
  reuse list picks rather than adds, and says which dataset each column
  already measures
- adding a case is the last row of the list it lands in
- the pane shows what it has read rather than an empty state it has not
  earned yet, and its rows say they open
- the linked agent card loses a border it had inside another one

* fix: keep the linked agent card's outline

The card is a thing inside the step's inputs rather than a section of
them, and the outline is what says so. Only the rule inside it goes: the
detail it separates is already set apart by being detail.

* refactor: fit the eval surface to the shipped design

* feat: give a nested dialog a back control and the runs list its own moves

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* feat: put a dialog's description under its title

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* refactor: fold a dialog's back control into the crumb it returns to

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* feat: edit a dataset's cases as a table rather than a list beside a form

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* feat: edit a dataset's cases in the grid the data tables are edited in

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* feat: edit a grid cell of prose in place, and cap a dataset at one page

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* refactor: keep the cell editor's styles beside it, not in the vendored theme

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* fix: keep an empty cell empty and cap the editor's growth

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* feat: name the step that assembles a run for the scorers

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* feat: run the payload step natively, and say so when nothing serves that tag

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* fix: report an answer as answered while its scorers are still running

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* feat: let a scorer say a case is not one it measures

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* fix: score the answer, and leave a case with no expected answer unmeasured

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* refactor: split the evals backend into modules

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* fix: record what a run produced so it outlives its jobs

* fix: read only the agent step's own tool jobs into the payload

* fix: pin a run's configuration and give the judge the attachments

* feat: write a dataset's cases in one transaction

* chore: refresh the sqlx cache for the eval queries

* fix: drop results a newer selection has superseded

* fix: keep a draft the agent editor never opened on

* feat: let a run record what it produced instead of waiting to be read

* fix: serialize the replacements of a dataset's cases

* fix: stop the poller from superseding a read slower than its interval

* chore: refresh the sqlx cache

* fix: keep a failed read from settling a cell as a case with no answer

* fix: hold the case grid while its save is in flight

* fix: keep a failed collect step from failing the run it recorded

* chore: refresh the sqlx cache

* fix: commit an open cell into the save that reads it

* refactor: size the eval buttons with unifiedSize

* docs: describe a run as the one flow it is

* fix: show a run's recorded rows when part of it cannot be collected

* refactor: size the remaining PR-added buttons with unifiedSize

* fix: save the dataset name that was submitted, not the one typed after

* fix: force an open cell into the save that was pressed for it

* fix: refuse to score a run whose evidence could not be read

* fix: hold one lock over a dataset's case count and its writes

* fix: keep one unreadable run from costing the whole runs list

* refactor: drop the banned bindable-default from the eval props

* fix: hold the scorer controls while the dataset is written

* fix: read only the caller's own draft of an agent

* docs: say in the contract that a run pins its configuration

* fix: say a scorer did not run rather than blaming a missing answer

* feat: resume the agent draft you already had when you press Edit

* refactor: build the trail and dataset controls from Button

* fix: clear the open-cell flag when the drawer reopens

* chore: refresh the sqlx cache

* fix: read a run's configuration and its version from one snapshot

* fix: refuse a dataset path or summary the column cannot hold

* refactor: handle the agent draft the way the resource editor does

* fix: run only a configuration the launch actually read

* docs: bound dataset path and summary where they are submitted

* fix: surface a stalled agent draft instead of claiming it is kept

* fix: stop claiming a draft holds edits a failed write never sent

* fix: word a missing score only once the run says whether the case answered

* fix: let a breadcrumb crumb shrink so its truncation applies

* docs: describe where an agent's unsaved edits live and what drops them

* fix: keep harvesting scores when the run cannot yet word a missing one

* fix: report a refused draft write the card was reading as a save

* fix: drop the refused draft write when the server copy is taken instead

* refactor: build the scorer and dataset pickers from the design system

* fix: say what removing a scorer column actually does

* fix: drop a refused draft write wherever the server copy is read

* fix: let a picker row be as tall as the two lines it holds

* docs: record what removing a scorer column does to recorded runs

* fix: send a queued draft write before reopening, and drop only what it refuses

* refactor: write the agent draft at commit points instead of mirroring keystrokes

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* refactor: run an agent's edits from the step instead of keeping them as a draft

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* fix: make the diff badge keyboard operable and refuse an edits run without its edits

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* fix: drop the dataset icon from the scorer picker rows

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* fix: size the evals buttons like the rest of windmill and call a run of edits edits

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* fix: count a brain expression as an edit of the linked agent

* fix: cap scorers per dataset and report a launched run as launched

* fix: harvest scores in one read, refuse duplicate case ids, allow group paths

* fix: mint scorer ids server-side, save a dataset edit in one request, check attachments

* fix: write a dataset edit and its cases in one transaction

* fix: atomic dataset create/edit, reset eval pane per agent, stable pending scorer ids

* refactor: govern eval_case writes by RLS so a dataset edit is one transaction

* fix: pin launch snapshot, order case locks, cap dataset size, guard stale load

* fix: cap dataset bytes on single-case writes, reset run-dialog flag on load failure

* feat: migrate eval datasets on username change, settle unspawned cases, drop unused case endpoints

* fix: resolve scorer scripts as the caller and pin their hash; migrate scorer paths on rename

* fix: bound a failed tool call's error to the payload truncation cap

* fix: pin scorer hash as a hex string, reject missing judges, migrate eval authorship

* fix: record an out-of-range scorer result as an error, not a score

* fix: resolve judges in one caller-scoped read, pin deployed scripts, bound pass_if

* fix: settle unspawned cases only when the run completes, and their score cells too

* feat: reassign eval datasets and their path references when offboarding a user

* fix: use the regex backreference in offboarding eval path rewrites

* fix: register eval datasets in offboarding registries, keep resource-version param name

* refactor: name the resource-version path param id, since it is the row id not the version

* fix: validate dataset paths canonically, clone eval data on fork, surface eval load and launch failures

* docs: note MCP tool results are not yet surfaced to eval scorers

* fix: show the eval error state on any load failure, not only an empty dataset list

* fix: preserve eval case order across a batched save

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01LVjewUvXFEjNLLw7kxz41h

* docs: scope the eval launch delete-safety guarantee to the assembly window

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01LVjewUvXFEjNLLw7kxz41h

* fix: only offer deployed scripts as eval scorers, drop unbuilt rescore claim

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01LVjewUvXFEjNLLw7kxz41h

* fix: enforce 0-1 scorer threshold in the settings drawer and clear stale eval load errors

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01LVjewUvXFEjNLLw7kxz41h

* fix: scope subject version/hash reads to the caller and keep a 0 pass threshold

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01LVjewUvXFEjNLLw7kxz41h

* fix: select the saved dataset when creating or renaming from the Run dialog

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01LVjewUvXFEjNLLw7kxz41h

* fix: gate eval dataset rename on path ownership, not just write access

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01LVjewUvXFEjNLLw7kxz41h

* fix: tolerate a malformed agent config when resolving the deployed label

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01LVjewUvXFEjNLLw7kxz41h

* refactor: trim eval code and comments, fix shared select and modal paths

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* fix: drop the rename warning when editing an eval dataset path

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* fix: add eval dataset delete, keep summary on partial edits, settle resultless scorer cells

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* test: cover parseThreshold and subjectLabel

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* fix: hold dataset Save during a scorer write, derive draft_hash only from the carried draft

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
 2026-08-24 11:08:56 +02:00
Diego Imbert 5974f5491d Merge remote-tracking branch 'origin/main' into datatable-perms-3 
# Conflicts:
#	backend/ee-repo-ref.txt
#	backend/parsers/windmill-parser/src/asset_parser.rs
#	backend/windmill-common/src/workspaces.rs
 2026-08-24 08:39:01 +02:00