Commit Graph
14663 Commits
Author SHA1 Message Date
Diego ImbertandClaude Opus 5 1f661f5816 chore(datatables): don't mount the roles UI until the ACL editor lands
Enforcement ships first. The permissions drawer is what turns roles on, and the
catalog section is what creates them — both are only useful once there is a way
to grant a role the privileges it needs, which arrives with the ACL editor. Left
mounted they would offer a feature whose other half does not exist.

The two components are complete and reviewed; only their call sites here are
commented out, with a note pointing the follow-up PRs at them.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BjfMkJyKzodxkobqGZ6Lqb
2026-09-09 17:19:30 +02:00
Diego ImbertandClaude Opus 5 397116926f fix(datatables): carry the role annotation into the row_to_json retry
The retry rebuilds its SQL from `pruneComments(code)`, so the leading comment
block never reached the second attempt — and with it the `-- role <name>` line
that decides which login the query runs as. The retry connected as the data
table's default role instead, so a query the first attempt was denied could
succeed on the second, reported as "recovered with the row_to_json fix".

Carry the leading comment block over. The retry itself is unchanged.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BjfMkJyKzodxkobqGZ6Lqb
2026-09-09 16:56:35 +02:00
Diego ImbertandClaude Opus 5 7b01f35c3b fix(datatables): refuse a malformed role query string instead of ignoring it
`?Role=analytics`, `?role=` and `?x=1&role=…` all fell through the reference
parser's exact-match rule, so the connection resolved to the data table's default
role and ran under a login the caller never asked for — the URI half of the same
trap as a malformed `-- role` annotation.

The key now matches case-insensitively, and anything else in the query string is
an error naming it; `role` is the only parameter a reference takes. Callers that
only need the entry keep a lenient `datatable_ref_name`, since they never act on
the role. The DuckDB `ATTACH` parser propagates it rather than attaching under
the default.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012ti5HyeTikPMYyW8YSdiHR
2026-09-09 16:00:19 +02:00
Diego ImbertandClaude Opus 5 41ea76f806 fix(datatables): refuse a malformed role annotation instead of ignoring it
`-- Role operator`, `-- role operator;` and `-- role operator -- why` all failed
the annotation parser's exact-match rule, so the query fell through to the data
table's default role and ran, silently, under a login the author did not choose.
Naming a role exists precisely to not do that.

A leading comment whose first word is `role` is now an annotation attempt: the
keyword matches case-insensitively, one trailing `;` is tolerated, and anything
else is an error naming the line. Only callers that already know the target is a
`datatable://` reference ever run this, so ordinary SQL keeps its comments.

Also bumps the dev shell's postgres client to 18 — it trailed the server the dev
database runs, which takes out every data table export, clone and fork-with-data.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012ti5HyeTikPMYyW8YSdiHR
2026-09-09 15:22:39 +02:00
Diego Imbert b3448d0111 fix pg_dump stuck on version 17 on nix 2026-09-09 10:07:15 +02:00
Diego Imbert f41aa85226 nit worker error msg 2026-09-09 10:06:55 +02:00
Diego ImbertandClaude Opus 5 9c2dbe423f fix(datatables): refuse the clone's database too, not only its data
A clone is two endpoints: `create_pg_database` then `import_pg_database`. Only
the second refused a data table under roles, so a fork asking to clone one
created and registered an empty `wm_fork_…` instance database and then failed —
and nothing collects it, since `drop_forked_datatable_databases` only drops
entries carrying `forked_from` and no entry names this one.

Refuse in both, so the clone stops before a database exists.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012ti5HyeTikPMYyW8YSdiHR
2026-09-08 21:40:54 +02:00
Diego ImbertandClaude Opus 5 e3a2f92c47 fix(datatables): refuse to copy a data table that is under roles
pg_dump carries no roles and the import runs with --no-privileges, so a copied
data table arrives owned by the admin connection with no GRANT for any role.
The settings clone brings `permissions` across, so the fork's tenants pass
Windmill's check, connect as the role they were given, and are denied by
Postgres on everything: an entry that reads as configured and answers nothing.

Refuse the copy — in the import endpoint before any data moves, and in the fork
path the CLI takes. Replaying the source's owners and ACLs into the clone is
what lifts this, and is a change of its own. Dropping `permissions` from the
copy instead would be the unsafe half, since the copy holds the parent's rows.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012ti5HyeTikPMYyW8YSdiHR
2026-09-08 21:29:51 +02:00
Diego ImbertandClaude Opus 5 31292ef9f2 fix(datatables): cascade on the leave route that is used, gate migrations before the admin connection, and drop a role atomically
The tenant cascade on leaving went onto `/users/leave`. The UI and the generated client call
`/workspaces/leave` — a different handler in a different crate with the same name — which
deleted the membership and left `u/<username>` in the tenant lists. Leaving and rejoining
therefore restored the access the leave was supposed to end, and a later account taking the
username would have inherited it. The regression test drives the route the client actually
calls; without the fix it fails with "leaving kept the tenant".

The migration endpoints authorized too late. `run_datatable_migrations` opened the data table's
admin connection, created `_wm_migrations` and read it before reaching the per-migration role
check — so with nothing pending, nothing was checked at all. Rollback returned before its check
when nothing was applied, and the status endpoint had none. All three now ask, before any
connection is opened, whether the caller can reach the data table as any role at all; which role
a given migration runs as is still decided per migration, and by the executor after that.

Deleting a role committed the cluster drop and the catalog row, then swept the tenant lists in
separate transactions. A sweep failing part-way left workspaces naming a role nothing can connect
as, while the retry answered `NotFound` because the catalog entry was already gone. The sweep now
runs in the same transaction, so the drop, the row and every tenant list commit together.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012ti5HyeTikPMYyW8YSdiHR
2026-09-08 21:04:18 +02:00
Diego Imbert 33aef570ab Merge remote-tracking branch 'origin/datatable-roles-redesign' into datatable-roles-redesign 2026-09-08 19:38:04 +02:00
Diego Imbert 0a7fa38926 Merge branch 'main' into datatable-roles-redesign 2026-09-08 18:02:13 +02:00
Ruben FiszelandClaude Opus 5 d23da87c16 chore(cli): restore module mocks so they stop leaking into later test files
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01SoXmLSb53SU8jfCMANugoY
2026-09-08 18:00:18 +02:00
Diego Imbert aaf234a4af Merge remote-tracking branch 'origin/datatable-roles-redesign' into datatable-roles-redesign 2026-09-08 17:53:17 +02:00
Ruben FiszelandClaude Opus 5 0b37226078 fix: chain redeploys onto a retired path's version history (#11029)
* fix: chain auto_parent onto the archived lineage tip at a path

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01GoTAmRTjG2KAH4g4mco6T5

* fix: check descendants unscoped and drop the hash-guard move

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01GoTAmRTjG2KAH4g4mco6T5

* fix: widen retired-path adoption, lock it, drop inherited grants

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01KMhCSFeWSezY61g6woQiz2

* fix: check lineage linearity unscoped, under the parent lock

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01GoTAmRTjG2KAH4g4mco6T5

* fix: do not name a lineage conflict the caller cannot read

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01GoTAmRTjG2KAH4g4mco6T5

---------

Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-08 17:44:22 +02:00
Diego ImbertandClaude Opus 5 67091eec69 fix(datatables): validate a rename against the save it describes, and re-check under the locks
Three from the round, all about deciding on state that could already have moved.

A permission save resolved the data table and checked it was instance-backed before taking any
lock, then wrote under one. A config save committing in between could move the table onto a
PostgreSQL resource — recreating exactly what the transition guard refuses — or rename it, in
which case the write targeted a key that no longer existed and reported success having changed
nothing. It now re-resolves and re-checks on the locked state.

Rename validation checked that the source existed before and the target existed after, which
still accepts `main -> decoy` against a save that keeps both: every fork of `main` then follows
onto a different data table, silently, because it keeps resolving. The rule is now the actual
old-to-new key transition — a source may only survive if another rename took its name, and a
target may only pre-exist if another rename freed it. That also stops two sources sharing one
target, and it admits a swap, which the previous guard refused: `datatables` is keyed by name, so
a swap cannot be done one save at a time, and refusing it was a regression against main. The
pointer cascade now runs in two passes through a temporary name, the way the migration cascade
one layer down already handles the same shape, so `A -> B` with `B -> C` moves each pointer once
from what it named before the save.

The tenant mutators say what they are for: they write an access decision for any workspace named,
with an arbitrary mutation, and exist for the transaction that frees or renames a principal.
Editing a decision on purpose belongs in the permissions endpoint.

Carried in the same change: the stranded-fork list is a field rather than a phrase to grep out of
a success string; the pointer cascade matches with `EXISTS` instead of a `LIKE` over the whole
document, so a workspace whose pointers name something else is not rewritten to a byte-identical
value under an exclusive lock; and `InstanceDatatableRole` drops the serde derives left over from
the JSON document, one of which would emit `pwd`.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012ti5HyeTikPMYyW8YSdiHR
2026-09-08 17:40:19 +02:00
Diego Imbert 2ef865c774 Merge branch 'main' into datatable-roles-redesign 2026-09-08 17:08:58 +02:00
hugocasaandClaude Opus 5 448fce93f7 fix: make the native trigger disable/enable toggle actually save (#11024)
* feat: let a native trigger be disabled without deleting it

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* fix: show and control the native trigger pause outside the flow editor

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* fix: create a native trigger already paused instead of pausing it after

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* docs: correct the native trigger enabled comments for create-time init

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-08 17:07:26 +02:00
Ruben FiszelandClaude Opus 5 785277e0bb feat(nativets): bound fetch on a peer that never answers (#11026)
* feat(nativets): bound fetch on a peer that never answers

deno_fetch applies no deadline of any kind. A peer that completes the TCP
handshake, accepts the request and then goes silent leaves `await fetch(...)`
pending indefinitely, holding its worker slot until the *job* timeout -- which
on self-hosted defaults to DEFAULT_SELFHOSTED_TIMEOUT, i.e. 7 days.

Nothing else catches this. Zombie-job detection keys off a stale
v2_job_runtime.ping, and a worker blocked inside a pending fetch keeps pinging
normally throughout: the worker is alive and healthy, only the work is dead.

What this bounds is the wait for a response to begin, and it stops there:

  - a peer that never answers            -> rejected after N seconds
  - a peer slow to answer, but under N   -> unaffected
  - a body that then streams for an hour,
    or is read slowly by the caller      -> unaffected, always

That last line rules out the obvious implementation: AbortSignal.timeout(N)
around every fetch would bound the hang and break every streaming response and
long download. This is a hang detector, not a latency budget.

Default 300s via WINDMILL_FETCH_RESPONSE_TIMEOUT_SECS (0 disables), with a
per-script `//fetch_response_timeout <seconds>` annotation alongside the
existing //useragent and //proxy. Both nativets paths inherit it, since
eval_fetch_timeout and the dedicated-worker path in bun_executor both funnel
through create_nativets_runtime.

The ms value is clamped to i32::MAX: deno_web's setTimeout runs its delay
through webidl.converters.long, a 32-bit conversion that *wraps*, so a setting
past ~24.8 days would come out negative and fire immediately -- turning an
over-generous timeout into an instant one on every fetch.

The window covers connect, TLS and request upload as well as server think
time, so a very slow large upload is bounded by it too; the error message says
so rather than claiming the connection went silent.

Not covered: a body that stalls midway. Reaching that needs the response's
InnerBody, which deno_fetch keeps module-private, and every way to wrap it
from outside changes observable Response semantics (locking, bodyUsed,
double-consume errors). Left for a follow-up in deno_fetch itself.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01JshNT5XVH78ZsTMvDWfFHb

* test(nativets): cover the instance-wide response-timeout env var

A typo in WINDMILL_FETCH_RESPONSE_TIMEOUT_SECS would compile, pass every
other test, and silently hand every operator the 300s default -- the same
class of silent-default failure the timeout itself exists to prevent. Its
own test binary, since a LazyLock resolves the value once per process.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01JshNT5XVH78ZsTMvDWfFHb

* fix(nativets): inherit the caller's RequestInit, and clear the long-poll ceiling

Two problems with the first cut, both found in review.

`{ ...init, signal }` copied only own enumerable properties, but RequestInit is
a WebIDL dictionary whose members deno_fetch reads with plain property gets
that walk the prototype chain. Anything inherited or non-enumerable was
dropped: `fetch(url, Object.create({method: "POST"}))` silently became a GET.
Worse, a non-object init went from a loud TypeError to a silent GET, because
spreading "POST" yields {0:"P",1:"O",...} -- a valid dictionary with ignored
keys. Now the init is inherited from rather than copied, and a non-dictionary
is handed straight back to deno_fetch for its own TypeError.

The 300s default also sat at half of TIMEOUT_WAIT_RESULT (600s), which
run_wait_result long-polls against with no response headers. A script running
another job synchronously for 300-600s would have timed out client-side while
the server was still legitimately holding the request open -- the long-poll
risk class, instantiated inside the product and reachable without writing a
raw fetch. Default raised to 900s, with the constraint recorded where someone
would break it.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01JshNT5XVH78ZsTMvDWfFHb

* fix(nativets): hand the caller's RequestInit to Request untouched

Carrying a WebIDL dictionary across by hand has no safe form, and both
previous attempts were wrong in opposite directions. Spreading a copy drops
inherited and non-enumerable members, and turns a non-object init from a loud
TypeError into a silent GET. Inheriting from it via Object.create fixes those
but makes the child the receiver, so an accessor on the original runs against
an object that lacks its private-field brand:

    Cannot read private member #body from an object whose class did not
    declare it

So don't carry it at all. fetch()'s own first act is `new Request(input,
init)`; doing that here hands the init to the same constructor, read exactly
as it would be without this wrapper, and our signal travels in an init we own.
`req.signal` is then deno's own resolution of init.signal over an input
Request's signal, which removes the hand-rolled version of that rule too.

The Request is built twice as a result, once here and once inside fetch. That
is cheap: cloneInnerRequest carries method, headers, redirect mode, clientRid
and blob entry, and a body is proxied rather than buffered -- a static body is
a shallow {body, consumed} copy sharing its bytes, a stream gets a one-chunk
pass-through.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01JshNT5XVH78ZsTMvDWfFHb

* fix(nativets): keep an aborted fetch settling in the same tick

deno_fetch keeps its outer fetch non-async on purpose: "WPT has a test that
aborted fetch is settled in the same tick. This means we cannot wrap the
promise if it is already settled" (26_fetch.js). An `async` wrapper adopts
that promise through another one, so a rejection that used to land before any
microtask queued after the call now lands after it.

Made the wrapper non-async, with an early return that hands deno's settled
rejection straight back for an already-aborted signal, and no timer armed
there since there is no response to wait for. Construction still has to reject
rather than throw, so it is caught and returned as a rejection, which is what
the `async` was buying.

The comment claiming this matched deno_fetch's own `async function fetch` was
wrong on two counts -- that function is not async, and the wrapper was not
matching it. Replaced with the constraint that actually holds.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01JshNT5XVH78ZsTMvDWfFHb

* fix(nativets): keep fetch's observable shape and reach intrinsics safely

Three ways the wrapper was distinguishable from the fetch it replaces, all
observable from a script sharing the isolate.

`.then` was an ordinary property lookup, so `Promise.prototype.then =
undefined` broke fetch after the request had already gone out. deno's own
modules reach intrinsics through primordials, and this file already captured
setTimeout, clearTimeout and Promise.reject for exactly that reason, so the
lookup was the odd one out. Now captured alongside them.

Declaring `init` without a default made `fetch.length` 2 where the standard
says 1. And the empty-call branch forwarded two explicit `undefined`s, so
deno's required-argument check saw two arguments and raised "Invalid URL:
'undefined'" instead of "1 argument required". Forwarding through
ReflectApply preserves the count.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01JshNT5XVH78ZsTMvDWfFHb

* docs: clarify fetch timeout restart requirements

* fix: capture native fetch abort helpers

---------

Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-08 16:46:12 +02:00
Diego ImbertandClaude Opus 5 bc78383c74 refactor(datatables): put the role catalog in its own table, not in global_settings
Five findings across three rounds were all the same choice. A set of live Postgres credentials
was living in `global_settings`, which has generic read, list, write, config-export and CLI
round-trip paths that know nothing about what they carry: the passwords reached the instance
config and its YAML editor, a full-row upsert of a neighbouring key erased the catalog,
`GET /settings/global/{key}` and the settings listing returned them raw, and this round the
redaction that fixed the last two turned `wmill instance push` into something that wipes every
password — a fix breaking the assumption the previous fix made. `POST /settings/global/datatable_roles`
could also empty it outside the lock.

The approved plan offered a table or `global_settings`, so this is the other option it already
allowed rather than a new design. `datatable_role` is a table: no generic settings path can read
it, list it, export it, write it or round-trip it, so none of the five needs a guard. The
redaction, the hidden/protected/agent-denylist entries and the JSON document all go with it.

One row per role also removes the read-modify-write the concurrency work was about: two
concurrent creates are two inserts, and the unique index on `name` is what settles a collision.
The advisory lock stays for the one window rows do not cover — `CREATE ROLE` is invisible to
another transaction until commit, so without it both creates pass their `pg_roles` check.

Also from this round: rename mappings are checked against the configuration they claim to
describe, since fork pointers are rewritten from them — a caller could otherwise submit
`main -> missing` against an unchanged config and repoint every fork of `main` at a name nothing
has, and `A -> B` plus `B -> C` moved what pointed at `A` all the way to `C`. And the warning
naming forks a delete stranded reached the response but not the screen: both the data table
settings save and the workspace delete now show it.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012ti5HyeTikPMYyW8YSdiHR
2026-09-08 16:29:51 +02:00
Diego ImbertandClaude Opus 5 30803e4e5a fix(datatables): close the last ways a role or a pointer can be left pointing at nothing
The raw settings readers hand back whatever is in the row, so moving the catalog into its own
`global_settings` key protected the config machinery and left `GET /settings/global/datatable_roles`
and the settings listing returning every live password. Both now filter that one key. The
neighbouring `custom_instance_replication_pwd` has the same shape and is not touched here: it
predates this and widening the fix to it is a decision about an operator workflow, not a
consequence of this change.

Three ways a save could leave something resolving to nothing:

A permissioned data table could be moved to a PostgreSQL resource. The block was carried across
as a server-owned field, the runtime refuses roles on a resource-backed table, so the save
succeeded and every job afterwards failed. Refused instead — turning roles off first is one step,
and it keeps discarding an access decision something somebody chose.

Renaming a governing data table left every fork pointing at the old name: the data table
disappears from their pickers and their jobs stop, with nothing in the renaming workspace to
suggest why. The rename now follows into the pointers in the same transaction.

Deleting one cannot be followed the same way, so it is reported instead — the response names what
it stranded, the way deleting a workspace does, and the fork's own error already says which
workspace is gone.

Also: `ensure_instance_db_grant_options_unchecked` claimed superadmin while the permissions
handler reaches it as a workspace admin (the same class fixed last commit, one instance missed);
the role entry kept an `instance_config_schema` derive it no longer needs; `write_role_catalog`
was the one writer of that table not stamping `updated_at`; and the concurrency test dropped its
roles only on success — a failing run is exactly the one that creates them without recording them.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012ti5HyeTikPMYyW8YSdiHR
2026-09-08 15:56:03 +02:00
Diego ImbertandClaude Opus 5 035897b9b9 fix(datatables): make the concurrency test pin the handlers, and the contracts describe what is enforced
The concurrency test reimplemented the read-modify-write inline, so deleting the lock from all
three handlers left it green — it pinned Postgres, not the code it was written for. It now
drives `create_datatable_role` twice concurrently and asserts the catalog kept both names.
Checked the way the last one should have been: removing the lock from the handler makes it
fail with "wmtest_a_… is a live cluster login the catalog forgot".

The contracts added last commit were stricter than this PR's own callers, which is worse than
none — the next reader sees a rule already broken and learns to ignore it.
`read_role_catalog` said superadmin-only while two of its four callers are open to any
workspace member, and `converge_connect_grants` said superadmin while
`set_datatable_permissions` reaches it as a workspace admin. Both were fine on substance: the
rule that actually holds is about the credential never reaching a response, log, audit record
or export, not about who may call. They now say that. `read_datatable_entry` gets the same
treatment rather than the one the earlier message claimed for it: it is the primitive every
resolution goes through, so it is deliberately open, and what must not escape is `permissions`
— it names the governing workspace's users, groups and folders.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012ti5HyeTikPMYyW8YSdiHR
2026-09-08 15:30:21 +02:00
Diego ImbertandClaude Opus 5 0c16d3a91c fix(datatables): give the role catalog its own row, out of reach of the config machinery
Putting it inside `custom_instance_pg_databases` was the wrong call, and it cost two ways.
The catalog serializes a generated Postgres password per role, and that row is the
operator-facing instance config, so the passwords reached `get_instance_config` and its YAML
editor — a live cluster credential in a response body, a UI field and any log of either.
Worse in the other direction: `to_settings_map` strips the catalog, so a full-row upsert of
that key writes the row back without it and the catalog is gone, while the cluster keeps every
login it described.

`custom_instance_replication_pwd` is the precedent and says exactly why — a generated secret,
written only by the server, never operator-authored, hidden so the config machinery cannot
read, rewrite or drop it. The catalog is the same thing, so it now has the same shape:
`datatable_roles`, in `HIDDEN_SETTINGS`, `PROTECTED_SETTINGS` and the agent-worker denylist.
No redaction to keep in step with three code paths, and no way for a neighbouring write to
take it out.

Two races on the same shared documents. `edit_datatable_config` read the stored data tables
outside its transaction and then wrote the whole `datatable` document, so a permissions save
committing in between was silently rolled back; it now reads under `FOR UPDATE`. And
`set_datatable_permissions` validated role ids against the catalog before opening its
transaction, so a deletion in between let it write a deleted role back — including as the
default, which every later job then fails on; it now holds the catalog lock and the settings
row across validation and write.

Completes the authorization contracts the previous commit claimed but did not finish:
`read_datatable_entry` (which it named and missed), `resolve_governing_datatable`, whose whole
job is to answer for a workspace the caller may not belong to, and
`converge_connect_grants_with`, which had not inherited its wrapper's.

Also the generic Python SDK reference: `_format_py_params` learned the bare `*` last time, but
`extract_py_functions` is a second formatter and still rendered `datatable(name, role)`, so
code written from that page passed a keyword-only argument positionally.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012ti5HyeTikPMYyW8YSdiHR
2026-09-08 15:26:24 +02:00
Ruben Fiszel 946756ae83 perf: reduce shared worker debug polling frames (#11028)
* fix: reduce php parser stack use in debug workers

* test: document php stack regression expression depth

* fix: offload php signature parsing from async workers

* fix: address php parser review nits

* perf: reduce shared worker debug polling frames

* test: refresh agent volume fixtures
2026-09-08 15:21:10 +02:00
Ruben Fiszel 2cb02e3b33 fix: offload php signature parsing from async workers (#11027)
* fix: reduce php parser stack use in debug workers

* test: document php stack regression expression depth

* fix: offload php signature parsing from async workers

* fix: address php parser review nits
2026-09-08 15:20:02 +02:00
Diego ImbertandClaude Opus 5 b2479e87fa fix(datatables): serialize role catalog mutations, and state each helper's authorization contract
The catalog is one JSON document, so create, rename, enable and delete are all
read-modify-write. Two concurrent creates read the same snapshot, both succeed in the
cluster, and the second write drops the first — leaving a live Postgres login with a password
nobody recorded, which is the exact state the delete path exists to prevent. Every mutation
now runs in one transaction holding an advisory lock across the read, the cluster DDL and the
write, so a lost update cannot happen and a failure rolls the whole thing back. The DDL
helpers take that transaction rather than the pool, which is what makes the lock cover them.

Their statements moved off `sqlx::raw_sql`: the simple protocol is only needed for genuinely
multi-statement SQL, and its future is not `Send`, which an axum handler holding the
transaction requires. Each of these is one statement anyway.

The new cross-crate surface now says what callers must do. `read_role_catalog` returns
plaintext credentials; `create`/`rename`/`set_login`/`drop_instance_role` and
`converge_connect_grants` mutate cluster-wide state; `read_datatable_entry` reads a workspace's
raw config. All of them are superadmin-gated by their current handlers, but nothing said so at
the definition, which is where the next caller looks.

Also: the roles table reloads after a failed login toggle instead of leaving it claiming a flip
that did not land; the rename affordance is the design-system `Button`, not a raw one; and
`resolve_datatable_pg_as_caller` drops a `role` parameter no caller ever filled — browsing
resolves as the data table's default until the database manager grows a picker.

Why role passwords stay a plain `String` while the instance user's password beside them is a
`StringOrSecretRef`, asked three times across reviews: that one is a secret ref because an
operator supplies it and may want it from their own backend, while these are minted here and
never entered by anyone, so there is nothing for a ref to point at. Encrypting generated
secrets at rest is a separate change that would take the replication password with it. Now
said at the field.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012ti5HyeTikPMYyW8YSdiHR
2026-09-08 14:44:31 +02:00
Ruben Fiszel 2ae8509b14 fix: reduce php parser stack use in debug workers (#11025)
* fix: reduce php parser stack use in debug workers

* test: document php stack regression expression depth
2026-09-08 14:33:47 +02:00
Ruben FiszelandClaude Opus 5 de98adf055 feat: let a worker group override the dependency cache object store (#11019)
* feat: let a worker group override the dependency cache object store

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01R6hBAWqNUsQMAJ7P59juug

* fix: address review findings on the worker-group cache override

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01R6hBAWqNUsQMAJ7P59juug

* fix: close the remaining config read route and re-evaluate the override on plan change

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01R6hBAWqNUsQMAJ7P59juug

* fix: serialize override reloads and keep a store a failed rebuild still serves

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01R6hBAWqNUsQMAJ7P59juug

* fix: require enterprise for the cache override and lock its whole transition

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01R6hBAWqNUsQMAJ7P59juug

---------

Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-08 14:10:51 +02:00
Diego Imbert 0f557ad408 Merge remote-tracking branch 'origin/main' into datatable-roles-redesign
# Conflicts:
#	backend/ee-repo-ref.txt
2026-09-08 14:00:07 +02:00
Diego ImbertandClaude Opus 5 f0ae50becf fix(datatables): unbreak two operator messages and two comments that described other code
The two strings this branch added for states an operator hits once — the catalog write that
matched nothing, and the delete that stranded a pointer — were collapsed from their multi-line
form with the indentation left in, so both rendered with a fourteen-space gap mid-sentence.

`list_datatables` claimed to report a chain it cannot follow and then dropped it; it does drop
it, and the comment now says why that is the right place to stay quiet. The non-superadmin
check in `edit_datatable_config` was introduced as also covering references, which it does not
and need not: `reference` is overwritten from the stored entry for every caller before the
check runs.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012ti5HyeTikPMYyW8YSdiHR
2026-09-08 13:59:03 +02:00
Diego ImbertandClaude Opus 5 8421579133 fix(datatables): fail loudly where a role or a pointer can be left half-recorded
Three ways the feature could end up in a state nobody could see or undo.

Creating a role writes the cluster first and the catalog second, but the catalog write was an
`UPDATE` that matched nothing when the instance Postgres settings row was absent — leaving a
live login with a password nobody recorded: invisible to the catalog, un-recreatable because
the name is taken, and un-deletable because there is no entry to delete. It now errors, so
the operation is retryable once the row is restored.

Deleting a workspace only nulls the fork lineage; the data table entries pointing at it are
left resolving to nothing. Sweeping them is not an option — turning a pointer back into a copy
would hand each fork the database outright — so the delete now names the data tables it
stranded, and resolving one says which workspace is missing rather than reporting a data table
this workspace never had.

`InstanceDatatableRole` derived `Debug` while holding a Postgres password; it is now
hand-written so `{:?}` on the catalog cannot put a live credential in a log line.

Adds the two branches the reviews found unpinned: a caller who is not a member of the
governing workspace at all, and `NoIdentity` — the compatibility path for an agent worker that
predates this and sends no job id.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012ti5HyeTikPMYyW8YSdiHR
2026-09-08 13:46:54 +02:00
Ruben FiszelandClaude Opus 5 f081fb1070 feat: recognize // volume: mounts in PHP scripts (#11018)
* feat: recognize `// volume:` mounts in PHP scripts

Volume annotations were parsed for every language but PHP, so a PHP script
could not mount a workspace volume. Two things stood in the way: PHP had no
entry in the comment-prefix maps, and a PHP script opens with `<?php`, which
ends the leading comment block the parsers scan before any annotation is read.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01T3FR7iS9nRhpFt615cnuQ7

* fix: tolerate a PHP opener that carries code, drop the inert CLI hunk

The open-tag skip matched `<?php` exactly, so `<?php declare(strict_types=1);`
still ended the leading comment block and every annotation below it was silently
ignored. Match the tag as a case-insensitive prefix and skip the whole line.

The CLI local-graph hunk could never fire: PHP has no wasm asset parser, so
`fallbackParse` handles it, and its own header scan stops at `<?php` — the script
is dropped as a non-pipeline-member before any volume asset is read. Making only
the CLI PHP-aware would also put the local graph out of parity with the deployed
one, whose `parse_pipeline_annotations` stops there too.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01T3FR7iS9nRhpFt615cnuQ7

* docs: correct the CLI mirror comment, state the own-line annotation rule

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01T3FR7iS9nRhpFt615cnuQ7

---------

Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-08 13:23:35 +02:00
Diego ImbertandClaude Opus 5 6a97f67713 fix(datatables): confine roles to the instance database, and stop a fork reaching the parent's bookkeeping
A data table role is a login on Windmill's own Postgres. Nothing stopped a workspace admin
putting a *resource-backed* data table under roles, at which point the executor dialled the
host that resource names — one the admin chose — with the role's real cluster password, and
`CONNECT` is granted to every registered instance database. Both ends now refuse: the
permissions endpoint rejects the save, and the chokepoint refuses to substitute credentials
on a non-instance entry rather than trusting the record it read.

Two more places reached the governing database without answering to it. The initial-migration
generator returned a `pg_dump` of the whole schema to any member. And the migration
rename/delete cascade followed a fork's pointer into the parent, so a fork admin renaming or
removing their own local entry relabelled or wiped the parent's `_wm_migrations` — after
which the parent re-runs every migration from zero. The remote half is now skipped when the
entry resolves into another workspace, which is also just correct: a fork renaming what it
calls a data table changes nothing about the data table.

Also: revoking a tenant now bounces the replication streams of every workspace holding an
entry that resolves here, not only the governing one, so a fork's trigger stops rather than
living on inside its open connection; the instance role catalog and the governing workspace's
tenant lists are no longer returned to someone who cannot edit them; and the tenant rename
dedup collapses non-adjacent duplicates, per role rather than once any role changed.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012ti5HyeTikPMYyW8YSdiHR
2026-09-08 13:22:20 +02:00
Diego ImbertandClaude Opus 5 0ec1cb2bdd fix(datatables): gate the paths that reach a whole database as admin
Auditing what still resolved through the unchecked resolver turned up three that act for a
caller and hand back the admin connection: `resolve_pg_source_checked` (behind schema
export, the full-schema read, database creation, import and the forked-database drop), the
connection test, and the schema snapshot a fork clone takes of its parent. On a data table
under roles each let any workspace member — or a fork admin who is nobody in the governing
workspace — read or copy the whole database whatever its roles grant.

All three now require admin reach on the governing workspace. A dump taken under a
restricted role would be a silently truncated copy rather than an error, so refusing is the
only right answer for the copy paths.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012ti5HyeTikPMYyW8YSdiHR
2026-09-08 13:08:34 +02:00
Diego ImbertandClaude Opus 5 c5446a7a26 feat(datatables): put a data table's connection under Postgres roles
A data table backed by the instance database resolved to exactly one Postgres connection,
`custom_instance_user`, for everyone who could reach it at all. There was no way to say
this job reads, that one writes, this one never sees the salaries table.

A data table role is now a real Postgres login on the cluster, defined once for the
instance by a superadmin and named exactly as they named it. A script that declares
`-- role analytics` connects as `analytics`, and Postgres decides what it may touch —
grants are ordinary SQL. Windmill answers only "may this caller ask for this role", from
the tenant lists on the data table entry: `u/alice`, `g/analysts`, `f/finance` or `*`.
A data table with no `permissions` block behaves exactly as before.

Everything that opens a connection on someone's behalf goes through one chokepoint,
`get_datatable_resource_from_db`, which takes the identity explicitly and fails closed when
there is none. The role logs in as itself — never `SET ROLE`, which a script could
`RESET ROLE` its way out of.

A fork's data table entry becomes a pointer at the workspace that governs it rather than a
copy of it. The settings clone used to hand a fork a byte-identical entry naming the
parent's database, which a fork admin could edit to grant themselves `admin` there; a
pointer has nothing local to edit, and its tenants are evaluated as a member of the
governing workspace, by email. `permissions` is stripped from the workspace export and
ignored on import: tenants name principals of one workspace, and a settings push is not
where an access decision should be made.

Operations that see the whole database whatever the roles grant stay with the governing
workspace's admins: editing the roles, a migration that declares none, and opening a
replication stream for a Postgres trigger or capture.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012ti5HyeTikPMYyW8YSdiHR
2026-09-08 13:04:32 +02:00
Ruben FiszelandClaude Opus 5 3e3a41d418 feat: report a WAC task failure the workflow body never awaited (#11017)
* feat: warn when a WAC task fails and the body never awaited it

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Q6FHYvk7z4eGvhZXKZB9JF

* fix: report unawaited WAC failures on the failing round and in stream order

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Q6FHYvk7z4eGvhZXKZB9JF

* docs: state the WAC warn-placement invariant where the wrapper enforces it

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Q6FHYvk7z4eGvhZXKZB9JF

---------

Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-08 12:19:26 +02:00
9444049d60 feat: bring gitlab repositories to parity for git sync (#10938)
* feat: track and rotate gitlab git-sync repository tokens

* chore: point ee-repo-ref at the gitlab credential branch

* fix: strip server-owned credential status and correct expiry copy

* fix: gate credential maintenance on enterprise and alert on stalled renewal

* fix: alert on an auto-renewed token only once it has actually expired

* feat: receive gitlab push webhooks for instant git sync pull

* feat: open gitlab merge requests and post diff previews on them

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01C1xHmkxuxYb1GYvth1BS75

* fix: keep gitlab merge request previews out of the project's own pipeline

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01C1xHmkxuxYb1GYvth1BS75

* fix: bound the credential maintenance pass and gate the gitlab picker on a license

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01C1xHmkxuxYb1GYvth1BS75

* fix: create the gitlab picker's variable in the edited workspace

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01C1xHmkxuxYb1GYvth1BS75

* fix: make the gitlab picker's variable path collision-resistant

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01C1xHmkxuxYb1GYvth1BS75

* docs: state the gitlab scope and rotation facts the code relies on

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01C1xHmkxuxYb1GYvth1BS75

* fix: resolve the check marker's repository from its path, not a stored url

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01C1xHmkxuxYb1GYvth1BS75

* fix: refuse to finish a check whose repository has been repointed

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01C1xHmkxuxYb1GYvth1BS75

* fix: trust a check marker's captured url when it carries no identity

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01C1xHmkxuxYb1GYvth1BS75

* fix: recreate a missing webhook from credential maintenance

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01C1xHmkxuxYb1GYvth1BS75

* docs: state that relative-url gitlab installs are out of scope

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01C1xHmkxuxYb1GYvth1BS75

* fix: keep credential status out of exports and clear stale webhook warnings

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01C1xHmkxuxYb1GYvth1BS75

* fix: refuse an unprovable check and guard the picker on the stored repository

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01C1xHmkxuxYb1GYvth1BS75

* fix: re-check the picker's target path at the moment it is written

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01C1xHmkxuxYb1GYvth1BS75

* fix: snapshot the picker's inputs before it starts writing

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01C1xHmkxuxYb1GYvth1BS75

* docs: recommend a project access token per repository

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01C1xHmkxuxYb1GYvth1BS75

* [ee] feat: keep the git-sync credential in workspace settings

* [ee] fix: drop a removed repo's credential and honor the workspace override

* [ee] fix: resolve a fork's git credential from its whole ancestry

* [ee] refactor: reuse fork_ancestor_chain instead of a second ancestry walk

* [ee] fix: resolve an app installation from the whole ancestry, not the parent

* [ee] revert: keep the app installation fallback at one level

* fix: store the git credential only once the resource is saved

* fix: keep a repository's credential when it leaves git sync settings

* docs: cut the gitlab picker's token guidance down to what it needs

* feat: mark a repository whose credential windmill holds

* fix: ignore the managed-credential marker when the url carries a token

* docs: drop the picker's setup alert for a line by the token field

* feat: replace a repository's stored token from its resource

* fix: store a picked credential for its own workspace, before the resource

* refactor: key a stored git credential by its repository, not its resource

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01C1xHmkxuxYb1GYvth1BS75

* chore: refresh the sqlx cache for the repository-keyed credential queries

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01C1xHmkxuxYb1GYvth1BS75

* fix: gate the credential pass budget on the features that use it

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01C1xHmkxuxYb1GYvth1BS75

* fix: decide credential rotation ownership by repository, not resource path

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01C1xHmkxuxYb1GYvth1BS75

* refactor: renew only the credentials windmill holds, not tokens in a repo url

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01C1xHmkxuxYb1GYvth1BS75

* fix: authenticate the fork-branch poll and correct the renewal guidance

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01C1xHmkxuxYb1GYvth1BS75

* fix: do not claim a managed credential for a url the client cannot resolve

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01C1xHmkxuxYb1GYvth1BS75

* fix: define the credential facade for private builds without enterprise

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01C1xHmkxuxYb1GYvth1BS75

* fix: pin the listed token before the await and name the real renewal blocker

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01C1xHmkxuxYb1GYvth1BS75

* fix: pin the token the replace flow checked, and derive the scope test once

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01C1xHmkxuxYb1GYvth1BS75

* fix: classify the renewal state once so the card cannot contradict itself

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01C1xHmkxuxYb1GYvth1BS75

* refactor: ask only whether the token gets renewed, not why it does not

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01C1xHmkxuxYb1GYvth1BS75

* refactor: replace the managed-credential marker with a server answer

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01C1xHmkxuxYb1GYvth1BS75

* fix: read renewal from the credential and its origin, not a removed field

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01C1xHmkxuxYb1GYvth1BS75

* fix: read the provider for url-token repos, await the origin before defaulting, and visit unchecked repos last

The maintenance pass sorted repositories with no recorded check first on
the premise that they cost nothing, but a token-in-URL remote on a host
that is not GitLab is probed every pass and never records a check, so it
held the head of the list ahead of the tokens that expire. Such
repositories now sort last.

The card decided its delivery defaults before the origin lookup landed,
so a freshly picked GitLab repository never got webhook delivery; the two
lookups are awaited together. The resource editor offers to replace a
token only where it is held, not in a fork that borrows it, and the
replace flow refuses a URL it cannot parse instead of keying the token to
it. Attaching a stored credential to a commit-hash probe now requires
admin, matching the installation credential beside it.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01C1xHmkxuxYb1GYvth1BS75

* docs: describe the gitlab listing token the way the picker and the setup guide do

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01C1xHmkxuxYb1GYvth1BS75

* refactor: a token in the repository url is a plain remote, not a tracked credential

Drops the status fingerprint that told one URL token from another, the
docs' promise that such a token's expiry is reported, and the test's
expectation that a URL-token repository declares a host.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01C1xHmkxuxYb1GYvth1BS75

* fix: the card reads the credential origin for managed controls and honours the licence for a borrowed token

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01C1xHmkxuxYb1GYvth1BS75

* chore: bump the ee ref

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01C1xHmkxuxYb1GYvth1BS75

* fix: hide a repository's credential line once nothing is held for the repository it names

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01C1xHmkxuxYb1GYvth1BS75

* docs: describe the exported credential status as it is

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01C1xHmkxuxYb1GYvth1BS75

* refactor: run the credential maintenance pass as its own task, without a budget

The pass ran inside the monitor's join, whose deadline cancels every
future in it, and a rotation cancelled between GitLab issuing a token and
Windmill storing it loses the token family. A wall-clock budget with a
least-recently-checked ordering kept it under the deadline. Spawning the
pass instead makes the deadline irrelevant, so the budget, the ordering
and the counter go; the advisory lock keeps a slow pass from overlapping
the next, as it already did.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01C1xHmkxuxYb1GYvth1BS75

* docs: say what detaching the maintenance pass buys, and what it does not

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01C1xHmkxuxYb1GYvth1BS75

* chore: run git sync on the hub script version that reads a stored credential

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01C1xHmkxuxYb1GYvth1BS75

* chore: run the deploy push and the connection test on the hub versions that read a stored credential

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01C1xHmkxuxYb1GYvth1BS75

* fix: keep App repositories and plain remotes out of the stored-credential paths

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01C1xHmkxuxYb1GYvth1BS75

* chore: host-neutral deploy preview wording, drop the project filter from the GitLab picker

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01C1xHmkxuxYb1GYvth1BS75

* chore: bump ee ref, rotation no longer retains a second connection per repository

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01C1xHmkxuxYb1GYvth1BS75

* chore: bump ee ref, the rotation write-back holds a single connection

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01C1xHmkxuxYb1GYvth1BS75

* chore: hold the credential maintenance lock in a transaction so a dead sweep releases it

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01C1xHmkxuxYb1GYvth1BS75

* docs: describe the credential-stored callback as it fires

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01C1xHmkxuxYb1GYvth1BS75

* fix: keep the credential maintenance lock past the pool's idle-in-transaction timeout

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01C1xHmkxuxYb1GYvth1BS75

* chore: update ee-repo-ref to e092518ee60e33160fee9ae91a4d109566f7b0ee

This commit updates the EE repository reference after PR #771 was merged in windmill-ee-private.

Previous ee-repo-ref: 74481f7cc345757aebb2a8b04d3a22978328c348

New ee-repo-ref: e092518ee60e33160fee9ae91a4d109566f7b0ee

Automated by sync-ee-ref workflow.

---------

Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
Co-authored-by: windmill-internal-app[bot] <windmill-internal-app[bot]@users.noreply.github.com>
2026-09-08 11:54:43 +02:00
Ruben FiszelandClaude Opus 5 6860521b4e refresh a dbt column trace with its graph, and stop calling whole ones cut (#11015)
* fix: refresh a dbt column trace with its graph, and stop calling whole ones cut

Follow-up to #11014, addressing two findings from the review round that landed
after it merged.

**A deliberate graph refresh now re-asks for the lineage.** The dedup key held
only the workspace, the pin and the seed relations, all of which a redeploy
leaves alone — so the pipeline page's Refresh refetched the graph and left the
trace as it was, pairing the new version's SQL and columns with the previous
one's edges. The key now carries which fetch of the graph is on screen, taken
from `graphRes.current`'s identity: it moves on a Refresh, a deploy and a folder
switch, and on nothing else, so an editor keystroke still cannot make the pane
re-ask.

**`truncated` is set only with evidence.** `pending` was read as proof the
component had been cut, but it only says a relation's owners have not been asked
about yet — and those owners are usually the project already in hand. A project
holding more than the expansion budget across unrelated families therefore
reported a small, complete component as truncated. The owners query now runs
before the budget and round stops, so a trace is called cut only when a project
this caller may read is left unread, or when the walk itself was cut.

Two smaller things from the same round: a failed lineage request says so instead
of rendering the empty trace a project without the analysis pass renders — the
two were indistinguishable, and a Refresh now retries it — and `asset_paths` is
capped as well as refused when empty. `MAX_HELD_EDGES` is renamed
`EXPANSION_EDGE_BUDGET`: it never bounded what its name claimed, since the seeds'
own projects are read whole whatever their size.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01NY4kuFy2jAnGEzaCc1CseL

* fix: show a failed column-lineage request beside a partial trace, not only instead of one

Review-round findings on this PR.

The failure line only rendered when the trace had no nodes at all. A ducklake
selection whose producers declare column lineage has nodes from the graph the
canvas already carries, so a failed dbt request left a trace that rendered and
was missing a half — which is the reading the line exists to prevent. It now
renders beside a drawn trace as well, and says the trace may be incomplete
rather than that nothing loaded.

The dbt branch of the details pane also opened on `selectionColumnLoading` but
not on the failed state, so a relation with neither SQL nor a column schema fell
through to "no inline preview" and the line never rendered at all.

Dropped "Refresh to try again": the dbt editor has no Refresh for this, and its
recovery is a re-parse or reselecting. The comment on the error handler says both
paths again rather than only the one the pipeline page uses.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01NY4kuFy2jAnGEzaCc1CseL

* fix: clear a column-lineage failure when the next request goes out

Round-2 nits, both reviewers on the same state.

`failed` was cleared only when an answer landed, so a retry kept saying the trace
may be incomplete while it was being fetched, and a new selection inherited the
previous one's failure until its own answer arrived. It is cleared as the request
goes out instead.

Also documents the bounds on `asset_path` in the two routes that take it: the
1000-relation cap and the at-least-one rule were both enforced and neither was
written down, so a caller met them as a 400 with no way to have known.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01NY4kuFy2jAnGEzaCc1CseL

---------

Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-08 11:41:05 +02:00
Ruben FiszelandClaude Opus 5 d3f305db98 feat: retry a workflow-as-code task from its task options (#11013)
* feat: retry a workflow-as-code task from its task options

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WvjgKMRNtRNPnAkg6MkiTA

* fix: claim every retry attempt key up front, so a step cannot alias one

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WvjgKMRNtRNPnAkg6MkiTA

* fix: bound retry attempts, which now claim their keys up front

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WvjgKMRNtRNPnAkg6MkiTA

* fix: honour an explicit zero retry multiplier in the python client

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WvjgKMRNtRNPnAkg6MkiTA

* docs: state the retry validation rules once in the task docstring

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WvjgKMRNtRNPnAkg6MkiTA

---------

Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-08 11:37:56 +02:00
Ruben FiszelandClaude Opus 5 33f9828c3e feat: draw a dbt column trace, across projects and the pipeline boundary (#11014)
Serves the edges `dbt_column_edge` has been storing. `assets/column_lineage`
answers the connected component a set of relations' columns sit in, and the
details pane draws it beside the model's SQL — in the dbt editor, on the
pipeline page, and for a run through `jobs/dbt_column_lineage/{id}`.

The unpinned component crosses projects. A relation one project produces is
another's source, so resolving owners once — for the relations asked about —
stops the trace at the first boundary. Owners are resolved to a fixpoint
instead, and the caller's gate is re-applied to every project the expansion
discovers: reaching a relation says nothing about who may read the project on
the far side of it. A pinned answer needs none of it, by version or by job: the
pin says which stored graph is on screen, and another project's live graph is
not part of it.

One request per selection, whatever it reaches: the endpoint takes every
relation at once and answers their union, so nothing is held between selections
and there is no staleness, retry bookkeeping or per-click dedup to balance.

The answer is bounded. A synthetic 3000-model project whose models share a
column has 58k direct edges and returns 7.3MB, which no column diagram can draw;
the walk is breadth-first from the asked-for relations and stops at 5000 edges,
so what survives is the part nearest the selection, and `truncated` says the
trace was cut rather than ended.

Also adds the columns section the pipeline page's asset pane was missing, so
`column_schema` is visible there and not only in the dbt editor.


Claude-Session: https://claude.ai/code/session_01NY4kuFy2jAnGEzaCc1CseL

Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-08 08:29:36 +00:00
Ruben FiszelandClaude Opus 5 0139467b01 feat: ingest dbt column lineage and real column schemas from the engine's parquet index (#10977)
* feat: column-level lineage for dbt from the engine's parquet index

`manifest.json` carries no column-to-column edges, which is why decision 14
recorded column lineage as unavailable. The edges live in a different artifact:
`dbt compile --static-analysis strict --write-index` writes `target/index/`,
whose `dbt.column_lineage.parquet` holds them and whose
`dbt.node_columns.parquet` holds every column of every node, typed and ordered
rather than only the ones an author documented.

Strict analysis rejects SQL the default accepts, so this is a separate compile
with its own `--target-path`, opt-in per project via `column_lineage: true`, and
best-effort throughout: a project it cannot analyze keeps exactly the graph it
had, with the engine's own diagnostics in the job log.

Storage mirrors `dbt_edge`: `dbt_column_edge` keyed by (path, version, job) with
the same composite FK to `script` and the same sweeps. The typed column list
lands in `dbt_node.column_schema`, beside `columns` rather than merged into it,
so `columns` stays what the author declared.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PRtsPQ3Ck69Fu9DNr7bMcJ

* fix: address review findings on the dbt column-lineage pass

- The workspace fork copied every other dbt sidecar table and not this one, so
  a fork lost its column lineage silently and could not recover it: the cloned
  digest covers the column edges, so a dynamic run in the fork matched it and
  stored nothing.
- The parquet was collected whole before the edge cap applied, which is exactly
  the input the cap exists for — a project whose `scan` lineage is quadratic in
  its widest model could take the worker process down. Decoded a row at a time
  with the bound enforced during the decode.
- The pass swallowed every error from the runner, including the job poller's
  cancellation and deadline, so a run that blew its timeout inside an optional
  annotation could still publish a graph and report success. `run_captured`
  now carries the exit status in its value, so only a failed COMPILE is
  downgraded, and the pass may spend at most half the remaining wall clock so
  it cannot starve the build that follows it.
- `scan` edges are stored but no longer served: they are most of a project's
  lineage, nothing renders them, and the graph endpoint is polled by the run
  page. They are also the first thing the storage cap gives up now, rather than
  evicting the direct edges the trace draws.
- `column_schema` and the column edges take the same gate as the model's SQL. A
  column-level view is the shape of what the author wrote, one level finer than
  the `ref()` graph, which is ungated only because it draws relations the
  caller already sees.
- `graph_digest` hashes the new section only when it has edges, so a project
  that never asked for the pass keeps the digest it has instead of
  re-snapshotting on every dynamic run until it is redeployed.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PRtsPQ3Ck69Fu9DNr7bMcJ

* fix: the editor buffer's column lineage, and three bounds that were wrong

Round-2 review found four defects, all of them introduced by the round-1 fixes.

- The `script_visible` gate on the column edges was copied from the node query
  without its `script_hash IS NULL` arm. `= NULL` is never true, so every
  version-less row was filtered out and an editor buffer's parse rendered its
  typed columns and none of their lineage — the one place the feature is meant
  to be used. Pinned by an assertion in `dbt_pinned_graph.rs`, which is where
  this class of bug already had a home.
- The phase budget was handed to the poller, whose expiry is an `Err`
  indistinguishable from a cancellation or the job's own deadline, so a slow
  but valid analysis aborted the build it exists to annotate. The runner gets
  the full deadline again — those two must still fail the job — and the budget
  is a race around the whole pass, where expiring is this budget and nothing
  else.
- The decode cap counted parquet ROWS, so `scan` and out-of-graph rows could
  spend it before a single drawn edge was read. It now counts what is kept,
  takes direct kinds in a first pass, and is handed the graph's own nodes so
  the budget cannot go on rows that could never be stored.
- Hashing the new digest section conditionally did not preserve old digests,
  because an absent `column_schema` still serialized as `null` inside the
  nodes. It is skipped when absent instead.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PRtsPQ3Ck69Fu9DNr7bMcJ

* refactor: split the lineage pass by error contract, and read it in one query

Round 3's findings were all consequences of round 1 and 2's fixes, clustered in
the same two files, so this reshapes those two seams rather than patching again.

The worker pass was one function being three things at once — a subprocess
runner with job-lifecycle error semantics, a bounded decoder, and a best-effort
degrader — which is why each fix to one perturbed another. It is now
`compile_index`, which owns the JOB's semantics (only a cancellation or the
job's deadline can `Err`; a non-zero exit, the output ceiling and the phase
budget are outcomes), and `read_index`, which owns the ARTIFACT's and knows
nothing about the job. The budget wraps the compile alone, so a decode can no
longer outlive the timeout that reported the build would get the rest. The
output ceiling likewise becomes a value rather than a job error, for the caller
that can carry on without the tail of a compile's stdout.

The column edges were read by a fourth hand-written copy of the `live`/`chosen`
CTEs and the version/editor-buffer join conditions, and copying them is what
dropped the `script_hash IS NULL` arm and hid every buffer parse's lineage. Both
kinds of edge now come from ONE statement over a `UNION ALL`'d edge source, so
those conditions exist once. The union is at the source rather than a join
because column lineage can name a node pair `dbt_edge` has no row for: a model
reading `{{ this }}` gets edges from itself to itself, and `parent_map` has no
self-loop.

The cap on the column half now sits after the scope filter, the visibility
check and the graph joins — the scope moved into SQL via the existing
`ScopePathFilter` — so a row the caller may not read can no longer spend it and
leave an allowed project's trace short.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PRtsPQ3Ck69Fu9DNr7bMcJ

* refactor: serve dbt column lineage from its own endpoint

The column edges rode on the folder-wide asset graph, which a run page polls,
while the trace is drawn for one selected relation. That needed a cap, and a cap
has to be applied after every filter that can drop a row.

Keyed to the asset there is no cap: `assets/column_lineage` answers for one
relation, and the caller's `scripts:read` scope and the project's visibility are
decided once, for the script that owns it. Pinning to a run's snapshot or the
editor's parse of its buffer costs the job-read gate, so that form is
`jobs/dbt_column_lineage/{id}` — the same shape `jobs/dbt_graph/{id}` has.

The worker's decode now bounds work and memory separately, and a compile stopped
by the output ceiling reports as truncated rather than complete.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* fix: resolve the owning dbt version the way the graph does

The unpinned arm picked the newest live version at the path without narrowing to
dbt, so a path since redeployed in another language answered with no lineage
while the graph beside it still drew that project's stale nodes.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* fix: gate pinned column lineage on reading the project, and answer the component

Four things round 5 found, three of them in code this branch rewrote:

- The pinned arm resolved the version from the job and stopped there, so a
  share-link viewer entitled to a run got the project's column names and edges
  while the graph beside it still redacted `raw_code` and `column_schema`.
  Resolving WHICH version answers is not deciding whether the caller may read
  it; the version-less editor buffer keeps its exemption, having no `script` row
  to ask.
- The answer was the whole owning project's edges. The canvas lays out the
  connected component of the selected relation's columns, so the rest was
  unrenderable weight; a recursive walk over both directions returns exactly
  what is drawn, and the project key travels with it so a `unique_id` two
  projects share cannot walk from one graph into the other.
- The decode had no exit but the 4M-row backstop once its buckets were full,
  spending wall clock the build below does not get.
- An unreadable index was reported as a missing one, sending the reader to look
  at their engine rather than at the file.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* fix: stitch the two column graphs, and walk the component in Rust

Round 6's two findings, both regressions this branch introduced:

- The decode returned `Continue` on the edge that FILLED the direct-edge
  budget, so a `scan`-only tail after it decoded to the 4M-row backstop with
  nowhere to put anything. The read now ends on that edge.
- Seam 3 made the pipeline page choose between the dbt graph and the producer
  one. They share node ids — `// column total <- dbt://wh/analytics/orders.amount`
  mints the same `(dbt, path, column)` node dbt's own lineage does — so choosing
  ended a trace at the boundary in both directions. They are merged again, and
  a ducklake selection asks about the dbt relation its producers name so the
  chain continues past it. The dbt editor gets the same merge.

Also: the component is walked in Rust rather than by a recursive CTE. A CTE has
no index, so the recursive term rescanned the doubled edge set once per level —
1243ms against 59ms for the query alone on a 3000-model project, 11.7M rows in
the plan. Same answers, same tests; end to end 1.48s to 0.73s there and 1.60s to
0.26s on a 1000-deep chain. The client stops re-asking for a component it
already holds, which is most clicks within one project.

The four doc sites that described a whole-project answer are rewritten around
what it now is, rather than edited where they disagreed.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* fix: expand every dbt boundary a selection reaches, and only skip what was asked

Round 7's findings, all in the frontend seam this branch added:

- A ducklake selection seeded the dbt fetch from the FIRST boundary relation it
  found, so a table derived from two unconnected dbt relations expanded one and
  left the other a leaf — the same "stops at the boundary" symptom the round-6
  fix removed, one hop further along. Every distinct boundary is fetched now and
  the components merged.
- The component cache skipped a relation merely PRESENT in the graph in hand.
  A relation two projects describe has an owner row in each, and a component
  fetched for one carries it as an endpoint without the other's half, so that
  skipped the request that would have resolved the second owner. Only a relation
  actually asked about under this pin is skipped.
- A comment still called the producer graph gated to ducklake selections after
  it was widened to dbt.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* refactor: land dbt column lineage as storage and ingest only

The API surface that draws a column trace moves to a follow-up PR, on
`dbt-column-lineage-surface`. It kept generating findings — a client cache
whose premise was wrong for a two-owner relation, then staleness and a lost
retry from tightening it, and a seed walk that stopped at the first boundary —
and the fix for the last of them is a transitive owner expansion, which has to
re-apply the caller's gate to every newly discovered project. That is the same
shape as the leak four reviewers caught in the pinned arm, and it wants its own
review rather than being the fourth fix at the end of this one.

What lands here stands on its own: the analysis pass, `dbt_column_edge`,
`dbt_node.column_schema`, the engine gating and the error-contract split — plus
the one user-visible half, the typed and ordered column list, which rides the
asset graph the details pane already fetches and replaces a panel that could
only show the columns an author had documented.

Also fixes a real bug in the pass, found in review: it compiled without the
build's `--full-refresh`. `is_incremental()` branches on that flag, so an
incremental model reading `{{ this }}` compiles its self-join — and any `ref()`
inside that branch — only when the flag is absent, and the pass was storing
lineage for SQL a full-refresh run never executed. The flag now comes from one
place shared with the build, and a run that overrides it gets its own graph
rather than standing as the version's.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* docs: say why direct kinds get the budget without naming a view

The bucketing comments explained the priority by what a trace draws, which is
a forward reference now that the surface moved out. The reason stands on its
own: `copy`/`mod` say the value travelled, `scan` says the column was read to
produce the row and so reaches every output column of its model.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* fix: round-9 findings on the descoped PR

- The `full_refresh` helper was inserted between `selection_is_overridden` and
  its doc comment, so thirteen lines about `select`/`exclude` echoes documented
  the wrong function and the one they were written for had none. Moved below it.
- The parse path ran the analysis compile and the parquet decode BEFORE the
  guard that returns when there is no warehouse identity, paying for both and
  dropping the result. Moved after it.
- Three sites still described a `/column_lineage` endpoint this branch no longer
  has, and two user-facing strings promised a column trace it no longer renders:
  the panel's hint and the descriptor template now say what the flag actually
  buys, which is the typed column schema.
- Dropped test scaffolding the removed suite left behind: a `raw_orders` node
  and `dbt_edge` whose only assertion re-tested pre-existing graph behaviour,
  and a second editor-buffer node nothing asserts on.

Documented rather than fixed: an incremental model has two shapes, and which one
the index holds depends on whether the target existed when the pass ran.
`is_incremental()` is false with no target as well as under `--full-refresh`, and
dbt has no mode that emits both — so a version's graph describes the compile that
produced it, and only a re-ingesting run describes its own run.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* fix: keep lineage_kind in the edge key, and one answer for --full-refresh

- Both unique indexes omitted `lineage_kind`, so a column that is projected AND
  used as a predicate for the same output column — an ordinary shape — had its
  `copy` and `scan` edges collapse under `ON CONFLICT DO NOTHING`, while the
  digest counted both. The kind is part of the fact, so it is part of the key.
  Edited in the migration rather than added as a second one: it has not landed.
- `full_refresh` was shared between the build and the analysis pass without the
  `command != "test"` condition that sat at the build's call site, so the two
  disagreed for exactly the runs that build nothing. The condition moved inside
  the function, which is the point of sharing it, and the command is threaded to
  the pass.
- The "what a trace draws" rewrite missed the copy in `dbt_manifest.rs`.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* fix: drop the unreachable full_refresh threading, test the uniqueness key

`DBT_COMMANDS` is `["build", "retry", "show", "parse"]` and `default_command`
returns `build` in every arm, so `command == "test"` cannot happen — the guard
the last commit moved into `full_refresh` was already inert where it came from.
Threading the command through five signatures to preserve it bought nothing, so
it is gone; the build and the pass call one function of the descriptor and the
invocation, which is what the sharing was for.

The uniqueness-key fix now has a test: a column projected AND used as a
predicate for the same output column stores both its `copy` and its `scan` row.
Verified against the old key, where it returns 1 instead of 2.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* fix: restore the dbt test --full-refresh guard I removed on a wrong premise

The previous commit removed it after reading `DBT_COMMANDS` and concluding
`"test"` was unreachable. That is only true of the command a CALLER can name:
`run_dbt` is invoked with `"test"` directly for the `after_all` test phase, so
an `after_all` project with `full_refresh: true` reached it — and dbt rejects
`--full-refresh` on `test`, failing the phase. Both reviewers caught it.

The guard is back inside the shared function, where the build and the pass get
one answer, and its doc now records why reading the allowlist alone is
misleading. The test covering the `test` case is restored with it.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* fix: notice a job that ended during the decode, and name truncation as the cause

- The parquet decode runs on a blocking thread with no poller watching it, so a
  cancellation or an expired deadline during it was invisible: `dbt_dep` went on
  to publish the graph and the job returned success. The job's state is checked
  once the decode returns, before the caller publishes anything, and an ended
  job `Err`s — which this module may always do for the job's own semantics.
- A compile stopped by the output ceiling could leave no artifact, and the log
  then blamed the engine's capability, sending the reader to check their adapter
  rather than the ceiling. Truncation now names itself in the missing and
  unreadable branches too.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* fix: read cancellation from the DB after the decode, not from a poller's field

`ctx.canceled_by` is only ever written by a poller, and no poller runs during
the blocking decode — which is the exact window the check was added for. So the
guard caught only a cancellation already observed before it, and the comment
beside it claimed more than it did. It now queries `v2_job_queue` directly, the
same probe `worker_lockfiles` uses before it overwrites a flow.

A failed probe answers "still running": this decides whether to discard work
already done, so an unreachable database must not be the reason a healthy deploy
loses its graph.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* refactor: reuse job_is_canceled rather than a second copy of it

The probe added last round was `job_is_canceled` from the same file, retyped —
same query, same `Connection::Http` behaviour. Reused instead.

Its doc said a non-database connection was "a failed probe", which reads as an
error path. It is not: it is the agent worker, and on one there is no database
to ask, so only the deadline answers and a cancel issued during the decode is
not observable. The retry path avoids that by refusing to run on an agent worker
at all — which an optional annotation has no business doing — so the gap is
recorded at both ends instead.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* fix: close the agent-worker cancellation gap instead of documenting it

The previous commit said a cancel issued during the decode is not observable on
an agent worker. It is: `ping_job_status` returns `canceled_by` over both
connection kinds, and is how the poller itself notices one there. So the check
asks through the ping rather than querying `v2_job_queue` directly, and holds on
an agent worker, where a direct query reaches no database at all.

`job_is_canceled` goes back to private and its doc to what it said before — the
retry that calls it still refuses to run on an agent worker for its own reasons.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* fix: decode the index under the job poller instead of checking after it

Two findings with one cause: the decode was the only phase of this pass with no
subprocess behind it, so nothing heartbeated while it ran. A large index left
the worker silent for as long as it took, which the zombie sweep reads as a dead
job and restarts — and the cancellation check bolted on afterwards could only
ever report what had already happened, while dropping the ping's
`already_completed`, so a force-cancelled deploy still published its graph.

Running it under `run_future_with_polling_update_job_poller` answers all of it:
the poller pings throughout, and ends the phase with an `Err` on cancellation,
`AlreadyCompleted` or the phase timeout. The bespoke probe is gone with it.

Verified on a live deploy: 32 edges and 4 typed schemas ingested through the
polled decode.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* fix: stop a cancelled decode, and say what the read phase can now do

Putting the decode under the poller heartbeats it and ends the phase when the
job does, but dropping a `JoinHandle` detaches a blocking task rather than
cancelling it — so a cancelled job left a thread decoding up to four million
rows for a job that was over. The row loop reads an abandonment flag that a drop
guard on the awaiting future sets, so the decode stops at its next row.

That same change made the read phase able to `Err`, and three places still said
it could not — decision 14 in as many words. The distinction that holds is
narrower: nothing the ARTIFACT does or fails to do can fail a job, so absent,
unreadable and partial are all values; the JOB can still end the phase the read
runs in. Stated that way in the module doc, the `Artifact` doc, `MAX_INDEX_ROWS`
and the decision.

Verified on a live deploy: 32 edges and 4 typed schemas.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* refactor: share AbortOnDrop, and stop citing a hazard that is now handled

`Abandon` was `ansible_executor`'s `AbortOnDrop` retyped — same struct, same
reason, same `spawn_blocking` shape. Moved to `common` and used from both.

The paragraph explaining why the phase budget wraps the compile alone gave as
its reason "a decode still running on a blocking thread", which is exactly what
the abandonment flag now prevents. The reason that survives is the one that was
always the point: the budget exists to leave the build its share of the clock,
and only the compile can spend that share unboundedly. The decode's end is the
job's, through the poller it runs under.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* fix: put both doc comments back on the items they describe

Moving AbortOnDrop orphaned a doc at each end: it landed between
`raw_to_string`'s doc and `raw_to_string`, and the doc of the struct it replaced
stayed behind to prefix `fetch_repo_archive`.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* docs: name the binding the row loop actually reads

`Abandoned` was neither the type nor the binding; the flag is `abandoned`.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-08 09:58:38 +02:00
Ruben FiszelandClaude Opus 5 621fac55ab feat: durable dbt state per environment, and --defer onto it (#10975)
* feat: durable dbt state per environment, and `--defer` onto it

`dbt retry` worked off two artifacts and only one was durable: `dbt_run_state`
holds `run_results.json` keyed by principal, and the manifest lived on
worker-local disk under a four-generation cache. That is enough to resume the
last run and nothing else — the next run of a project usually lands on a worker
holding neither artifact — so deferral had nothing to read.

Adds `dbt_environment_state`: one row per (workspace, script path, environment),
holding `manifest.json` and `run_results.json` from the last successful run, with
the blob inline under `DBT_STATE_INLINE_MAX_BYTES` and in the workspace's object
storage above it. Environment is the warehouse, the target, and the database and
schema they resolve to, so a repointed warehouse or a moved schema reads as an
environment nothing has published rather than as state whose relation names no
longer fit.

A run publishes it when its graph becomes what the script owns and it succeeded
— the same condition, and the same reason: an invocation that scoped its own
model set describes where the caller put those relations, not where the
project's models live.

`defer` is a `build` command-block field defaulting to the descriptor's own, and
the state is materialised into the job directory for `--defer --state`. The
retry path already did that materialisation for `dbt retry`; both go through one
`write_state_dir` now.

`--state` is also where `dbt retry` reads the run it resumes, so a retry on
dbt-core 1.x takes `--defer-state` instead, and one on an engine without that
flag is refused before the build rather than rebuilding its nodes with every
unbuilt `ref()` resolving into the schema this run writes into.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Ned2pmRJwB3GpenEcrA9TF

* fix: address the local review of the dbt environment state

The oversized-artifact home moves from the workspace's object storage to the
instance's, where every other internal worker artifact already lives. The
workspace bucket is the one members read and write through `job_helpers/*` with
a caller-supplied key and only `volumes/` is reserved there, so a manifest under
it is one any member could replace — and the next deferring run would hand dbt
an attacker-chosen `defer_relation` for every unbuilt `ref()` while holding the
script's warehouse credentials.

The environment key takes the target dbt actually runs rather than the
descriptor's `profile.target`, which is absent whenever the target is inherited
from the workspace warehouse or the project's own `profiles.yml` — filing every
inherited target under one empty name, while a `target.name` macro decides where
a model is built. `write_profiles` returns a named struct now that it resolves
one more thing.

Publishing takes the row's lock before uploading, so two publishers of one
environment cannot interleave their uploads and leave one run's manifest beside
another's results, and carries the live-dbt-script guard the retry state already
had, so a job finishing after its script was renamed, archived or deleted cannot
recreate state at a path for whatever is created there next.

A rename now clears the environment state instead of moving it: an oversized
artifact's key is derived from the path, so a moved row would keep pointing at a
key a script created at the old path publishes over.

A build recovered by the automatic in-job node retry publishes its manifest
without results — `run_results.json` is then the retry's, naming only the nodes
it redid — and the refusal for an environment with nothing published names the
runs that cannot publish rather than suggesting a run that would not help.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* fix: serialize dbt state publishers on an advisory lock

The row lock only serializes publishers once a row exists, and the first
publish of an environment — two runs of a newly deployed script — is exactly
when two of them are most likely to race and interleave their uploads.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* fix: make dbt state publication atomic and bind it to the version that ran

Every publication now writes its own object keys and the row switches to them in
one statement, so an upload never overwrites an artifact the committed row still
names: a run failing between its two uploads, or between them and its row, leaves
the state pointing at the pair it already had. The objects a commit displaces are
dropped afterwards — never before, since a reader that has already read the row
is about to fetch them — and a reader that loses that race re-reads the row once
rather than reporting a state that is there. What a publication uploaded and then
could not commit is dropped on the way out.

The write's guard names the VERSION rather than the path: the live dbt script
there must be the one this job ran, or a later version of it. "Some live dbt
script is here" is also satisfied by a script created at a path this one was
renamed away from, and this job's manifest would then become that project's
deferral state. A preview names no version and so publishes nothing.

A `show` defers too. It compiles the model it previews, so a model whose upstream
this environment built and this run did not is exactly the case a deferral exists
for, and every engine takes the flags on it.

Three comments said "the workspace's object storage" where the code deliberately
uses the instance's, which is the whole security argument; `mib()` labelled MiB
values MB.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* fix: hold the script row across a dbt state publication, and let a rename move it

The version guard read `script` without a lock, so lifecycle cleanup could find
no environment row to clear, finish, and leave this transaction to commit state
at a path a new script goes on to occupy. It now holds that row (`FOR SHARE`) for
the rest of the publication — taken before the sidecar, the order every other dbt
writer takes — and the artifacts are uploaded before the transaction, so the lock
covers the row work rather than a network round trip.

A commit that reports an error may still have committed: what was lost can be the
acknowledgement. Dropping this run's objects then leaves the committed row naming
objects that are gone, so an orphan is the cheaper side to take.

A failed second upload left the manifest it had already written behind; it is
dropped now.

Per-publication keys retired the reason a rename cleared the environment state
rather than moving it: the path is only a prefix, and the row is what names an
artifact, so a script created at the old path can no longer publish over a moved
row. The rename moves both halves again.

`dbt ls` gets the deferral flags too, without which a `result:` selector — which
reads `run_results.json` out of the state directory, and which `select` passes to
dbt verbatim — fails before the build that would have honoured it.

Also: the migration was the last site describing the workspace's object storage
rather than the instance's, `publication_lock` folded 32 bits where it claimed
64, and `ResolvedProfile` had taken `write_profiles`'s doc block.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* fix: a deferring dbt run never publishes the state it read

`publishes_ownership` reads the CALLER's overrides, so a descriptor that already
narrows `select` needs none and a run of it with `defer: true` published. A
deferring run built some of the relations its manifest names and resolved the
rest out of the state it read, so recording that manifest claims relations
nothing built — and a model renamed since is recorded under a name only a full
build creates, breaking every later deferral until one repairs it.

Also: `publication_lock` parsed 16 hex digits as `i64`, which overflows for every
digest with the top bit set — half of them — collapsing those environments onto
one advisory key; and a failure to open the transaction returned without dropping
the objects already uploaded.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* fix: only a deployed dbt run publishes state, and key its objects per execution

A preview carries a caller-supplied `script_hash` into `runnable_id`
(`run_preview_script`), so the version guard alone let anyone who may run a job
publish arbitrary content as a deployed script's deferral state. The job's KIND
is checked beside it now. Verified: a preview submitted with the deployed path
and hash builds and leaves the row untouched.

Object keys carry a per-execution nonce. Zombie recovery re-runs a job under its
own id, so keyed on that alone a second attempt overwrote the objects the first
attempt's committed row still named, then read those same keys back as displaced
and dropped them — leaving the row unreadable. The displaced set is also filtered
against this publication's own keys, so the invariant is stated rather than
re-derived from the key format.

A project-owned `profiles.yml` that templates its schema or database is refused a
deferral: dbt renders those and Windmill does not, so two renderings resolve to
one `relation_root` and would share one environment key. Plainly absent is left
alone — that is the adapter's default, which does not move.

The deferral log line now says the run publishes no state of its own, which was
otherwise invisible.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* fix: a templated profile location publishes no dbt state either, on every path

A `dbt_profile` resource is one block of the user's own `profiles.yml` copied
through unchanged, and `profile.schema` is written as given, so either can carry
a template dbt renders and this runtime does not — exactly as a project-owned
file can. Only the project-owned path detected it.

And the refusal now covers publication as well as deferral: a published template
would sit under a key a literal profile shares, so de-templating later would make
that stale manifest readable as the new location's.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* fix: recognise Jinja statement blocks as a rendered dbt profile location

dbt renders a profile through Jinja, so `{% if env_var('ENV') == 'prod' %}…{% endif %}`
moves a schema exactly as an `env_var()` substitution does — and only `{{` was
detected, so such a profile published and deferred under one environment key for
every rendering. One predicate now serves both profile paths, with a test for
each delimiter.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* fix: a dbt state read outruns successive publications rather than one

The loader re-read once, which answers a single publication overtaking it: a
reader takes no lock and the advisory lock is released before the displaced
objects are dropped, so back-to-back publications could each overtake the same
read and the second was reported as a missing object. It now re-reads for as long
as the row keeps MOVING, bounded, and reports only when an unmoved row's objects
are genuinely gone.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* docs: a dbt state read outruns successive publications, not one

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* docs: name both ways a dbt state read can fail

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* fix: length-prefix the dbt environment key's components

A dbt target name and a schema are both the user's own strings, so joining them
on `|` let one component spell another tuple's key: `prod|analytics` + `scratch`
and `prod` + `analytics|scratch` were one environment, and a profile moving
between them read as the same one rather than as one nothing has published — the
collision the key exists to prevent. The schema and database are also taken apart
now rather than through `relation_root`'s own join, so neither can absorb the
other's delimiter.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* fix: name the dbt environment in words where a message shows it

The key is length-prefixed for storage, which is not something to put in front of
a caller: the "nothing published yet" refusal now reads "warehouse `main`, target
`prod`, relations in `dbt_wh_defer.analytics`". The worked example of the encoding
also miscounted a component.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* fix: delete a script version in the transaction that cleans up after it

`delete_script_by_hash` soft-deleted through the pool, committing before the
cleanup that follows it in `tx`. In that window the path has no live version, so
a concurrent deploy can take it — and `clear_dbt_script_state_if_path_retired`
then finds that new script live, keeps the deleted project's dbt state, and
leaves the replacement able to defer through its manifest. The update moves into
the same transaction, which is what `archive_script_by_hash` beside it already
does.

The retirement guard itself was pinned by nothing: the existing test moved the
only row away before calling the conditional clear, so it could not fail.
`state_goes_only_once_no_live_version_is_left` covers both directions — a second
live version keeps the state, the last one leaving takes it — and fails if the
predicate is inverted.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* fix: archive a script by path in the transaction that cleans up after it

The last of the four routes still writing outside its own cleanup transaction.
Archived on its own, a cleanup that then fails leaves dbt state at a path no live
version occupies, and whatever is created there next can defer through it. The
by-hash archive and both deletes already take their write in `tx`; this makes the
set uniform.

Two comments beside those clears still called the state the RETRY state alone,
which the rename made false — they cover both halves now — and the merged
verification list had two `11.`, main's #10978 having inserted an item above it.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* feat: refuse a dbt state selector the engines resolve inconsistently

`state:`, `result:` and `source_status:` selectors resolve against the
artifacts in `--state`, which only a deferring run is handed. The engines
disagree about what happens without one, and two of the three disagree
silently: dbt-core 1.x raises, but dbt-sa-cli 2.x and fusion read a missing
state as an empty one and exit 0, so `state:modified` builds nothing and
`state:new` builds the whole project, each reporting success.

Refuse them up front instead, naming `defer`. From the descriptor they are
refused outright, since that selection also decides which nodes the script
owns and the deploy resolves it with no state at all.

`source_status:` is refused under any setting: it compares `sources.json`,
which no run publishes here.

A caller's selection is now allowed to match nothing, which is what
`state:modified+` returns when nothing changed since the published state. It
is stored as that run's own snapshot and never becomes what the script owns,
so the ownership-wipe the refusal guarded against cannot happen. The
descriptor's selection still may not.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* fix: refuse a dbt result selector the published state cannot answer

Round 18 findings.

Codex P1: `defer` alone was enough to allow a `result:` selector, but a build
recovered by node retry publishes a manifest with no `run_results.json` — the
only file such a selector reads. dbt-core then raises an internal error and the
Rust engines match nothing and exit 0. The deferral now reports whether the
state carries results, and a `result:` selection against one that does not is
refused, naming the run that published it.

Claude P2: a `parse` returns before `defer` is read, so its deferral is always
absent and "turn `defer` on" was advice that led nowhere. The check now
distinguishes a run that could defer from a command that never does, and the
parse path says so.

Codex P2 / Claude P2: the roadmap still listed `state:modified` as out of scope
while the same file documented it as working. Narrowed both that line and the
scope list to the slim-CI work that genuinely remains.

Also pins the invariant the relaxed empty-selection guard rests on: an
overridden selection must not publish ownership, or an empty caller selection
would wipe the script's graph.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* fix: exempt an empty dbt selection by method, not by who chose it

Round 19 findings.

Codex P1: the empty-selection exemption keyed on whether the caller overrode
the selection, so a misspelled model name resolved to nothing, passed the guard
and reported a build that did its work. Key it on the selector instead: only a
`state:` or `result:` method may match nothing, its empty answer being a real
one. Every other selection matching nothing is refused again, from a run as
from the descriptor, each with the message that applies to it.

Claude P2: the spec still described a node-retry-recovered publication as one
where `result:` selectors merely lose their input, which the previous commit
stopped being true, and the section stating the selector rules recorded neither
the `result:`-without-results refusal nor the `parse` one. Both written down.

Also drops the refusal's claim that the publishing run WAS recovered by node
retry: an unreadable file reaches the same absent-results state, and the remedy
is the same either way.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* docs: record why an exempted empty dbt selection cannot wipe the graph

The safety argument left with the origin-based condition it justified. Under
the method-based one it is a consequence of the descriptor refusal in
check_state_selectors, two hops from this site, so state it here: relaxing that
refusal would let a descriptor-narrowed `state:modified+` reach the exemption
and be ingested as owning nothing.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-08 08:32:30 +02:00
Ruben FiszelandClaude Opus 5 15c2b81d6c chore: run local codex review on gpt-6-astra, bump codex cli pin (#11011)
* chore: run local codex review on gpt-6-astra and bump codex cli pin

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01X8u6o9MRAs2Rz16UbKQaD9

* fix: keep local codex review alive when --version is unparseable

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01X8u6o9MRAs2Rz16UbKQaD9

---------

Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-08 08:19:32 +02:00
Ruben Fiszelandrubenfiszel a9d42b489f chore(main): release 1.805.0 (#10995)
* chore(main): release 1.805.0

* Apply automatic changes

---------

Co-authored-by: rubenfiszel <275584+rubenfiszel@users.noreply.github.com>
v1.805.0
2026-09-07 18:20:28 +00:00
c3f7f8a458 fix: stop an untouched item's form from saving a draft nobody wrote (#10964)
* feat: gate drafts on real user input so a moved-on schema is not a draft

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01ScVgqGpuyDMWdzVNPm7Q5f

* Revert "feat: gate drafts on real user input so a moved-on schema is not a draft"

This reverts commit 6cd86cf727.

* fix: stop counting empty schema-added fields and server metadata as drafts

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01ScVgqGpuyDMWdzVNPm7Q5f

* feat: sweep away existing drafts that carry no changes, once per workspace

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01ScVgqGpuyDMWdzVNPm7Q5f

* Reapply "feat: gate drafts on real user input so a moved-on schema is not a draft"

This reverts commit b7b18e345e.

* Revert "fix: stop counting empty schema-added fields and server metadata as drafts"

This reverts commit 9787270ad8.

* docs: describe the sweep by the gate that now prevents new phantom drafts

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01ScVgqGpuyDMWdzVNPm7Q5f

* fix: make the draft sweep a compare-and-delete so it cannot eat live edits

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01ScVgqGpuyDMWdzVNPm7Q5f

* fix: close the gate's load-time window and stop sealing a failed sweep

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01ScVgqGpuyDMWdzVNPm7Q5f

* fix: open the gate on the edit itself, and stop the sweep at ownerless drafts

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01ScVgqGpuyDMWdzVNPm7Q5f

* fix: count a click as an edit, and keep an unjudged row from sealing the sweep

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01ScVgqGpuyDMWdzVNPm7Q5f

* fix: release the sweep's sync baseline when its delete is refused

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01ScVgqGpuyDMWdzVNPm7Q5f

* fix: drop the refused delete before re-baselining, and bound the sweep's retries

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01ScVgqGpuyDMWdzVNPm7Q5f

* refactor: send the sweep's delete straight to the API, not through the syncer

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01ScVgqGpuyDMWdzVNPm7Q5f

* fix: never absorb a change the resource type's schema could not have made

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01ScVgqGpuyDMWdzVNPm7Q5f

* fix: push an edit the gate only notices after the write has landed

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01ScVgqGpuyDMWdzVNPm7Q5f

* fix: stop the gating effect re-suspending a resource opened on a draft

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01ScVgqGpuyDMWdzVNPm7Q5f

* chore: update ee-repo-ref to d33ea730c550cdbc7d050aeb6d40dcef3d134e07

This commit updates the EE repository reference after PR #782 was merged in windmill-ee-private.

Previous ee-repo-ref: 313c572c9dcbcaafd8a1594df4054f9dd26f395c

New ee-repo-ref: d33ea730c550cdbc7d050aeb6d40dcef3d134e07

Automated by sync-ee-ref workflow.

---------

Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
Co-authored-by: windmill-internal-app[bot] <windmill-internal-app[bot]@users.noreply.github.com>
2026-09-07 19:31:59 +02:00
hugocasaandClaude Opus 5 7feaf619cf feat: run a linked AI agent's draft when testing a flow, and offer to deploy it (#10993)
* feat(frontend): run a linked agent's draft when testing a flow, and offer to deploy it

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_017B4omp8dRgmLitbpQEqFMp

* fix(frontend): settle an agent's autosave before reading it, and refresh its card on a draft save

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_017B4omp8dRgmLitbpQEqFMp

* fix(frontend): deploy the agent draft that was validated, and make the draft-tools flag explicit

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_017B4omp8dRgmLitbpQEqFMp

* fix(frontend): refuse a stale agent deploy, and warn when a never-deployed agent is kept as a draft

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_017B4omp8dRgmLitbpQEqFMp

* docs: record what inlining an agent draft puts in a preview job

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_017B4omp8dRgmLitbpQEqFMp

* fix(frontend): name the draft-changes dialog after what it lists

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_017B4omp8dRgmLitbpQEqFMp

* fix(frontend): refuse a draft deploy when the draft row is gone

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_017B4omp8dRgmLitbpQEqFMp

* fix(frontend): apply the missing-draft refusal to raw apps too

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_017B4omp8dRgmLitbpQEqFMp

* fix(frontend): stop reading a deployed resource row as a draft on deploy

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_017B4omp8dRgmLitbpQEqFMp

* fix(frontend): do not mistake an outage or a vanished draft for a deploy

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_017B4omp8dRgmLitbpQEqFMp

* fix(frontend): refuse an agent read whose pending draft save failed

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_017B4omp8dRgmLitbpQEqFMp

* fix(frontend): space the trigger badges and right-align the agent actions

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_017B4omp8dRgmLitbpQEqFMp

* fix(frontend): surface a failed agent-draft read instead of dropping it

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_017B4omp8dRgmLitbpQEqFMp

* fix(frontend): give the agent draft delete a baseline so a newer edit survives

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_017B4omp8dRgmLitbpQEqFMp

* fix(frontend): drop the agent draft cell locally instead of deleting twice

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_017B4omp8dRgmLitbpQEqFMp

* test(frontend): pass the withDraft flag the guard tests were missing

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_017B4omp8dRgmLitbpQEqFMp

* refactor(frontend): deploy agent drafts the way Review & Deploy does

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_017B4omp8dRgmLitbpQEqFMp

* fix(frontend): give the read-only flow graph its own linked-tools bucket

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_017B4omp8dRgmLitbpQEqFMp

* fix(frontend): base the resource draft delete on the read that promoted it

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_017B4omp8dRgmLitbpQEqFMp

* fix(frontend): refresh every step linking an agent when its draft is saved

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_017B4omp8dRgmLitbpQEqFMp

* fix(frontend): write nothing at all when a resource draft has gone

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_017B4omp8dRgmLitbpQEqFMp

* test(frontend): pin that the resource draft delete follows its baseline seed

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_017B4omp8dRgmLitbpQEqFMp

---------

Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-07 19:04:38 +02:00
hugocasaandClaude Opus 5 48a56158c1 feat: report resource type picks to the hub and rank pickers by popularity (#10982)
* feat: report resource type picks to the hub and rank pickers by popularity

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FqHuykfRrkDHj9dHCJQQcE

* fix: scope the hub pick route as a write and keep an alphabetical floor

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FqHuykfRrkDHj9dHCJQQcE

* fix: rank the types a workspace already uses above the hub's own picks

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* fix: total local usage per integration, not per resource type name

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* fix: remember a failed hub index read briefly instead of retrying every open

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-07 18:59:24 +02:00
Diego ImbertandClaude Fable 5.1 519a5c8bc7 fix(frontend): stop hover flicker on asset nodes shared with an overflow popover (#10996)
Claude-Session: https://claude.ai/code/session_01HNugALVxFkkAeM5mce4CFQ

Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
2026-09-07 18:57:53 +02:00
Ruben FiszelandClaude Opus 5 8d0f4754e4 fix: let a draft-only schedule, trigger or resource be deleted (#11010)
* fix: let a draft-only schedule, trigger or resource be deleted

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012SV5kjTis3AFtTx2nW2VRi

* fix: keep the legacy-draft write gate out of the draft-only delete

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012SV5kjTis3AFtTx2nW2VRi

* fix: don't gate a draft-only resource discard on the deployment rules

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012SV5kjTis3AFtTx2nW2VRi

* docs: condense the draft-only delete comments per the comment policy

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012SV5kjTis3AFtTx2nW2VRi

---------

Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-07 18:55:40 +02:00
Diego ImbertandClaude Fable 5.1 1be390aa87 fix(frontend): no phantom draft when opening a CLI-pushed script (#10997)
* fix(frontend): no phantom draft when opening a CLI-pushed script

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01QpqVdiTiqVGCvgmpBzL6m3

* fix(frontend): infer the dbt descriptor schema on mount like ScriptEditor

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01QpqVdiTiqVGCvgmpBzL6m3

* fix(frontend): retry the baseline schema inference once like the editors do

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01QpqVdiTiqVGCvgmpBzL6m3

---------

Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
2026-09-07 17:50:28 +02:00
Ruben FiszelandClaude Opus 5 8f553eab35 fix: point the app viewer's edit button at the editor for the app's kind (#11009)
Claude-Session: https://claude.ai/code/session_01GDiZaPzhC4R9G4hLPgy1B2

Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-07 17:38:38 +02:00