1124 Commits

Include renames

Author	SHA1	Message	Date
Ruben FiszelandClaude Opus 5	de98adf055	feat: let a worker group override the dependency cache object store (#11019) ... * feat: let a worker group override the dependency cache object store Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01R6hBAWqNUsQMAJ7P59juug * fix: address review findings on the worker-group cache override Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01R6hBAWqNUsQMAJ7P59juug * fix: close the remaining config read route and re-evaluate the override on plan change Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01R6hBAWqNUsQMAJ7P59juug * fix: serialize override reloads and keep a store a failed rebuild still serves Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01R6hBAWqNUsQMAJ7P59juug * fix: require enterprise for the cache override and lock its whole transition Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01R6hBAWqNUsQMAJ7P59juug --------- Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>	2026-09-08 14:10:51 +02:00
3 people	9444049d60	feat: bring gitlab repositories to parity for git sync (#10938) ... * feat: track and rotate gitlab git-sync repository tokens * chore: point ee-repo-ref at the gitlab credential branch * fix: strip server-owned credential status and correct expiry copy * fix: gate credential maintenance on enterprise and alert on stalled renewal * fix: alert on an auto-renewed token only once it has actually expired * feat: receive gitlab push webhooks for instant git sync pull * feat: open gitlab merge requests and post diff previews on them Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01C1xHmkxuxYb1GYvth1BS75 * fix: keep gitlab merge request previews out of the project's own pipeline Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01C1xHmkxuxYb1GYvth1BS75 * fix: bound the credential maintenance pass and gate the gitlab picker on a license Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01C1xHmkxuxYb1GYvth1BS75 * fix: create the gitlab picker's variable in the edited workspace Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01C1xHmkxuxYb1GYvth1BS75 * fix: make the gitlab picker's variable path collision-resistant Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01C1xHmkxuxYb1GYvth1BS75 * docs: state the gitlab scope and rotation facts the code relies on Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01C1xHmkxuxYb1GYvth1BS75 * fix: resolve the check marker's repository from its path, not a stored url Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01C1xHmkxuxYb1GYvth1BS75 * fix: refuse to finish a check whose repository has been repointed Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01C1xHmkxuxYb1GYvth1BS75 * fix: trust a check marker's captured url when it carries no identity Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01C1xHmkxuxYb1GYvth1BS75 * fix: recreate a missing webhook from credential maintenance Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01C1xHmkxuxYb1GYvth1BS75 * docs: state that relative-url gitlab installs are out of scope Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01C1xHmkxuxYb1GYvth1BS75 * fix: keep credential status out of exports and clear stale webhook warnings Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01C1xHmkxuxYb1GYvth1BS75 * fix: refuse an unprovable check and guard the picker on the stored repository Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01C1xHmkxuxYb1GYvth1BS75 * fix: re-check the picker's target path at the moment it is written Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01C1xHmkxuxYb1GYvth1BS75 * fix: snapshot the picker's inputs before it starts writing Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01C1xHmkxuxYb1GYvth1BS75 * docs: recommend a project access token per repository Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01C1xHmkxuxYb1GYvth1BS75 * [ee] feat: keep the git-sync credential in workspace settings * [ee] fix: drop a removed repo's credential and honor the workspace override * [ee] fix: resolve a fork's git credential from its whole ancestry * [ee] refactor: reuse fork_ancestor_chain instead of a second ancestry walk * [ee] fix: resolve an app installation from the whole ancestry, not the parent * [ee] revert: keep the app installation fallback at one level * fix: store the git credential only once the resource is saved * fix: keep a repository's credential when it leaves git sync settings * docs: cut the gitlab picker's token guidance down to what it needs * feat: mark a repository whose credential windmill holds * fix: ignore the managed-credential marker when the url carries a token * docs: drop the picker's setup alert for a line by the token field * feat: replace a repository's stored token from its resource * fix: store a picked credential for its own workspace, before the resource * refactor: key a stored git credential by its repository, not its resource Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01C1xHmkxuxYb1GYvth1BS75 * chore: refresh the sqlx cache for the repository-keyed credential queries Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01C1xHmkxuxYb1GYvth1BS75 * fix: gate the credential pass budget on the features that use it Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01C1xHmkxuxYb1GYvth1BS75 * fix: decide credential rotation ownership by repository, not resource path Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01C1xHmkxuxYb1GYvth1BS75 * refactor: renew only the credentials windmill holds, not tokens in a repo url Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01C1xHmkxuxYb1GYvth1BS75 * fix: authenticate the fork-branch poll and correct the renewal guidance Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01C1xHmkxuxYb1GYvth1BS75 * fix: do not claim a managed credential for a url the client cannot resolve Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01C1xHmkxuxYb1GYvth1BS75 * fix: define the credential facade for private builds without enterprise Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01C1xHmkxuxYb1GYvth1BS75 * fix: pin the listed token before the await and name the real renewal blocker Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01C1xHmkxuxYb1GYvth1BS75 * fix: pin the token the replace flow checked, and derive the scope test once Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01C1xHmkxuxYb1GYvth1BS75 * fix: classify the renewal state once so the card cannot contradict itself Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01C1xHmkxuxYb1GYvth1BS75 * refactor: ask only whether the token gets renewed, not why it does not Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01C1xHmkxuxYb1GYvth1BS75 * refactor: replace the managed-credential marker with a server answer Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01C1xHmkxuxYb1GYvth1BS75 * fix: read renewal from the credential and its origin, not a removed field Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01C1xHmkxuxYb1GYvth1BS75 * fix: read the provider for url-token repos, await the origin before defaulting, and visit unchecked repos last The maintenance pass sorted repositories with no recorded check first on the premise that they cost nothing, but a token-in-URL remote on a host that is not GitLab is probed every pass and never records a check, so it held the head of the list ahead of the tokens that expire. Such repositories now sort last. The card decided its delivery defaults before the origin lookup landed, so a freshly picked GitLab repository never got webhook delivery; the two lookups are awaited together. The resource editor offers to replace a token only where it is held, not in a fork that borrows it, and the replace flow refuses a URL it cannot parse instead of keying the token to it. Attaching a stored credential to a commit-hash probe now requires admin, matching the installation credential beside it. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01C1xHmkxuxYb1GYvth1BS75 * docs: describe the gitlab listing token the way the picker and the setup guide do Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01C1xHmkxuxYb1GYvth1BS75 * refactor: a token in the repository url is a plain remote, not a tracked credential Drops the status fingerprint that told one URL token from another, the docs' promise that such a token's expiry is reported, and the test's expectation that a URL-token repository declares a host. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01C1xHmkxuxYb1GYvth1BS75 * fix: the card reads the credential origin for managed controls and honours the licence for a borrowed token Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01C1xHmkxuxYb1GYvth1BS75 * chore: bump the ee ref Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01C1xHmkxuxYb1GYvth1BS75 * fix: hide a repository's credential line once nothing is held for the repository it names Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01C1xHmkxuxYb1GYvth1BS75 * docs: describe the exported credential status as it is Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01C1xHmkxuxYb1GYvth1BS75 * refactor: run the credential maintenance pass as its own task, without a budget The pass ran inside the monitor's join, whose deadline cancels every future in it, and a rotation cancelled between GitLab issuing a token and Windmill storing it loses the token family. A wall-clock budget with a least-recently-checked ordering kept it under the deadline. Spawning the pass instead makes the deadline irrelevant, so the budget, the ordering and the counter go; the advisory lock keeps a slow pass from overlapping the next, as it already did. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01C1xHmkxuxYb1GYvth1BS75 * docs: say what detaching the maintenance pass buys, and what it does not Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01C1xHmkxuxYb1GYvth1BS75 * chore: run git sync on the hub script version that reads a stored credential Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01C1xHmkxuxYb1GYvth1BS75 * chore: run the deploy push and the connection test on the hub versions that read a stored credential Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01C1xHmkxuxYb1GYvth1BS75 * fix: keep App repositories and plain remotes out of the stored-credential paths Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01C1xHmkxuxYb1GYvth1BS75 * chore: host-neutral deploy preview wording, drop the project filter from the GitLab picker Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01C1xHmkxuxYb1GYvth1BS75 * chore: bump ee ref, rotation no longer retains a second connection per repository Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01C1xHmkxuxYb1GYvth1BS75 * chore: bump ee ref, the rotation write-back holds a single connection Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01C1xHmkxuxYb1GYvth1BS75 * chore: hold the credential maintenance lock in a transaction so a dead sweep releases it Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01C1xHmkxuxYb1GYvth1BS75 * docs: describe the credential-stored callback as it fires Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01C1xHmkxuxYb1GYvth1BS75 * fix: keep the credential maintenance lock past the pool's idle-in-transaction timeout Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01C1xHmkxuxYb1GYvth1BS75 * chore: update ee-repo-ref to e092518ee60e33160fee9ae91a4d109566f7b0ee This commit updates the EE repository reference after PR #771 was merged in windmill-ee-private. Previous ee-repo-ref: 74481f7cc345757aebb2a8b04d3a22978328c348 New ee-repo-ref: e092518ee60e33160fee9ae91a4d109566f7b0ee Automated by sync-ee-ref workflow. --------- Co-authored-by: Claude Opus 5 <noreply@anthropic.com> Co-authored-by: windmill-internal-app[bot] <windmill-internal-app[bot]@users.noreply.github.com>	2026-09-08 11:54:43 +02:00
Ruben FiszelandClaude Opus 5	0139467b01	feat: ingest dbt column lineage and real column schemas from the engine's parquet index (#10977) ... * feat: column-level lineage for dbt from the engine's parquet index `manifest.json` carries no column-to-column edges, which is why decision 14 recorded column lineage as unavailable. The edges live in a different artifact: `dbt compile --static-analysis strict --write-index` writes `target/index/`, whose `dbt.column_lineage.parquet` holds them and whose `dbt.node_columns.parquet` holds every column of every node, typed and ordered rather than only the ones an author documented. Strict analysis rejects SQL the default accepts, so this is a separate compile with its own `--target-path`, opt-in per project via `column_lineage: true`, and best-effort throughout: a project it cannot analyze keeps exactly the graph it had, with the engine's own diagnostics in the job log. Storage mirrors `dbt_edge`: `dbt_column_edge` keyed by (path, version, job) with the same composite FK to `script` and the same sweeps. The typed column list lands in `dbt_node.column_schema`, beside `columns` rather than merged into it, so `columns` stays what the author declared. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01PRtsPQ3Ck69Fu9DNr7bMcJ * fix: address review findings on the dbt column-lineage pass - The workspace fork copied every other dbt sidecar table and not this one, so a fork lost its column lineage silently and could not recover it: the cloned digest covers the column edges, so a dynamic run in the fork matched it and stored nothing. - The parquet was collected whole before the edge cap applied, which is exactly the input the cap exists for — a project whose `scan` lineage is quadratic in its widest model could take the worker process down. Decoded a row at a time with the bound enforced during the decode. - The pass swallowed every error from the runner, including the job poller's cancellation and deadline, so a run that blew its timeout inside an optional annotation could still publish a graph and report success. `run_captured` now carries the exit status in its value, so only a failed COMPILE is downgraded, and the pass may spend at most half the remaining wall clock so it cannot starve the build that follows it. - `scan` edges are stored but no longer served: they are most of a project's lineage, nothing renders them, and the graph endpoint is polled by the run page. They are also the first thing the storage cap gives up now, rather than evicting the direct edges the trace draws. - `column_schema` and the column edges take the same gate as the model's SQL. A column-level view is the shape of what the author wrote, one level finer than the `ref()` graph, which is ungated only because it draws relations the caller already sees. - `graph_digest` hashes the new section only when it has edges, so a project that never asked for the pass keeps the digest it has instead of re-snapshotting on every dynamic run until it is redeployed. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01PRtsPQ3Ck69Fu9DNr7bMcJ * fix: the editor buffer's column lineage, and three bounds that were wrong Round-2 review found four defects, all of them introduced by the round-1 fixes. - The `script_visible` gate on the column edges was copied from the node query without its `script_hash IS NULL` arm. `= NULL` is never true, so every version-less row was filtered out and an editor buffer's parse rendered its typed columns and none of their lineage — the one place the feature is meant to be used. Pinned by an assertion in `dbt_pinned_graph.rs`, which is where this class of bug already had a home. - The phase budget was handed to the poller, whose expiry is an `Err` indistinguishable from a cancellation or the job's own deadline, so a slow but valid analysis aborted the build it exists to annotate. The runner gets the full deadline again — those two must still fail the job — and the budget is a race around the whole pass, where expiring is this budget and nothing else. - The decode cap counted parquet ROWS, so `scan` and out-of-graph rows could spend it before a single drawn edge was read. It now counts what is kept, takes direct kinds in a first pass, and is handed the graph's own nodes so the budget cannot go on rows that could never be stored. - Hashing the new digest section conditionally did not preserve old digests, because an absent `column_schema` still serialized as `null` inside the nodes. It is skipped when absent instead. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01PRtsPQ3Ck69Fu9DNr7bMcJ * refactor: split the lineage pass by error contract, and read it in one query Round 3's findings were all consequences of round 1 and 2's fixes, clustered in the same two files, so this reshapes those two seams rather than patching again. The worker pass was one function being three things at once — a subprocess runner with job-lifecycle error semantics, a bounded decoder, and a best-effort degrader — which is why each fix to one perturbed another. It is now `compile_index`, which owns the JOB's semantics (only a cancellation or the job's deadline can `Err`; a non-zero exit, the output ceiling and the phase budget are outcomes), and `read_index`, which owns the ARTIFACT's and knows nothing about the job. The budget wraps the compile alone, so a decode can no longer outlive the timeout that reported the build would get the rest. The output ceiling likewise becomes a value rather than a job error, for the caller that can carry on without the tail of a compile's stdout. The column edges were read by a fourth hand-written copy of the `live`/`chosen` CTEs and the version/editor-buffer join conditions, and copying them is what dropped the `script_hash IS NULL` arm and hid every buffer parse's lineage. Both kinds of edge now come from ONE statement over a `UNION ALL`'d edge source, so those conditions exist once. The union is at the source rather than a join because column lineage can name a node pair `dbt_edge` has no row for: a model reading `{{ this }}` gets edges from itself to itself, and `parent_map` has no self-loop. The cap on the column half now sits after the scope filter, the visibility check and the graph joins — the scope moved into SQL via the existing `ScopePathFilter` — so a row the caller may not read can no longer spend it and leave an allowed project's trace short. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01PRtsPQ3Ck69Fu9DNr7bMcJ * refactor: serve dbt column lineage from its own endpoint The column edges rode on the folder-wide asset graph, which a run page polls, while the trace is drawn for one selected relation. That needed a cap, and a cap has to be applied after every filter that can drop a row. Keyed to the asset there is no cap: `assets/column_lineage` answers for one relation, and the caller's `scripts:read` scope and the project's visibility are decided once, for the script that owns it. Pinning to a run's snapshot or the editor's parse of its buffer costs the job-read gate, so that form is `jobs/dbt_column_lineage/{id}` — the same shape `jobs/dbt_graph/{id}` has. The worker's decode now bounds work and memory separately, and a compile stopped by the output ceiling reports as truncated rather than complete. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix: resolve the owning dbt version the way the graph does The unpinned arm picked the newest live version at the path without narrowing to dbt, so a path since redeployed in another language answered with no lineage while the graph beside it still drew that project's stale nodes. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix: gate pinned column lineage on reading the project, and answer the component Four things round 5 found, three of them in code this branch rewrote: - The pinned arm resolved the version from the job and stopped there, so a share-link viewer entitled to a run got the project's column names and edges while the graph beside it still redacted `raw_code` and `column_schema`. Resolving WHICH version answers is not deciding whether the caller may read it; the version-less editor buffer keeps its exemption, having no `script` row to ask. - The answer was the whole owning project's edges. The canvas lays out the connected component of the selected relation's columns, so the rest was unrenderable weight; a recursive walk over both directions returns exactly what is drawn, and the project key travels with it so a `unique_id` two projects share cannot walk from one graph into the other. - The decode had no exit but the 4M-row backstop once its buckets were full, spending wall clock the build below does not get. - An unreadable index was reported as a missing one, sending the reader to look at their engine rather than at the file. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix: stitch the two column graphs, and walk the component in Rust Round 6's two findings, both regressions this branch introduced: - The decode returned `Continue` on the edge that FILLED the direct-edge budget, so a `scan`-only tail after it decoded to the 4M-row backstop with nowhere to put anything. The read now ends on that edge. - Seam 3 made the pipeline page choose between the dbt graph and the producer one. They share node ids — `// column total <- dbt://wh/analytics/orders.amount` mints the same `(dbt, path, column)` node dbt's own lineage does — so choosing ended a trace at the boundary in both directions. They are merged again, and a ducklake selection asks about the dbt relation its producers name so the chain continues past it. The dbt editor gets the same merge. Also: the component is walked in Rust rather than by a recursive CTE. A CTE has no index, so the recursive term rescanned the doubled edge set once per level — 1243ms against 59ms for the query alone on a 3000-model project, 11.7M rows in the plan. Same answers, same tests; end to end 1.48s to 0.73s there and 1.60s to 0.26s on a 1000-deep chain. The client stops re-asking for a component it already holds, which is most clicks within one project. The four doc sites that described a whole-project answer are rewritten around what it now is, rather than edited where they disagreed. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix: expand every dbt boundary a selection reaches, and only skip what was asked Round 7's findings, all in the frontend seam this branch added: - A ducklake selection seeded the dbt fetch from the FIRST boundary relation it found, so a table derived from two unconnected dbt relations expanded one and left the other a leaf — the same "stops at the boundary" symptom the round-6 fix removed, one hop further along. Every distinct boundary is fetched now and the components merged. - The component cache skipped a relation merely PRESENT in the graph in hand. A relation two projects describe has an owner row in each, and a component fetched for one carries it as an endpoint without the other's half, so that skipped the request that would have resolved the second owner. Only a relation actually asked about under this pin is skipped. - A comment still called the producer graph gated to ducklake selections after it was widened to dbt. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * refactor: land dbt column lineage as storage and ingest only The API surface that draws a column trace moves to a follow-up PR, on `dbt-column-lineage-surface`. It kept generating findings — a client cache whose premise was wrong for a two-owner relation, then staleness and a lost retry from tightening it, and a seed walk that stopped at the first boundary — and the fix for the last of them is a transitive owner expansion, which has to re-apply the caller's gate to every newly discovered project. That is the same shape as the leak four reviewers caught in the pinned arm, and it wants its own review rather than being the fourth fix at the end of this one. What lands here stands on its own: the analysis pass, `dbt_column_edge`, `dbt_node.column_schema`, the engine gating and the error-contract split — plus the one user-visible half, the typed and ordered column list, which rides the asset graph the details pane already fetches and replaces a panel that could only show the columns an author had documented. Also fixes a real bug in the pass, found in review: it compiled without the build's `--full-refresh`. `is_incremental()` branches on that flag, so an incremental model reading `{{ this }}` compiles its self-join — and any `ref()` inside that branch — only when the flag is absent, and the pass was storing lineage for SQL a full-refresh run never executed. The flag now comes from one place shared with the build, and a run that overrides it gets its own graph rather than standing as the version's. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * docs: say why direct kinds get the budget without naming a view The bucketing comments explained the priority by what a trace draws, which is a forward reference now that the surface moved out. The reason stands on its own: `copy`/`mod` say the value travelled, `scan` says the column was read to produce the row and so reaches every output column of its model. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix: round-9 findings on the descoped PR - The `full_refresh` helper was inserted between `selection_is_overridden` and its doc comment, so thirteen lines about `select`/`exclude` echoes documented the wrong function and the one they were written for had none. Moved below it. - The parse path ran the analysis compile and the parquet decode BEFORE the guard that returns when there is no warehouse identity, paying for both and dropping the result. Moved after it. - Three sites still described a `/column_lineage` endpoint this branch no longer has, and two user-facing strings promised a column trace it no longer renders: the panel's hint and the descriptor template now say what the flag actually buys, which is the typed column schema. - Dropped test scaffolding the removed suite left behind: a `raw_orders` node and `dbt_edge` whose only assertion re-tested pre-existing graph behaviour, and a second editor-buffer node nothing asserts on. Documented rather than fixed: an incremental model has two shapes, and which one the index holds depends on whether the target existed when the pass ran. `is_incremental()` is false with no target as well as under `--full-refresh`, and dbt has no mode that emits both — so a version's graph describes the compile that produced it, and only a re-ingesting run describes its own run. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix: keep lineage_kind in the edge key, and one answer for --full-refresh - Both unique indexes omitted `lineage_kind`, so a column that is projected AND used as a predicate for the same output column — an ordinary shape — had its `copy` and `scan` edges collapse under `ON CONFLICT DO NOTHING`, while the digest counted both. The kind is part of the fact, so it is part of the key. Edited in the migration rather than added as a second one: it has not landed. - `full_refresh` was shared between the build and the analysis pass without the `command != "test"` condition that sat at the build's call site, so the two disagreed for exactly the runs that build nothing. The condition moved inside the function, which is the point of sharing it, and the command is threaded to the pass. - The "what a trace draws" rewrite missed the copy in `dbt_manifest.rs`. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix: drop the unreachable full_refresh threading, test the uniqueness key `DBT_COMMANDS` is `["build", "retry", "show", "parse"]` and `default_command` returns `build` in every arm, so `command == "test"` cannot happen — the guard the last commit moved into `full_refresh` was already inert where it came from. Threading the command through five signatures to preserve it bought nothing, so it is gone; the build and the pass call one function of the descriptor and the invocation, which is what the sharing was for. The uniqueness-key fix now has a test: a column projected AND used as a predicate for the same output column stores both its `copy` and its `scan` row. Verified against the old key, where it returns 1 instead of 2. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix: restore the dbt test --full-refresh guard I removed on a wrong premise The previous commit removed it after reading `DBT_COMMANDS` and concluding `"test"` was unreachable. That is only true of the command a CALLER can name: `run_dbt` is invoked with `"test"` directly for the `after_all` test phase, so an `after_all` project with `full_refresh: true` reached it — and dbt rejects `--full-refresh` on `test`, failing the phase. Both reviewers caught it. The guard is back inside the shared function, where the build and the pass get one answer, and its doc now records why reading the allowlist alone is misleading. The test covering the `test` case is restored with it. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix: notice a job that ended during the decode, and name truncation as the cause - The parquet decode runs on a blocking thread with no poller watching it, so a cancellation or an expired deadline during it was invisible: `dbt_dep` went on to publish the graph and the job returned success. The job's state is checked once the decode returns, before the caller publishes anything, and an ended job `Err`s — which this module may always do for the job's own semantics. - A compile stopped by the output ceiling could leave no artifact, and the log then blamed the engine's capability, sending the reader to check their adapter rather than the ceiling. Truncation now names itself in the missing and unreadable branches too. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix: read cancellation from the DB after the decode, not from a poller's field `ctx.canceled_by` is only ever written by a poller, and no poller runs during the blocking decode — which is the exact window the check was added for. So the guard caught only a cancellation already observed before it, and the comment beside it claimed more than it did. It now queries `v2_job_queue` directly, the same probe `worker_lockfiles` uses before it overwrites a flow. A failed probe answers "still running": this decides whether to discard work already done, so an unreachable database must not be the reason a healthy deploy loses its graph. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * refactor: reuse job_is_canceled rather than a second copy of it The probe added last round was `job_is_canceled` from the same file, retyped — same query, same `Connection::Http` behaviour. Reused instead. Its doc said a non-database connection was "a failed probe", which reads as an error path. It is not: it is the agent worker, and on one there is no database to ask, so only the deadline answers and a cancel issued during the decode is not observable. The retry path avoids that by refusing to run on an agent worker at all — which an optional annotation has no business doing — so the gap is recorded at both ends instead. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix: close the agent-worker cancellation gap instead of documenting it The previous commit said a cancel issued during the decode is not observable on an agent worker. It is: `ping_job_status` returns `canceled_by` over both connection kinds, and is how the poller itself notices one there. So the check asks through the ping rather than querying `v2_job_queue` directly, and holds on an agent worker, where a direct query reaches no database at all. `job_is_canceled` goes back to private and its doc to what it said before — the retry that calls it still refuses to run on an agent worker for its own reasons. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix: decode the index under the job poller instead of checking after it Two findings with one cause: the decode was the only phase of this pass with no subprocess behind it, so nothing heartbeated while it ran. A large index left the worker silent for as long as it took, which the zombie sweep reads as a dead job and restarts — and the cancellation check bolted on afterwards could only ever report what had already happened, while dropping the ping's `already_completed`, so a force-cancelled deploy still published its graph. Running it under `run_future_with_polling_update_job_poller` answers all of it: the poller pings throughout, and ends the phase with an `Err` on cancellation, `AlreadyCompleted` or the phase timeout. The bespoke probe is gone with it. Verified on a live deploy: 32 edges and 4 typed schemas ingested through the polled decode. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix: stop a cancelled decode, and say what the read phase can now do Putting the decode under the poller heartbeats it and ends the phase when the job does, but dropping a `JoinHandle` detaches a blocking task rather than cancelling it — so a cancelled job left a thread decoding up to four million rows for a job that was over. The row loop reads an abandonment flag that a drop guard on the awaiting future sets, so the decode stops at its next row. That same change made the read phase able to `Err`, and three places still said it could not — decision 14 in as many words. The distinction that holds is narrower: nothing the ARTIFACT does or fails to do can fail a job, so absent, unreadable and partial are all values; the JOB can still end the phase the read runs in. Stated that way in the module doc, the `Artifact` doc, `MAX_INDEX_ROWS` and the decision. Verified on a live deploy: 32 edges and 4 typed schemas. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * refactor: share AbortOnDrop, and stop citing a hazard that is now handled `Abandon` was `ansible_executor`'s `AbortOnDrop` retyped — same struct, same reason, same `spawn_blocking` shape. Moved to `common` and used from both. The paragraph explaining why the phase budget wraps the compile alone gave as its reason "a decode still running on a blocking thread", which is exactly what the abandonment flag now prevents. The reason that survives is the one that was always the point: the budget exists to leave the build its share of the clock, and only the compile can spend that share unboundedly. The decode's end is the job's, through the poller it runs under. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix: put both doc comments back on the items they describe Moving AbortOnDrop orphaned a doc at each end: it landed between `raw_to_string`'s doc and `raw_to_string`, and the doc of the struct it replaced stayed behind to prefix `fetch_repo_archive`. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * docs: name the binding the row loop actually reads `Abandoned` was neither the type nor the binding; the flag is `abandoned`. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>	2026-09-08 09:58:38 +02:00
Ruben FiszelandClaude Opus 5	621fac55ab	feat: durable dbt state per environment, and --defer onto it (#10975) ... * feat: durable dbt state per environment, and `--defer` onto it `dbt retry` worked off two artifacts and only one was durable: `dbt_run_state` holds `run_results.json` keyed by principal, and the manifest lived on worker-local disk under a four-generation cache. That is enough to resume the last run and nothing else — the next run of a project usually lands on a worker holding neither artifact — so deferral had nothing to read. Adds `dbt_environment_state`: one row per (workspace, script path, environment), holding `manifest.json` and `run_results.json` from the last successful run, with the blob inline under `DBT_STATE_INLINE_MAX_BYTES` and in the workspace's object storage above it. Environment is the warehouse, the target, and the database and schema they resolve to, so a repointed warehouse or a moved schema reads as an environment nothing has published rather than as state whose relation names no longer fit. A run publishes it when its graph becomes what the script owns and it succeeded — the same condition, and the same reason: an invocation that scoped its own model set describes where the caller put those relations, not where the project's models live. `defer` is a `build` command-block field defaulting to the descriptor's own, and the state is materialised into the job directory for `--defer --state`. The retry path already did that materialisation for `dbt retry`; both go through one `write_state_dir` now. `--state` is also where `dbt retry` reads the run it resumes, so a retry on dbt-core 1.x takes `--defer-state` instead, and one on an engine without that flag is refused before the build rather than rebuilding its nodes with every unbuilt `ref()` resolving into the schema this run writes into. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Ned2pmRJwB3GpenEcrA9TF * fix: address the local review of the dbt environment state The oversized-artifact home moves from the workspace's object storage to the instance's, where every other internal worker artifact already lives. The workspace bucket is the one members read and write through `job_helpers/*` with a caller-supplied key and only `volumes/` is reserved there, so a manifest under it is one any member could replace — and the next deferring run would hand dbt an attacker-chosen `defer_relation` for every unbuilt `ref()` while holding the script's warehouse credentials. The environment key takes the target dbt actually runs rather than the descriptor's `profile.target`, which is absent whenever the target is inherited from the workspace warehouse or the project's own `profiles.yml` — filing every inherited target under one empty name, while a `target.name` macro decides where a model is built. `write_profiles` returns a named struct now that it resolves one more thing. Publishing takes the row's lock before uploading, so two publishers of one environment cannot interleave their uploads and leave one run's manifest beside another's results, and carries the live-dbt-script guard the retry state already had, so a job finishing after its script was renamed, archived or deleted cannot recreate state at a path for whatever is created there next. A rename now clears the environment state instead of moving it: an oversized artifact's key is derived from the path, so a moved row would keep pointing at a key a script created at the old path publishes over. A build recovered by the automatic in-job node retry publishes its manifest without results — `run_results.json` is then the retry's, naming only the nodes it redid — and the refusal for an environment with nothing published names the runs that cannot publish rather than suggesting a run that would not help. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix: serialize dbt state publishers on an advisory lock The row lock only serializes publishers once a row exists, and the first publish of an environment — two runs of a newly deployed script — is exactly when two of them are most likely to race and interleave their uploads. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix: make dbt state publication atomic and bind it to the version that ran Every publication now writes its own object keys and the row switches to them in one statement, so an upload never overwrites an artifact the committed row still names: a run failing between its two uploads, or between them and its row, leaves the state pointing at the pair it already had. The objects a commit displaces are dropped afterwards — never before, since a reader that has already read the row is about to fetch them — and a reader that loses that race re-reads the row once rather than reporting a state that is there. What a publication uploaded and then could not commit is dropped on the way out. The write's guard names the VERSION rather than the path: the live dbt script there must be the one this job ran, or a later version of it. "Some live dbt script is here" is also satisfied by a script created at a path this one was renamed away from, and this job's manifest would then become that project's deferral state. A preview names no version and so publishes nothing. A `show` defers too. It compiles the model it previews, so a model whose upstream this environment built and this run did not is exactly the case a deferral exists for, and every engine takes the flags on it. Three comments said "the workspace's object storage" where the code deliberately uses the instance's, which is the whole security argument; `mib()` labelled MiB values MB. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix: hold the script row across a dbt state publication, and let a rename move it The version guard read `script` without a lock, so lifecycle cleanup could find no environment row to clear, finish, and leave this transaction to commit state at a path a new script goes on to occupy. It now holds that row (`FOR SHARE`) for the rest of the publication — taken before the sidecar, the order every other dbt writer takes — and the artifacts are uploaded before the transaction, so the lock covers the row work rather than a network round trip. A commit that reports an error may still have committed: what was lost can be the acknowledgement. Dropping this run's objects then leaves the committed row naming objects that are gone, so an orphan is the cheaper side to take. A failed second upload left the manifest it had already written behind; it is dropped now. Per-publication keys retired the reason a rename cleared the environment state rather than moving it: the path is only a prefix, and the row is what names an artifact, so a script created at the old path can no longer publish over a moved row. The rename moves both halves again. `dbt ls` gets the deferral flags too, without which a `result:` selector — which reads `run_results.json` out of the state directory, and which `select` passes to dbt verbatim — fails before the build that would have honoured it. Also: the migration was the last site describing the workspace's object storage rather than the instance's, `publication_lock` folded 32 bits where it claimed 64, and `ResolvedProfile` had taken `write_profiles`'s doc block. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix: a deferring dbt run never publishes the state it read `publishes_ownership` reads the CALLER's overrides, so a descriptor that already narrows `select` needs none and a run of it with `defer: true` published. A deferring run built some of the relations its manifest names and resolved the rest out of the state it read, so recording that manifest claims relations nothing built — and a model renamed since is recorded under a name only a full build creates, breaking every later deferral until one repairs it. Also: `publication_lock` parsed 16 hex digits as `i64`, which overflows for every digest with the top bit set — half of them — collapsing those environments onto one advisory key; and a failure to open the transaction returned without dropping the objects already uploaded. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix: only a deployed dbt run publishes state, and key its objects per execution A preview carries a caller-supplied `script_hash` into `runnable_id` (`run_preview_script`), so the version guard alone let anyone who may run a job publish arbitrary content as a deployed script's deferral state. The job's KIND is checked beside it now. Verified: a preview submitted with the deployed path and hash builds and leaves the row untouched. Object keys carry a per-execution nonce. Zombie recovery re-runs a job under its own id, so keyed on that alone a second attempt overwrote the objects the first attempt's committed row still named, then read those same keys back as displaced and dropped them — leaving the row unreadable. The displaced set is also filtered against this publication's own keys, so the invariant is stated rather than re-derived from the key format. A project-owned `profiles.yml` that templates its schema or database is refused a deferral: dbt renders those and Windmill does not, so two renderings resolve to one `relation_root` and would share one environment key. Plainly absent is left alone — that is the adapter's default, which does not move. The deferral log line now says the run publishes no state of its own, which was otherwise invisible. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix: a templated profile location publishes no dbt state either, on every path A `dbt_profile` resource is one block of the user's own `profiles.yml` copied through unchanged, and `profile.schema` is written as given, so either can carry a template dbt renders and this runtime does not — exactly as a project-owned file can. Only the project-owned path detected it. And the refusal now covers publication as well as deferral: a published template would sit under a key a literal profile shares, so de-templating later would make that stale manifest readable as the new location's. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix: recognise Jinja statement blocks as a rendered dbt profile location dbt renders a profile through Jinja, so `{% if env_var('ENV') == 'prod' %}…{% endif %}` moves a schema exactly as an `env_var()` substitution does — and only `{{` was detected, so such a profile published and deferred under one environment key for every rendering. One predicate now serves both profile paths, with a test for each delimiter. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix: a dbt state read outruns successive publications rather than one The loader re-read once, which answers a single publication overtaking it: a reader takes no lock and the advisory lock is released before the displaced objects are dropped, so back-to-back publications could each overtake the same read and the second was reported as a missing object. It now re-reads for as long as the row keeps MOVING, bounded, and reports only when an unmoved row's objects are genuinely gone. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * docs: a dbt state read outruns successive publications, not one Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * docs: name both ways a dbt state read can fail Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix: length-prefix the dbt environment key's components A dbt target name and a schema are both the user's own strings, so joining them on `|` let one component spell another tuple's key: `prod|analytics` + `scratch` and `prod` + `analytics|scratch` were one environment, and a profile moving between them read as the same one rather than as one nothing has published — the collision the key exists to prevent. The schema and database are also taken apart now rather than through `relation_root`'s own join, so neither can absorb the other's delimiter. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix: name the dbt environment in words where a message shows it The key is length-prefixed for storage, which is not something to put in front of a caller: the "nothing published yet" refusal now reads "warehouse `main`, target `prod`, relations in `dbt_wh_defer.analytics`". The worked example of the encoding also miscounted a component. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix: delete a script version in the transaction that cleans up after it `delete_script_by_hash` soft-deleted through the pool, committing before the cleanup that follows it in `tx`. In that window the path has no live version, so a concurrent deploy can take it — and `clear_dbt_script_state_if_path_retired` then finds that new script live, keeps the deleted project's dbt state, and leaves the replacement able to defer through its manifest. The update moves into the same transaction, which is what `archive_script_by_hash` beside it already does. The retirement guard itself was pinned by nothing: the existing test moved the only row away before calling the conditional clear, so it could not fail. `state_goes_only_once_no_live_version_is_left` covers both directions — a second live version keeps the state, the last one leaving takes it — and fails if the predicate is inverted. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix: archive a script by path in the transaction that cleans up after it The last of the four routes still writing outside its own cleanup transaction. Archived on its own, a cleanup that then fails leaves dbt state at a path no live version occupies, and whatever is created there next can defer through it. The by-hash archive and both deletes already take their write in `tx`; this makes the set uniform. Two comments beside those clears still called the state the RETRY state alone, which the rename made false — they cover both halves now — and the merged verification list had two `11.`, main's #10978 having inserted an item above it. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * feat: refuse a dbt state selector the engines resolve inconsistently `state:`, `result:` and `source_status:` selectors resolve against the artifacts in `--state`, which only a deferring run is handed. The engines disagree about what happens without one, and two of the three disagree silently: dbt-core 1.x raises, but dbt-sa-cli 2.x and fusion read a missing state as an empty one and exit 0, so `state:modified` builds nothing and `state:new` builds the whole project, each reporting success. Refuse them up front instead, naming `defer`. From the descriptor they are refused outright, since that selection also decides which nodes the script owns and the deploy resolves it with no state at all. `source_status:` is refused under any setting: it compares `sources.json`, which no run publishes here. A caller's selection is now allowed to match nothing, which is what `state:modified+` returns when nothing changed since the published state. It is stored as that run's own snapshot and never becomes what the script owns, so the ownership-wipe the refusal guarded against cannot happen. The descriptor's selection still may not. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix: refuse a dbt result selector the published state cannot answer Round 18 findings. Codex P1: `defer` alone was enough to allow a `result:` selector, but a build recovered by node retry publishes a manifest with no `run_results.json` — the only file such a selector reads. dbt-core then raises an internal error and the Rust engines match nothing and exit 0. The deferral now reports whether the state carries results, and a `result:` selection against one that does not is refused, naming the run that published it. Claude P2: a `parse` returns before `defer` is read, so its deferral is always absent and "turn `defer` on" was advice that led nowhere. The check now distinguishes a run that could defer from a command that never does, and the parse path says so. Codex P2 / Claude P2: the roadmap still listed `state:modified` as out of scope while the same file documented it as working. Narrowed both that line and the scope list to the slim-CI work that genuinely remains. Also pins the invariant the relaxed empty-selection guard rests on: an overridden selection must not publish ownership, or an empty caller selection would wipe the script's graph. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix: exempt an empty dbt selection by method, not by who chose it Round 19 findings. Codex P1: the empty-selection exemption keyed on whether the caller overrode the selection, so a misspelled model name resolved to nothing, passed the guard and reported a build that did its work. Key it on the selector instead: only a `state:` or `result:` method may match nothing, its empty answer being a real one. Every other selection matching nothing is refused again, from a run as from the descriptor, each with the message that applies to it. Claude P2: the spec still described a node-retry-recovered publication as one where `result:` selectors merely lose their input, which the previous commit stopped being true, and the section stating the selector rules recorded neither the `result:`-without-results refusal nor the `parse` one. Both written down. Also drops the refusal's claim that the publishing run WAS recovered by node retry: an unreadable file reaches the same absent-results state, and the remedy is the same either way. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * docs: record why an exempted empty dbt selection cannot wipe the graph The safety argument left with the origin-based condition it justified. Under the method-based one it is a consequence of the descriptor refusal in check_state_selectors, two hops from this site, so state it here: relaxing that refusal would let a descriptor-narrowed `state:modified+` reach the exemption and be ingested as owning nothing. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>	2026-09-08 08:32:30 +02:00
Ruben FiszelandClaude Opus 5	8d0f4754e4	fix: let a draft-only schedule, trigger or resource be deleted (#11010) ... * fix: let a draft-only schedule, trigger or resource be deleted Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_012SV5kjTis3AFtTx2nW2VRi * fix: keep the legacy-draft write gate out of the draft-only delete Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_012SV5kjTis3AFtTx2nW2VRi * fix: don't gate a draft-only resource discard on the deployment rules Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_012SV5kjTis3AFtTx2nW2VRi * docs: condense the draft-only delete comments per the comment policy Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_012SV5kjTis3AFtTx2nW2VRi --------- Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>	2026-09-07 18:55:40 +02:00
Ruben FiszelandClaude Opus 5	c6e0302d7c	feat: let // materialize declare a dbt:// warehouse-relation write (#10978) ... * feat: let `// materialize` declare a `dbt://` warehouse-relation write `// materialize manual dbt://<warehouse>/<schema>/<name>` lets an ingestion script in any language declare that it writes a warehouse relation, so it and the dbt model reading that relation land on one asset node instead of two disconnected pictures. `manual` is the only mode a warehouse target has — nothing generates warehouse DDL — and the non-`manual` spelling is refused rather than silently degraded. The `<warehouse>` segment is resolved against the workspace's configured warehouses, like a descriptor's `profile.warehouse`. The run records the same `materialized_partition` row a DuckLake target does, from the generic job path rather than an executor: the DuckLake write engine is DuckDB's, this declaration is anyone's. With a non-dbt producer now possible, the blanket deploy-time refusal of `# on dbt://<relation>` narrows to the shape that still cannot fire — every writer of the relation being a dbt script, since a dbt run does not dispatch. "Nothing produces it yet" stays accepted, as for every other asset kind, so deploy order does not matter. A dbt script may not subscribe at all: its graph ingest clears its own `dbt://` trigger rows. The one ordering the deploy cannot catch — a subscription accepted before any producer, then claimed by a dbt project — is named in that project's deploy log. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Rw1WrKeRRzyYHjfkuB83ek * fix: address review — preview stamping, stale producer set, public doc Three findings from the local review round: - Record the warehouse write only for a DEPLOYED script job. The annotation is a deploy-time contract (`manual`, three segments, a configured warehouse) checked where write access to the path is also required; honouring it in a preview, hub or inline-flow body let `jobs:run` alone restamp any relation's last writer from a script that never touched it. - Exclude the deploying script's own rows from the producer set. Read committed, they describe the version being replaced, so a script dropping its `// materialize` while adding a subscription counted itself as the producer that would wake it and committed a dormant edge. It could not be that producer anyway — the dispatcher skips self-loops. - `AssetKind::Dbt`'s doc no longer claims dbt is the exclusive producer of a warehouse relation, on both the types and the parser enum. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix: review round 1 — dbt-script materialize, set-form rule, doc - Refuse `// materialize` on a dbt script, the producer half of the rule the trigger loop already applies to `// on`: the graph ingest republishes that path's asset rows wholesale, so a declared write is wiped by the deploy that accepted it while its runs keep stamping the relation. - `dormant_dbt_subscriptions` now spells the same predicate its singular sibling does: the producer set has to be non-empty (nothing produces it yet is deploy order, not a dormant edge) and excludes the subscriber's own path (a script never wakes itself). Both divergences are pinned by tests. - The docs no longer claim the dbt deploy log covers a native producer that drops its `// materialize`; it does not, and nothing else reports that case. - An integration test over the deploy contract, since only a real deploy proves the handler feeds `sole_dbt_producer` the canonical key `asset.path` holds — the spelling that has to agree across the materialize target, the `// on` ref and the refusal that joins them. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * docs: qualify the any-language claim, and pin the dbt-script refusal `AssetKind::Dbt`'s contract (both enums), the two runtime guides and the deploy comment said a script of any language may declare a `dbt://` write, which the dbt-script refusal added last round contradicts. They now say "any language but dbt's own", with the reason: a project's writes are read from its manifest. The deploy-contract integration test covers that refusal for both annotations. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * docs: teach the pipeline AI guidance the warehouse-relation target The pipeline prompt (both sources, plus the regenerated bundle) told the model `// materialize` is DuckDB-only and rejected on any other target, which now steers users away from the very thing this PR adds. It distinguishes the managed DuckLake write, still DuckDB-only, from the warehouse-relation declaration any language but dbt's own may make. `dbt_manifest.rs`'s module doc carried the same "the only thing that creates one" overclaim the other four sites lost last commit. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix: draw an explicit dbt:// subscription on the canvas The editor suppressed every `// on dbt://…` overlay, which was right while the deploy refused all of them. It now refuses only a relation dbt alone builds, so the suppression hid the author's own annotation for exactly the case this PR adds — a subscription woken by a native `// materialize manual dbt://…` producer. The deploy stays the gate. Also the two stale claims round 4 named: the live pipeline prompt dropped the dbt-script exception the base prompt carries, and the doc's e2e requirements still said every `dbt://` subscription is refused. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix: refuse `// data_test` beside a `dbt://` materialize target `// data_test` checks are verifier probes the DuckDB executor splices around a managed write. A warehouse relation is written by the script itself, in any language, so nothing would run them — and unlike the DuckLake `manual` case, which at least fails loudly in that executor, a declarer in another language deployed green with its data-quality assertions silently skipped. Covered in the deploy-contract test and documented beside the annotation. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix: exclude a renamed producer from the sole-dbt producer set The producer set already excluded the deploying script's own path, because its committed rows describe the version being replaced. Under a rename the write sits at the OLD path — still committed, and removed by the same uncommitted transaction — so a producer renamed while it drops its `// materialize` and adds `// on dbt://…` still counted as the producer that would wake it, and committed a dormant edge. The deploy-contract test covers it: without the exclusion the rename deploys 201 instead of being refused. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * test: take the rename test's parent hash from the create response `format!("{:x}", …)` over the stored i64 drops leading zeros, while `ScriptHash`'s deserializer hex-decodes and demands 8 bytes — so a hash below 2^60 would 422 the request instead of reaching the refusal it asserts on, on roughly one in sixteen spellings of that script body. The create response already carries the zero-padded form, as the rest of the suite uses. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * docs: state the concurrent-ingest interleaving honestly `sole_dbt_producer`'s doc claimed the concurrent-deploy race only ever resolves toward refusing. It does when the uncommitted producer is native; when it is the dbt ingest, the check sees an empty producer set and accepts, and if that ingest then commits and runs its warning query before the subscriber's trigger row lands, neither side reports the dormant edge. Not serialized: the two would have to share a per-relation lock, and the ingest takes `script … FOR UPDATE` before its own advisory lock, so a deploy holding relation locks first inverts that order into a cross-subsystem deadlock — a worse failure than the cosmetic edge. Recorded beside the other orphaning the deploy cannot catch, with the bound both share: the next deploy of that project warns. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix: refuse a `dbt://` subscription that is not a whole relation `# on dbt://main/analytics` deployed and persisted a trigger row. Every producer spells `<warehouse>/<schema>/<name>` — the manifest ingest derives it from `relation_name`, a `// materialize` target is checked against it — so a partial one is an edge nothing can ever wake, which is what the dbt-only refusal exists to prevent. The shape now has one definition (`is_full_relation_path`) that both halves of the deploy ask, rather than a segment count spelled twice: a subscription and a write that disagreed would refuse and accept the same string. Also rewrites the canvas test's comment as a current constraint per AGENTS.md. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix: hold both halves of the deploy to one `dbt://` relation validator A subscription checked the relation's shape but not its warehouse, so `# on dbt://<unconfigured>/<schema>/<name>` deployed and persisted a trigger row for something no producer can ever write: the write side refuses that exact string, and a dbt project's `profile.warehouse` resolves against the same config, so no later deploy fixes it and the dormant-edge warning cannot report it either. The shape rule and the warehouse rule now live in one `validate_dbt_relation` that both halves call, rather than being spelled per site — the previous two rounds each closed one half of one rule, which is the drift that invites. Also moves the parser test out from between a comment and the test it documents, and names both refusals in the doc's list. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix: drop the subscription-only clause from the shared refusal message "so nothing can produce it" reads backwards on the `// materialize` side, which is the producer. The remaining sentence says what is wrong on both. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix: bound a `dbt://` relation by the asset-path column in the shared validator `asset.path` is VARCHAR(255) and the manifest ingest drops a relation that outgrows it rather than failing the whole graph, so past the column no producer row can exist on either side. `script_trigger.trigger_ref` is unbounded text, so an overlong subscription deployed and stayed dormant for good; an overlong write reached Postgres and failed the deploy on a `value too long` instead of a message. Both now refuse in the validator the two halves share, against the ingest's own constant. The integration case computes the ref from that constant so it cannot drift back under the bound. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix: report a warehouse-lookup failure as the failure it is, and correct the boundary `dbt_warehouse_exists` fails three ways — no such warehouse, the query itself, and a setting with no `resource_path` — and all three became a 400 blaming the user's warehouse name. A pool timeout mid-deploy told a retrying sync that a transient server error was a permanent client one. Only `NotFound` is the annotation's fault now. The known-boundary paragraph claimed a flow-runner run still cascades. It does not: it is routed by `flow_step_id`, which `is_eligible_kind` rejects, as `asset_trigger_dispatch.rs` pins. Recording and cascading are decided separately, so the paragraph now names all three routes rather than merging two of them — and the row it omitted, an ordinary flow step, which records and never cascades. E2E item 7 said "deployable" where the rule is "wakeable": with only the dbt project reading the relation the producer set is empty, which deploys fine. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * docs: correct two rationales the last commit got wrong `Error::SqlErr` already maps to 400 in this codebase, so the query case's status was never the thing at stake. What the `NotFound` match earns is that a query failure and a malformed setting stop being described as an unconfigured warehouse name, and that the malformed-setting `InternalErr` reaches its own 500 instead of being flattened. And a flow step is two shapes, not one: a step running a deployed script is a `Script` job that records and never cascades, while a step with an inline body is `FlowScript`, which the recording guard excludes along with previews. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix: warn about dormant subscriptions from the run that publishes ownership too A run whose static descriptor finds its profile moved re-ingests the version's graph and republishes path ownership, exactly as a deploy does — so it can be what leaves a subscription accepted while the relation had no producer with dbt as its only one. That path discarded `persist_ingest`'s result and emitted no warning, which also made the doc's enumeration of unreported orphanings wrong. Both ownership-publishing points warn now. An agent worker still cannot: it reaches these tables only through the API and its ingest publishes without reading back, which the doc now says. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * docs: an agent run publishes no ownership, and the warning has two callers The agent-worker sentence called it an exception that publishes ownership without warning. It publishes none: `Connection::Http` forces per-run models, and `publishes_ownership()` is the negation of that, so an agent stores a job-pinned snapshot and leaves workspace ownership with the deployed graph — it cannot orphan a subscription at all. `warn_dormant_subscribers`' own doc still named the deploy log as the only place the warning shows, one commit after it gained its second caller. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * docs: stop the managed-write rule from contradicting the dbt:// target The sentence after the warehouse-relation paragraph says `// materialize` means the runtime writes the table for you and the body is a bare SELECT. That is the managed DuckLake rule, written before a `dbt://` target existed, and unqualified it tells the model the opposite of what the paragraph above it just said — a model following the more prominent one emits a SELECT for a warehouse relation, which deploys and then writes nothing. Both prompt sources now scope it, and both name the `// data_test` refusal beside a `dbt://` target, which the badge list advertised without the caveat. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>	2026-09-07 17:38:07 +02:00
3 people	8aab5034a6	feat: guest JWT entry for embedded apps (#10954) ... * feat: guest JWT entry for embedded apps (jwt_guest_) A second way in for a guest, alongside the signed-in guest session: a JWT the embedding customer's backend mints and signs, verified per request against a per-workspace key (a PEM public key or a JWKS URL), resolving to the same seatless guest identity confined to the one app its app_path claim names. Bearer prefix jwt_guest_, stateless (no token row). See PR #10954. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * feat: surface guest JWT as the embed method in the app deploy drawer The deploy drawer explained the secret-URL embed but not the guest JWT path, so the primary way to embed an app for a customer's own authenticated users was undiscoverable. For a guest-mode app with guests enabled, show how to mint a `jwt_guest_` token and append `guest.<jwt>` to the app URL, with a copyable iframe template pre-filled with this app's workspace_id and app_path, and a note that new guest emails are refused past the instance's free allowance (the live count is shown just above). Also log a guest JWT allowance refusal at warn, not info: the caller gets a bare 401 (the reason must not leak to an unauthenticated caller), so the log is the admin's signal that the instance hit its guest cap. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * fix: correct the guest JWT minting instructions in the embed block The block said "sign it with the workspace's guest JWT key", but that setting holds the public verification key. Clarify the keypair relationship (configure the public key or a JWKS URL in the workspace; sign with the matching private key), name the accepted algorithms (RS/PS/ES; HS* refused), and keep the required claims, so an embedder knows how to actually mint the token. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * feat: fall back to the instance JWT issuer for guest verification (off on cloud) A workspace with no guest key of its own now verifies guest JWTs against the instance issuer (JWT_EXT_JWKS_URL, already used by jwt_ext_), so an operator running one issuer configures it once. Verification and the guest grant are CE; granting a full login from that issuer stays EE (jwt_ext_, unchanged). Disabled under CLOUD_HOSTED, where one instance issuer must not be trusted to mint guests in every tenant's workspace — there the per-workspace key is the only source, which also stays the override everywhere. The workspace settings note (hidden on cloud) explains the fallback. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * fix: embed instructions cover both the workspace key and instance issuer The embed block said to set the workspace's guest JWT key; now it says Windmill verifies against the workspace key or, off cloud, the instance issuer (JWT_EXT_JWKS_URL) when no workspace key is set. The instance clause is hidden under isCloudHosted(). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * fix: show the guest JWT embed block only when Embed is toggled It belongs with the iframe snippet, not the plain-URL view, so gate it on embedMode alongside the guest-mode / guests-enabled checks. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * fix: trust the instance issuer in the guest fallback; refresh stale docs P1 (CI review): the fallback wrapped JWT_EXT_JWKS_URL as a workspace JwksUrl, so it hit validate_guest_jwks_url and was refused for http/private issuers unless ALLOW_PRIVATE_GUEST_JWKS_URLS was also set — a self-hosted internal issuer that works for jwt_ext_ failed for guests, though the UI says setting the env var is enough. fetch_jwks now fetches the instance issuer without the https/private restriction (matching the jwt_ext_ loader; it stays operator-trusted), while a workspace-admin URL is validated and pinned as before. All the size/key/URL bounds still apply to both. P2 (CI review): refresh the stale docs that said a missing workspace key always refuses a guest JWT — the module, bearer, key-source, and EditGuestJwtKey field docs now describe the workspace key with the off-cloud instance-issuer fallback. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * fix: fetch the trusted instance issuer like the jwt_ext_ loader P1 (CI review): the instance-issuer fetch skipped SSRF validation but still disabled redirects and default cert validation, so an instance issuer that works for jwt_ext_ through a redirect or an operator-approved self-signed cert failed the guest fallback. Fetch it with HTTP_CLIENT_PERMISSIVE (follows redirects, honors ACCEPT_INVALID_CERTS) — the same behavior jwt_ext_ has — while a workspace-admin URL stays validated, DNS-pinned and redirect-free. The body size cap still bounds both. P2 (CI review): the WorkspaceSettings field doc still said None/None means no JWT guests; it now names the off-cloud instance-issuer fallback. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * docs: schema summary + OpenAPI cover the guest JWT columns and fallback P2 (CI review): summarized_schema.txt was missing guest_activity.jwt_entry and the two workspace_settings guest-JWT key columns (required by docs/validation.md after a schema change). The edit_guest_jwt_key OpenAPI description now notes that clearing the workspace key falls back to the instance issuer (JWT_EXT_JWKS_URL) off cloud rather than necessarily stopping guest JWTs. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * fix: keep JWKS single-flight locks in a self-cleaning map, not a bounded cache P1 (CI review): JWKS_FETCH_LOCKS was a 200-entry quick_cache. Past 200 cold URLs it can evict a lock whose fetch is still in flight; the next request for that URL then mints a fresh lock and starts a second fetch, so cycling configured workspaces defeats single-flight and can storm the issuers. Replace it with a plain map guarded by a JwksFetchLock RAII handle that removes each entry once its last holder drops, so the map only ever holds the fetches in flight and never evicts an in-flight lock. Add a unit test pinning the shared-lock and self-cleaning invariants. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * chore: update ee-repo-ref to c2270eb5fe2d9f0968253e6b460c33186363f4e7 This commit updates the EE repository reference after PR #773 was merged in windmill-ee-private. Previous ee-repo-ref: 5a1d9dee34159512c0823fddcd3d096490edbcce New ee-repo-ref: c2270eb5fe2d9f0968253e6b460c33186363f4e7 Automated by sync-ee-ref workflow. --------- Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com> Co-authored-by: windmill-internal-app[bot] <windmill-internal-app[bot]@users.noreply.github.com>	2026-09-05 10:23:37 +00:00
3 people	fce635d3c4	feat: guest app execution mode, a role that takes no seat (#10929) ... * feat: guest app execution mode, a fourth role that takes no seat Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01BayTppRCstWX6qTf3LMco5 * fix: make the guest grant a server-minted label, not a declarable scope Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01BayTppRCstWX6qTf3LMco5 * chore: pin ee-repo-ref to the guest session companion branch Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01BayTppRCstWX6qTf3LMco5 * fix: close the relabel hole, guest embed tokens, read-path switch, custom-path entry Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01BayTppRCstWX6qTf3LMco5 * fix: guest tokens are not rescopable and guest embed tokens keep the sentinel Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01BayTppRCstWX6qTf3LMco5 * fix: guest-derived tokens share one constraint set; gate sign-in on guest discovery Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01BayTppRCstWX6qTf3LMco5 * fix: the label alone governs a guest; refuse guests with accounts; unserialize discovery Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01BayTppRCstWX6qTf3LMco5 * fix: guest discovery fails closed; SAML aborts if the guest cookie write fails Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01BayTppRCstWX6qTf3LMco5 * refactor: enforce the guest switch once at the auth door; sign-in for a guest of another app Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01BayTppRCstWX6qTf3LMco5 * fix: guest app-mode decided once at the on-behalf resolver; clear a stale guest session before offering another app's sign-in Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01BayTppRCstWX6qTf3LMco5 * fix: a guest may use anonymous apps; await the stale-session logout; trim comments Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01BayTppRCstWX6qTf3LMco5 * fix: a guest's path confinement waits for the app's mode, so anonymous apps stay open to it Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01BayTppRCstWX6qTf3LMco5 * fix: guest target survives http (Lax cookie), rides SAML RelayState; tell account holders on arrival * fix: a guest uses an anonymous app as itself; S3 uploads confined by app mode * fix: a guest upload needs an app policy; a missing app does not skip the confinement * fix: guests are gated on the Enterprise plan server-side; pin ee-repo-ref * fix: the guest plan gate fails closed on non-enterprise builds; settings report the effective switch * fix: guest controls read the plan, not the key; gate the guest tests on the features they need * docs: tighten the guest session invariant comments * feat: 100 free guests per 30 days, then a quarter seat each on Enterprise and a hard cap elsewhere; superadmin guest list; refusals reach the page Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01BayTppRCstWX6qTf3LMco5 * fix: the cap is exact, an account ends a guest session at the door, popups close, and guest mode survives the CLI round trip Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01BayTppRCstWX6qTf3LMco5 * feat: a superadmin switch over guests for the whole instance; the pre-existing-user flag keeps its meaning Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01BayTppRCstWX6qTf3LMco5 * fix: drop the dead guest-access helper, name the instance setting once, guests tab states, CE save order Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01BayTppRCstWX6qTf3LMco5 * fix: a guest app path is refused at the mint if it could widen the scope; the instance toggle waits for its reload Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01BayTppRCstWX6qTf3LMco5 * fix: guests stop at the launched-by-me job grant; canonical app paths at the mint and discovery; the toggle ends on the stored value Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01BayTppRCstWX6qTf3LMco5 * fix: only the scope grammar's own characters bar an app path from guests, refused at deploy as well as at the mint Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01BayTppRCstWX6qTf3LMco5 * fix: the deploy-time guest path guard checks the destination of a rename and refuses a leading slash Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01BayTppRCstWX6qTf3LMco5 * fix: a workspace rename keeps the guest switch; the rename guard reads the deployed mode under the row lock Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01BayTppRCstWX6qTf3LMco5 * fix: guest_activity follows a workspace rename and goes with a workspace delete Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01BayTppRCstWX6qTf3LMco5 * chore: pin ee-repo-ref to the state-bound guest target Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01BayTppRCstWX6qTf3LMco5 * chore: pin ee-repo-ref; the guest cookie is never cleared by a callback Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01BayTppRCstWX6qTf3LMco5 * docs: the workspace-scoped guest_activity delete moves an instance-wide count; assert the mint records the guest Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01BayTppRCstWX6qTf3LMco5 * test: the seeded allowance is a day old, so only the mint can write today's guest_activity row Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01BayTppRCstWX6qTf3LMco5 * chore: update ee-repo-ref to 1a10132e4f3cb442c7d0c2cf6e5d92d150bf6e07 This commit updates the EE repository reference after PR #769 was merged in windmill-ee-private. Previous ee-repo-ref: 32841072aa396bff91d30bd91854fa348cb3c439 New ee-repo-ref: 1a10132e4f3cb442c7d0c2cf6e5d92d150bf6e07 Automated by sync-ee-ref workflow. --------- Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com> Co-authored-by: windmill-internal-app[bot] <windmill-internal-app[bot]@users.noreply.github.com>	2026-09-04 22:47:28 +02:00
Ruben FiszelandClaude Fable 5.1	11138284ac	fix: deploy a relocked script version only when its lock changed (#10966) ... * fix: deploy a relocked script version only when its lock changed Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01GdEb6gzCZ2qXmAQJAeMf9W * fix: write the unchanged relock hash under the row lock and skip the phantom tally Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01GdEb6gzCZ2qXmAQJAeMf9W * fix: requeue a superseded relock and read the live head past the script cache Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01GdEb6gzCZ2qXmAQJAeMf9W * fix: re-read the relock head after waiting on its lock and keep module locks Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01GdEb6gzCZ2qXmAQJAeMf9W * fix: bound the relock head re-read instead of reading once Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01GdEb6gzCZ2qXmAQJAeMf9W * chore: refresh the sqlx cache entry for the re-indented lock write Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01GdEb6gzCZ2qXmAQJAeMf9W * test: pin the waiting-relock requeue and the multi-file importer no-op Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01GdEb6gzCZ2qXmAQJAeMf9W --------- Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>	2026-09-04 01:01:36 +02:00
3 people	79426a1a68	feat: reconcile IdP instance groups from the SSO groups claim (#10957) ... * feat: add sso_groups_claim setting for login-time instance group sync Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01YESxWqzt959S6TY6vbc4eG * chore: bump ee-repo-ref for the SSO groups claim reconcile Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01YESxWqzt959S6TY6vbc4eG * chore: update ee-repo-ref to 3b89bfc11314a326a191101cfe3ef65f6f7f82a8 This commit updates the EE repository reference after PR #774 was merged in windmill-ee-private. Previous ee-repo-ref: e388527f9adbbe466fe050ca8d1d236ce3342bc3 New ee-repo-ref: 3b89bfc11314a326a191101cfe3ef65f6f7f82a8 Automated by sync-ee-ref workflow. --------- Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com> Co-authored-by: windmill-internal-app[bot] <windmill-internal-app[bot]@users.noreply.github.com>	2026-09-03 22:23:43 +02:00
Ruben FiszelandClaude Fable 5.1	ca8800959a	fix: bump git sync hub scripts to cli 1.802.1, test the fork ui pull (#10955) ... * fix: bump git sync hub scripts to cli 1.802.1, test the fork ui pull Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_011yMLnAWdjpCEs5VyGMn9ww * test: guard the ui pull preview shape and pin the pull script ids together Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_011yMLnAWdjpCEs5VyGMn9ww --------- Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>	2026-09-03 11:23:23 +02:00
hugocasaandClaude Opus 5	d472193e5b	feat: add retention cleanup for the otel_traces table (#10949) ... * feat: add retention cleanup for the otel_traces table Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01NLhUaCPpLRAa29rSZDjS28 * fix: vacuum otel_traces and badge its retention setting EE Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01NLhUaCPpLRAa29rSZDjS28 --------- Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>	2026-09-02 19:41:46 +02:00
3 people	94af8d0fb5	fix: let a principal without a login account own a draft (#10925) ... * fix: let a principal without a login account own a draft Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_018Lu3hExEDPZu2dAEhZDVAi * fix: keep an accountless draft owner from colliding or reading as legacy Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_018Lu3hExEDPZu2dAEhZDVAi * fix: drop the unnameable draft owner everywhere and guard the no-op rename Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_018Lu3hExEDPZu2dAEhZDVAi * fix: drop the unused Acquire import in the draft rename test Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_018Lu3hExEDPZu2dAEhZDVAi * docs: drop the stale draft_users claim from the fork-clone rationale Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_018Lu3hExEDPZu2dAEhZDVAi * chore: update ee-repo-ref to f5b783d2f7608e1ff3a817caa8b719e06f8b8981 This commit updates the EE repository reference after PR #768 was merged in windmill-ee-private. Previous ee-repo-ref: f3dba016e9274ee9bbe46b4f070d3ed29843e5fd New ee-repo-ref: f5b783d2f7608e1ff3a817caa8b719e06f8b8981 Automated by sync-ee-ref workflow. --------- Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com> Co-authored-by: windmill-internal-app[bot] <windmill-internal-app[bot]@users.noreply.github.com>	2026-09-01 23:42:08 +02:00
3 people	815de49e23	feat: make the service log retention period an instance setting (#10889) ... * feat: make the service log retention period an instance setting Service log retention was a hardcoded 14 days with no override, unlike job retention. It becomes the `service_log_retention_secs` global setting (env `SERVICE_LOG_RETENTION_SECS`, default unchanged at 14 days), reloaded on change like the other retention settings. The constant becomes `DEFAULT_SERVICE_LOG_RETENTION_SECS` and every reader goes through `service_log_retention_secs()`, so the `log_file` sweep, the object-storage orphan scan, the columnar store's compaction and pruning, the retrieval clamp and the search index's trim window all follow the configured value. Loaded outside `initial_load`'s `server_mode` guard: a dedicated indexer trims the search index to a window derived from this value and is not a server. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01WsnpNSM6K3oyjwntRwJtVN * fix: never let a non-positive service log retention expire every log Every service log cutoff is `now - retention`, so a `0` or negative window puts the cutoff at or after `now` and the next sweep reads the whole history as expired — deleting the `log_file` rows and their object-storage files irreversibly. `0` is reachable two ways now that the window is configurable: it is what an operator types by analogy with the job retention period sitting directly above it, where `0` does mean keep forever; and `SecondsInput` writes a `0` into a field that was merely focused, so saving the Jobs panel is enough. Service logs always have a window, so clamp an unusable value back to the default in the accessor every reader already goes through. The upper bound is where `chrono::Duration::seconds` panics, which would abort the sweep that reads it. The settings field rejects a non-positive value rather than silently correcting it, and its description now names the database rows too — they are swept on every instance, including one with no object storage configured. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01WsnpNSM6K3oyjwntRwJtVN * fix: address review findings on the service log retention setting - Bound the monitor's `log_file` sweep. Every process rotates a log file a minute, so lowering the retention can make one ordinary setting change expire millions of rows; the unbounded `DELETE ... RETURNING` materialized all of them, and their deletion futures, in a single tick. Batched like the settings-page cleanup on the same table. - Make the retention atomic private and give it one writer, so a value that would expire every service log cannot reach a cutoff by any path, and say so in the log when one is rejected rather than falling back silently. - Cap the retention at a century. The previous ceiling only bounded `TimeDelta` construction, while consumers compute `now - retention`, which panics past year 262143, and build a Postgres interval that overflows well before the old cap. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01WsnpNSM6K3oyjwntRwJtVN * fix: cap an oversized service log retention instead of shortening it The two unusable directions were landing on the same fallback, so configuring a retention above the ceiling silently produced 14 days — deleting logs the operator had asked to keep for longer. Too large now caps at the maximum, which preserves that intent; only a non-positive value, which would expire everything and has no upward reading, falls back to the default. Also bound the `log_file` drain to ten batches per pass: `monitor_db` runs under a 600s timeout that cancels every maintenance future in the same `join!` and reports a critical error, so a backlog large enough to need batching has to drain across ticks, the way the neighbouring sweeps already do. The settings field carries the upper bound too, and the superseded query's offline entry is dropped. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01WsnpNSM6K3oyjwntRwJtVN * fix: route the new log-file registration cutoff through the retention accessor `send_log_files_to_object_store` arrived on main while this branch was open and reads the retention directly. The atomic behind it is private now, so it goes through the accessor like every other consumer — which also means the cutoff it uses to skip registering already-expired files follows the configured retention rather than a fixed two weeks. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01WsnpNSM6K3oyjwntRwJtVN * fix: say why every mode loads the service log retention setting A worker registers its rotated log files against the retention cutoff, so the comment naming only the indexer no longer covers why the setting sits outside the `server_mode` guard. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01WsnpNSM6K3oyjwntRwJtVN * fix: file service log retention under Monitoring, not Jobs Service logs are the Windmill processes' own logs — every process rotates and registers its own, no job involved — so the Jobs panel was grouping by the shape of the widget rather than by the subject. It sits under Monitoring now, beside the Indexer panel that holds the other service-log window. Its own section rather than inside that panel: the panel is badged EE, while this governs the database sweep that runs on every instance. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01WsnpNSM6K3oyjwntRwJtVN * chore: update ee-repo-ref to a6e3533b26195918a17fea58646f71d2bbcde288 This commit updates the EE repository reference after PR #752 was merged in windmill-ee-private. Previous ee-repo-ref: 1d93da24bd166b9a5a5cc204034a1d35ffc88474 New ee-repo-ref: a6e3533b26195918a17fea58646f71d2bbcde288 Automated by sync-ee-ref workflow. * feat: say on the service logs page where the logs actually are The retention number alone does not tell an operator what it governs, and the answer differs by instance. Two states are worth calling out because they are the ones where retention does not mean what it looks like: Without instance object storage, each process keeps its files on its own disk. The page lists what every host wrote, since the rows are in the shared database, but can only open the files of the replica serving the request, and a host's files go with it when it is replaced. With object storage but "Delete logs from s3 periodically" off — the backend default, since uploads are gated on a store existing while deletions are gated on that toggle — expiring a log removes the row and the local file and leaves the uploaded copy behind for good. The retention field itself now names every copy it covers and says that full-text search reaches back at most that far, and less when the indexer's own window is shorter. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01WsnpNSM6K3oyjwntRwJtVN * fix: describe raw log files as the transient copy they became Retiring the raw files landed while this was being written: the indexer now deletes each one as soon as it is ingested, and the log viewer rebuilds a file from the columnar store once the raw copy is gone. So the durable copy is the store, and warning that an uploaded file is kept forever when periodic s3 deletion is off only holds where no indexer runs to ingest it. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01WsnpNSM6K3oyjwntRwJtVN * chore: point ee-repo-ref at the EE compile fix EE main does not build on its own: extracting the index-window expression and adding a fourth copy of it landed in separate PRs that never conflicted textually. windmill-ee-private#756 is the one-line fix; this pins it so CI has a tree that compiles. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01WsnpNSM6K3oyjwntRwJtVN --------- Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com> Co-authored-by: windmill-internal-app[bot] <windmill-internal-app[bot]@users.noreply.github.com>	2026-08-29 19:39:14 +02:00
3 people	7c1a785f75	feat: serve service log retrieval from a columnar parquet store (#10886) ... * feat: always write service log files as json so they index structured Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_016ijGGPCFkYhVzisFexAHYx * feat: serve service log retrieval from a columnar parquet store Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_016ijGGPCFkYhVzisFexAHYx * feat: shrink the service log index to the per-host count it still serves Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_016ijGGPCFkYhVzisFexAHYx * fix: reclaim the superseded service log index on upgrade Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_016ijGGPCFkYhVzisFexAHYx * fix: address review findings in the service log store Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_016ijGGPCFkYhVzisFexAHYx * chore: update ee-repo-ref to ad9e899dfd2ee4e3d18ecf06d016f821968c5a83 This commit updates the EE repository reference after PR #751 was merged in windmill-ee-private. Previous ee-repo-ref: 6ad4064f9d58d83612b42b4ec870384994d64bcb New ee-repo-ref: ad9e899dfd2ee4e3d18ecf06d016f821968c5a83 Automated by sync-ee-ref workflow. * fix: address review nits on the service log store Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_016ijGGPCFkYhVzisFexAHYx --------- Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com> Co-authored-by: windmill-internal-app[bot] <windmill-internal-app[bot]@users.noreply.github.com>	2026-08-29 09:51:59 +02:00
3 people	7dd88c470c	fix: unify billable seat counting and prevent fork subscriptions (#10818) ... * fix: unify billable seat counting and prevent fork subscriptions * fix: authorize candidate before reading its plan, scope seat breakdown * chore: pin ee ref for the stripe checkout fork guard * fix: grant the billable_member view and widen the paid-plan check * refactor: keep the seat rule in rust instead of a view and function * docs: correct the attach guard summary after widening the plan check * revert: keep cloud out of the ci test feature set * chore: update ee-repo-ref to 9ff97cd818e85940fec282c92161e98c1b8583e2 This commit updates the EE repository reference after PR #742 was merged in windmill-ee-private. Previous ee-repo-ref: 0ec0b42565a41f271a45bf24a93467d110c36df3 New ee-repo-ref: 9ff97cd818e85940fec282c92161e98c1b8583e2 Automated by sync-ee-ref workflow. --------- Co-authored-by: windmill-internal-app[bot] <windmill-internal-app[bot]@users.noreply.github.com> Co-authored-by: Ruben Fiszel <ruben@windmill.dev>	2026-08-28 17:13:02 +02:00
Ruben FiszelandClaude Opus 5	69320b28f6	perf: index the suspended-job resume test instead of filtering it (#10863) ... * perf: index the suspended-job resume test instead of filtering it Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01SEUq14Wz4cC2NzcRyo6CNj * fix: keep the legacy suspended index until the replacement is recorded Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01SEUq14Wz4cC2NzcRyo6CNj * perf: drop the redundant suspend_until column from the suspended index Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01SEUq14Wz4cC2NzcRyo6CNj --------- Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>	2026-08-27 00:39:03 +02:00
3 people	8b80b09f33	fix: restrict filesystem workspace storage to debug builds (#10864) ... * fix: restrict filesystem workspace storage to debug builds Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Q7p2VbtYqaXHGaAskgwVk5 * chore: update ee-repo-ref to b58ad414b098d3d7787001a352bfbb13e43a335f This commit updates the EE repository reference after PR #747 was merged in windmill-ee-private. Previous ee-repo-ref: 1b4dada77a8fe2224579c643550c63b1ac2616de New ee-repo-ref: b58ad414b098d3d7787001a352bfbb13e43a335f Automated by sync-ee-ref workflow. --------- Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com> Co-authored-by: windmill-internal-app[bot] <windmill-internal-app[bot]@users.noreply.github.com>	2026-08-26 23:35:44 +02:00
Ruben Fiszelandwindmill-internal-app[bot]	f131c3920f	fix: keep connection string query parameters under token auth (#10859) ... * fix: keep connection string query parameters under token auth * refactor: fold the database url parsing into one connect-options helper * docs: state the narrower invariant on base_connect_options * chore: update ee-repo-ref to 212cc7d61ec38580d4a70d9ac38d7a2cc9daf409 This commit updates the EE repository reference after PR #746 was merged in windmill-ee-private. Previous ee-repo-ref: a15d08345d7e42526c28382079ad1f575a2d1674 New ee-repo-ref: 212cc7d61ec38580d4a70d9ac38d7a2cc9daf409 Automated by sync-ee-ref workflow. --------- Co-authored-by: windmill-internal-app[bot] <windmill-internal-app[bot]@users.noreply.github.com>	2026-08-26 23:08:36 +02:00
hugocasaandClaude Opus 5	2906504125	feat: add instance setting to mute zombie job restart alerts (#10813) ... * feat: add instance setting to opt out of zombie job restart alerts Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * fix: preserve explicit false for default-on boolean instance settings Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * refactor: invert zombie restart alert setting to a mute flag Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 5 <noreply@anthropic.com>	2026-08-24 22:36:21 +02:00
Ruben FiszelandClaude Opus 5	4b406e37c0	fix: size the ephemeral job token to the job timeout it must serve (#10804) ... * fix: size job token to the premium cloud job timeout Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix: give the job token setup headroom and drop dead MAX_TIMEOUT_DURATION Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix: cap job token setup slack so self-hosted tokens stay at 7d Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>	2026-08-22 09:30:08 +00:00
Ruben FiszelandClaude Opus 5	dc0df45c81	chore: bump git-sync hub scripts to windmill-cli 1.794.0 (#10802) ... * test: assert the unpacked repo symlink without following it `unpack_keeps_a_link_that_stays_in_the_repo` read through the link it had just unpacked. Windows stores a symlink's target verbatim and its object manager rejects the `/` in a POSIX one, so `read_to_string` came back with `ERROR_INVALID_NAME` and the release's `cargo_test_windows` job was red. Pin what the function is responsible for on every platform — the link is kept and materialized — and read through it only where a POSIX relative target resolves. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01P3WRtxdKNGdomWX9vaAYGx * test: key the cli sync-map fixtures with the platform separator A sync map is keyed with the platform separator on both sides — `FSFSElement` walks the tree with `path.join`, and the remote `ZipFSElement` starts at `"." + SEP` and joins from there — while an `!inline` reference is always forward-slash. `lock_dedup.ts` follows that convention; the fixtures did not, so on Windows they built a map shape the CLI never produces and 12 of them failed. `getTypeStrFromPath` is the same story: it matches `"dependencies" + SEP`, and the test handed it a forward-slashed path. Build the fixture keys through the separator, leaving the `!inline` references and the `present` map forward-slash, as `sync.ts` hands them over. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01P3WRtxdKNGdomWX9vaAYGx * ci: skip the discord comment relay when the thread lookup returns none A rate-limited or unauthorized Discord response carries no thread list, and under `bash -e` that aborted the step — jq cannot iterate null, nor parse the HTML error page Cloudflare answers a 429 with — before it reached the "thread not found, skipping" branch right below. Three comment relays failed that way on the 1.794.0 head. Keep the step green for both, but tell them apart: a response with no thread list is a delivery that was dropped for a reason worth seeing, so it warns with the body it got, while a PR that genuinely has no thread stays quiet. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01P3WRtxdKNGdomWX9vaAYGx * chore: bump git-sync hub scripts to windmill-cli 1.794.0 --------- Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>	2026-08-21 13:26:49 +00:00
Ruben FiszelandClaude Opus 5	3c8e4b43fd	fix: resolve a script path to its new version as soon as the lock lands (#10794) ... * fix: resolve a script path to its new version as soon as the lock lands Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_011H5ygpzQHkPeYsjiP9GzBy * fix: tell MCP script deploy callers to stop polling on a lock error Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_011H5ygpzQHkPeYsjiP9GzBy --------- Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>	2026-08-21 11:54:51 +02:00
Ruben Fiszel	d85050f505	feat: upgrade bun to 1.4.0 and demote deno in the language picker (#10784) ... * chore: upgrade bun to 1.4.0 in dockerfiles and CI pins * chore: move deno last in the language picker and relabel it Deno * chore: move deno last in the pipeline language picker too * chore: pin debugger image to bun 1.4.0 and trim the deno picker comment * chore: state the deno picker constraint without referencing the old order * fix: stamp bun lockfiles back to v1 while the fleet predates bun 1.4 * fix: ask bun for a v1 lockfile instead of rewriting one, and refuse an escalated lock * chore: warn instead of silently storing a lockfile with no readable version	2026-08-21 01:16:59 +02:00
3 people	5099f405d4	feat: make the Git Repo Viewer work with GitHub App repositories (#10765) ... * fix: resolve the head commit of GitHub App repos in the git repo viewer `get_git_commit_hash` ran `git ls-remote` against the raw resource URL. A GitHub-App-backed repository stores a tokenless URL, so the probe failed with "could not read Username" and the viewer never got past its first step. Resolve the head over the GitHub REST API with a server-side installation token instead, reusing the lookup the auto-pull poller already uses for app repos. Non-app repositories keep the ls-remote path. Also picks up the EE-side allowlist fix that lets the clone hub script request an installation token. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * chore: update ee-repo-ref to 63c67e2a2db198af26a0334f5be14af7d9987eb1 This commit updates the EE repository reference after PR #732 was merged in windmill-ee-private. Previous ee-repo-ref: 2a260961fa0a9bb5631c17e2f718cb8efb4f9aa2 New ee-repo-ref: 63c67e2a2db198af26a0334f5be14af7d9987eb1 Automated by sync-ee-ref workflow. * fix: honour the app-repo head lookup's not-app-backed result `get_app_repo_head_for_autopull` documents `Ok(None)` as "this repo is not app-backed, use the ls-remote path", which is what the other two callers do. Fall through to `ls-remote` on `None` instead of turning it into a 500, and drop the handler's own `is_github_app` read now that the callee's answer is honoured. Also bumps ee-repo-ref to pick up route-safe ref handling in that lookup. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * feat: serve GitHub App repositories as an archive instead of a token The viewer's clone script asked the server for an installation token and put it in the clone URL. That token is installation-wide and carries the App's full permissions, so minting one requires a workspace admin, and the viewer was therefore admin-only for app-backed repositories. The server now streams a tarball of the commit instead, authorized by read access to the git_repository resource, so no GitHub credential reaches the job. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * feat: run delegate_to_git_repo playbooks from GitHub App repositories An Ansible job's runnable_path is the user's own script, which no entry in the git-sync script allowlist can match, so `delegate_to_git_repo` could never obtain a token for an app-backed repo. It also gave up entirely on agent workers, whose connection has no database to mint one from. A playbook run only reads a working tree: the clone is followed by one rev-parse for a log line, and nothing after that touches git. So take the same archive route the viewer uses, extracting the commit's tarball into the job's repository directory. No GitHub credential reaches the worker, and agent workers work because the route is HTTP. Archive entries are joined onto the target by hand so a crafted archive cannot write outside the job directory. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * fix: drop the now-immutable secret_url binding Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * feat: point the repo viewer at the archive-based clone script hub/28905 reads app-backed repositories through the server's archive route instead of minting an installation token, which the backend in this release no longer grants it. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * fix: stream repository archives to disk rather than into memory The archive download went through `AuthedClient::get`, whose client caps a request at 20 seconds and whose response was then buffered whole. A repository is arbitrarily large, so that cut off slow downloads and put every job on the worker at risk of running the process out of memory. Add `get_streaming`, the read counterpart to the streaming upload path, and write the response out chunk by chunk. Extraction now creates each entry's parent directory: a tar carries directory entries only by convention, and the traversal guard now has tests, one of which caught the missing parent. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * fix: require admin to read an app-backed repository A `git_repository` resource names the repository rather than holding a credential for it, so read access to one authorizes nothing: anyone who can write a resource path can point one at any repository the GitHub App installation reaches, then read their own resource. The head lookup now requires admin for app-backed repos, matching the archive route and the repository picker, which already limits itself to workspaces where the caller is an admin. Repos that aren't app-backed are untouched and stay open to any reader. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * docs: describe the repo viewer's hub script as it stands The file read as a patch waiting to be applied, against a hub version two releases stale. Describe what the published script does, including the archive route app-backed repositories now take. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * fix: run the archive fetch under the job poller, off the job directory Three defects in the delegate path's fetch: The download and extraction ran outside the job poller that the git clone paths go through, so a cancelled or timed-out run kept streaming and extracting an arbitrarily large repository while holding the worker. There is no wall-clock bound on the download itself, by design, which is exactly why it needs the poller. The archive was written to a fixed name inside the job directory, where `create_file_resources` has already laid down the run's own files at paths the playbook chooses. A run naming a file `repo_archive.tar.gz` had it truncated and then deleted. It goes to a per-job temp path now. Link entries were unpacked with their target unchecked. `Entry::unpack` writes the link verbatim, so a link out of the tree plus a later entry descending through it writes wherever it points. Targets now face the same containment check as entry paths. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * fix: keep repo symlinks, refuse only writes that go through them The link check rejected any target containing `..`, which is ordinary in a repository — `docs/x -> ../README.md` resolves inside the tree, and a git checkout keeps it. Rejecting it failed the whole extraction for repositories the clone path handles, and app-backed repos have no clone path to fall back to. Targets are preserved as git preserves them. What would let one escape is a later entry written at or underneath the link, so that is what is refused. Extraction also polls an abort flag now: a `spawn_blocking` task outlives the join handle its caller drops, so a cancelled job left it unpacking in the background. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * fix: refuse hard links in a repository archive Leaving link targets verbatim is right for symlinks — git checks them out that way, and an escape needs a second entry descending through the link, which is refused. A hard link is not like that: unpacking one creates it against a target resolved there and then, so an escaping target is useful on its own. No git tree can express a hard link, so an archive carrying one did not come from a repository. Refuse it. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * chore: update ee-repo-ref to 21f79bbbd39ae89665d1a89738630978616aa309 Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * chore: update ee-repo-ref to 37695a769b25d16b34107eedc1076793a8b388c8 This commit updates the EE repository reference after PR #737 was merged in windmill-ee-private. Previous ee-repo-ref: 21f79bbbd39ae89665d1a89738630978616aa309 New ee-repo-ref: 37695a769b25d16b34107eedc1076793a8b388c8 Automated by sync-ee-ref workflow. --------- Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com> Co-authored-by: windmill-internal-app[bot] <windmill-internal-app[bot]@users.noreply.github.com>	2026-08-20 22:05:24 +02:00
3 people	1f59841a67	feat: add WM_ROOT_WORKSPACE, the closest dev or prod workspace of a job (#10776) ... * feat: add WM_ROOT_WORKSPACE, the closest dev or prod workspace of a job Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01HBMJwogo6jJ1P55uvB3YpF * fix: do not cache a failed root-workspace lookup, and sweep on fork create Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01HBMJwogo6jJ1P55uvB3YpF * fix: shorten the agent-worker root-workspace TTL and pin the sweep wiring Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01HBMJwogo6jJ1P55uvB3YpF * chore: update ee-repo-ref to a2fa58e5301d3865dd06ad73519e20ba7a5af0f0 This commit updates the EE repository reference after PR #736 was merged in windmill-ee-private. Previous ee-repo-ref: 07a9d26a79a403ae27c48abd508a6699f2c87c49 New ee-repo-ref: a2fa58e5301d3865dd06ad73519e20ba7a5af0f0 Automated by sync-ee-ref workflow. --------- Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com> Co-authored-by: windmill-internal-app[bot] <windmill-internal-app[bot]@users.noreply.github.com>	2026-08-20 15:40:22 +02:00
Ruben FiszelandClaude Opus 5	f6645af77e	fix: explain the 6-field cron format when a schedule is rejected (#10768) ... * fix: explain the 6-field cron format when a schedule is rejected Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_019mUJd8ZRXkzbhmHkZryYoE * fix: phrase the cron hint as a prepend, not an equivalent schedule Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_019mUJd8ZRXkzbhmHkZryYoE * fix: withhold the cron example where v1 shifts the weekday Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_019mUJd8ZRXkzbhmHkZryYoE * fix: withhold the cron example for any restricted weekday on v1 Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_019mUJd8ZRXkzbhmHkZryYoE --------- Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>	2026-08-19 22:37:06 +02:00
4 people	c2deea13b7	fix(security): a WM_TOKEN job token can never be a global superadmin (GHSA-hfh4-cx4h-3fcr) (#10124) ... * fix(security): a WM_TOKEN job token can never be a global superadmin (GHSA-hfh4-cx4h-3fcr) Privilege escalation: an app/flow/schedule/trigger execution policy's `on_behalf_of` (which a `wm_deployers` member can set) could point at a superadmin email. The resulting job `WM_TOKEN` then passed the email-based superadmin checks, granting instance superadmin. `forbid_superadmin_job_token` only guarded ~15 of ~75 routes. Fix at the token layer: a WM_TOKEN must never satisfy a superadmin gate, regardless of whose email it runs as (sentinel OR a real superadmin). - `ApiAuthed` gains a `job_id` field, stamped once in `AuthCache::get_opt_job_authed` from the resolved token's job_id (correct even on cache hits). - `require_super_admin(db, email)` -> `require_super_admin(db, &ApiAuthed)`, rejects `authed.job_id.is_some()`. `require_super_admin_email` kept for the few internal callers without an ApiAuthed. - `is_super_admin_authed(db, &ApiAuthed)` for the boolean `is_super_admin_email` authorization branches on request handlers (workspace deletion, fork drops, dev-workspace attach/archive, object-storage SSRF exemption, custom dbname, EE GHES + connected repositories, ...). Migrate ~75 sites (OSS + EE). - CUSTOM_INSTANCE_DB reads the *authenticated* job_id, not the caller-supplied `?job_id` query param. Worker-tag check takes a precomputed job-aware `is_super_admin` on the request path. Execution-time on-behalf checks (scheduled/flow worker-tag, Cloud enqueue quota, is_devops_email) are hardened in a follow-up — see docs/followup-onbehalf-execution-privilege-hardening.md. Regression tests: a superadmin-email WM_TOKEN is rejected on `require_super_admin` routes, on `DELETE /workspaces/delete/{w}` (403, workspace preserved), and on the CUSTOM_INSTANCE_DB lookup with no `?job_id` (401); real superadmin tokens still succeed. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix: cap devops role at workspace admin and reject reserved on_behalf_of identities Extends the job-token cap with three pieces: - `require_devops_role` takes `&ApiAuthed` and rejects job tokens. `is_devops_email` is true for superadmin emails, so every worker-management, instance-config and service-log route was reachable by the same superadmin `WM_TOKEN` that `require_super_admin` already rejects. - A `job_id` claim that does not parse as a uuid rejects the token rather than resolving to `None`, which would clear the job provenance and uncap it. Applies to the internal JWT and the external `jwt_ext_` path. - Defense in depth at store time: `validate_on_behalf_of` refuses the reserved internal sentinels as an `on_behalf_of` on apps/flows/scripts/schedules/triggers, and app execution refuses a policy carrying one — covering already-persisted and forked-app rows that predate the cap. Deploying on behalf of a real user, including a real superadmin, stays allowed; the cap handles that at execution. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * fix(mcp): preserve job-token provenance when minting the proxy JWT The MCP endpoint-tool proxy re-mints a JWT from the caller's ApiAuthed to forward the proxied request, but passed job_id: None. A job's WM_TOKEN is capped at workspace admin (GHSA-hfh4-cx4h-3fcr); dropping the job_id here re-minted an uncapped token that satisfies require_super_admin / require_devops_role on the proxied route (e.g. listWorkers exposing worker IPs, job/workspace IDs, and sensitive tags). Carry api_authed.job_id into create_jwt_token. Adds an in-module regression that decodes the forwarded JWT and asserts the job_id is preserved for a job caller and absent for a non-job caller. Reported by Codex CI review (P1) on #10124. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * fix: cap the admin-or-devops gate at workspace admin for job tokens require_admin_or_devops (the EE critical-alerts endpoints) grants when the caller is a workspace admin OR an instance devops. is_devops_email is true for superadmins, so a WM_TOKEN running on-behalf of a superadmin who is not a member of the target workspace could clear the devops branch and read/ack that workspace's critical alerts (GHSA-hfh4-cx4h-3fcr). This gate takes a bare email, not an ApiAuthed, so the token-layer cap could not see it. Thread the caller's job-token provenance and reject the devops branch for job tokens, matching require_devops_role. The workspace-admin branch stays allowed — that is the cap ceiling. Adds an enterprise-gated regression proving the bypass is closed and a real superadmin token still clears the gate. Found while auditing the PR for bare-email gates the choke-point cap misses. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * fix: cap instance-global is_admin gates at workspace admin for job tokens Three instance-global routes gate on the caller's own `is_admin` claim, which `ApiAuthed.is_admin` carries into a WM_TOKEN (it is a workspace-admin claim, true for superadmins too). A job token is capped at workspace admin (GHSA-hfh4-cx4h-3fcr), so its is_admin claim must not authorize instance actions on a route with no workspace binding: - `unarchive_workspace` — unarchive an arbitrary workspace by id - `prune_concurrency_group` — delete a global concurrency group - `list_worker_groups` — return unobfuscated `env_vars_static` (may hold secrets) Add job-token-aware `is_instance_admin` / `require_instance_admin` helpers (the same shape as `require_super_admin` / `require_devops_role`) and use them at these three sites. Workspace-scoped `require_admin(authed.is_admin, ...)` gates are intentionally left unchanged — a workspace-admin job token is within the cap there. Regression added covering all three; verified it lets a WM_TOKEN unarchive/leak without the fix and is blocked with it. Reported by Codex CI review (P1) on #10124. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * fix(mcp): drop orphaned path_field_renames from EndpointTool test helper The merge with main adopted main's mcp path-substitution refactor (#10162), which removed the `path_field_renames` field from `EndpointTool` and its consumer (`substitute_path_params` no longer takes per-field path renames). main's `runner.rs` `ep` test helper still constructed the struct with `path_field_renames: None`, so the workspace test build (cargo test --all, which compiles windmill-mcp's own #[cfg(test)] module under the `server` feature) failed with E0560. A plain `cargo check` does not compile that test module, so it only surfaced in CI's cargo_test. Remove the orphaned field to match the struct. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * test: describe the sentinel-rejection policy the forged-identity test asserts Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix: complete ApiAuthed initializers in feature-gated tests after merge Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix: stop job tokens minting credentials that shed their provenance Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix: cap the MCP OAuth approval mint at the same elevated-job-token gate Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix: cap the self-service password reset at the elevated-job-token gate Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix: cap app embed/SDK mints and scope widening at the elevated-job-token gate Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix: keep job tokens from destroying the account they run on behalf of Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix: deny job tokens a foreign-workspace admin claim and workspace ejection Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * docs: keep the follow-up inventory in the PR instead of the repo Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix: make the session workspace status gate job-token aware session_workspace_status derived its superadmin branch from a bare email check, so a job token carrying a superadmin identity resolved the existence of workspaces it has no relationship with rather than seeing them as deleted. Switch to is_super_admin_authed, matching every other instance gate reached from a request ApiAuthed. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * revert: leave the global concurrency-group listing on the plain admin gate The listing exposes concurrency keys across workspaces, which is metadata rather than a capability, and it 401s rather than degrading. Keep the guard on the prune route next to it, which is the destructive one. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix: keep the instance-admin gate on the global concurrency listing The listing spans every workspace's concurrency keys, and the gate rejects only job tokens: the !is_admin branch is the pre-existing check, so workspaced tokens and interactive admins are unaffected. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * chore: update ee-repo-ref to d30af67d38954f9012f7bad08da23e347344b4c6 This commit updates the EE repository reference after PR #664 was merged in windmill-ee-private. Previous ee-repo-ref: 7870573dbc3360f99bada143f094c67dce0d9e9c New ee-repo-ref: d30af67d38954f9012f7bad08da23e347344b4c6 Automated by sync-ee-ref workflow. --------- Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Co-authored-by: hugocasa <hugo@casademont.ch> Co-authored-by: windmill-internal-app[bot] <windmill-internal-app[bot]@users.noreply.github.com>	2026-08-19 22:33:46 +02:00
3 people	5fb145c79f	feat: guided setup wizard for data tables on Cloud (#10584) ... * feat(frontend): guided setup wizard for data tables On Cloud a data table cannot use the Windmill instance database, so a new workspace hit a dead end: an alert telling the user to go find a PostgreSQL resource somewhere else. Setting one up meant three disconnected places, and the connection could only be tested after the config had already been saved. Adds a three-step wizard (choose a database -> set it up -> name it) reached from the data tables settings page: - Supabase: signs in via the existing supabase_wizard OAuth client and creates the project from inside Windmill. Because db_pass is an input to project creation, Windmill sets the password and the user never visits a dashboard. - Your own database: picks an existing postgresql resource, or adds one with a connection string through the form that already supports it. - Windmill database: hands back to the inline row editor, since instance databases are provisioned by a superadmin. Verifying access is no longer a step the user takes: Continue runs the check and passing it is what advances the wizard, so a database that cannot create tables never reaches the workspace config. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * chore: pin ee-repo-ref to the Supabase provisioning endpoints Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix(frontend): do not claim the database is ready when its check failed Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix: address review findings on the data table wizard - The Supabase create branch advanced on `provisioning === 4` without consulting the check it had just run, so a role that cannot create tables could reach Finish. It now blocks and offers Try again. - Retrying no longer mints a fresh secret variable + resource each time: the credentials are only re-created when the password actually changed. - The generated password is captured before the create call rather than after, since a throw there can still leave a project behind. - On a failed provision the project list is refreshed, so the just-created project can be picked up from the other tab instead of provisioning a second. - Finish refuses a name that already belongs to another data table, which previously repointed it at the new database. - Secrets go to the acting user's namespace instead of a literal `u/admin/`. - The progress list no longer ticks "Created on Supabase" before the request is sent, and does not claim the database is ready when its check failed. - The wizard's resume state is cleared when it closes, so reopening after an abandoned OAuth round trip is not stuck on step 2. - The OAuth callback shares the session-storage key rather than repeating it. - SupabaseConnect uses the shared provisioning helpers instead of a fork. - Restores the doc comment displaced onto TestDataTableResourceQuery. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix(frontend): simplify Alert layout and balance its vertical padding The body was rendered by two near-duplicate branches, each wrapping the text in an extra div only to hang a margin on it, and the margins disagreed: the collapsible branch spaced above with mt-2, the static one below with mb-2. Since isCollapsed defaults to true, every non-collapsible alert took the static branch, so titled alerts read as 24px of space below the text against 16px above -- visibly off-centre -- with the title and body flush against each other. Collapse both branches into one and drop the margins; the container's own padding now sets top and bottom equally, with a small gap under the title row. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix(frontend): only offer Supabase when its OAuth client is configured The wizard offered the Supabase card unconditionally, so on an instance whose superadmin never configured a supabase_wizard client -- or whose backend is built without the oauth2 feature, which compiles the whole /api/oauth router out -- the card dead-ended at a 404. Gate it on listOauthConnects, the same check ApiConnectForm already makes, fetched on open so configuring the client mid-session does not require a reload. Also drop the Supabase project ref from the existing-project cards: it is an opaque identifier that means nothing outside Supabase's own dashboard URLs. Show the region instead, plus a status word when the project is not healthy, since a paused project is the one case where the connection check fails for a reason unrelated to the password. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix(frontend): run the Supabase OAuth leg in a popup A full-page redirect unmounts the wizard, so anything the user does on Supabase's side -- signing in, confirming an email, browsing their dashboard -- leaves them with nothing pointing back at Windmill, and the wizard had to park its state in sessionStorage to survive the trip. Open the connect endpoint in a popup instead. The modal stays on screen throughout and the callback hands the token back through postMessage rather than navigating. The parked-state path stays as the fallback for browsers that block the popup. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix(frontend): scope the connection check to the choice that produced it A failed check stayed on screen when the user switched Supabase mode or picked a different provider, so a fresh tab opened showing an error about a database it had nothing to do with. Clear the report and the error on both switches; re-clicking the tab already selected leaves an error the user is reading in place. Also polish the Supabase step: project cards get the provider-card treatment (icon, p-3, flex column) instead of a hand-rolled variant whose block layout left more padding above the name than below; form labels settle on text-emphasis; and the signup link sits under the primary button for anyone who does not have an account yet. Drop the "free" badge and the "Free on Supabase" line -- every option in the wizard is free, so neither told the user anything -- and say what the Supabase card actually does now that connecting an existing project is the default. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * refactor(frontend): one setup checklist and one Supabase step for every host The data table wizard, the instance database modal and the resource drawer each had their own version of the same two interactions, and they had already begun to drift: the wizard's Supabase resource shape was rebuilt by hand in the drawer, and the instance checks rendered with no notion of a step being in flight. SetupChecklist replaces LoggedWizardResult, whose only consumer was the instance modal. It adds the running state that component lacked, so a list driven by an endpoint that reports nothing until it returns still shows where it is. Both the instance checks and the Supabase provisioning stages render through it. SupabaseProjectStep owns picking or creating a project, and useSupabaseOauth owns the popup leg. Each host keeps only what is genuinely its own: the wizard saves a variable and resource then verifies the connection, the resource drawer fills in its own form. Both trigger authorization themselves, so a host can offer it a screen earlier than the step does. The lists load behind a spinner because which mode to open on depends on whether the account has projects; deciding that after rendering flipped the toggle under the user. Adds a kitchen_sink playground for the checklist so the animation and every failure position can be exercised without a backend, a superadmin, or a Supabase account. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix(frontend): tidy the resource drawer around the Supabase entry point Connect Supabase was a hand-styled anchor carrying Supabase's brand hex values rather than a Button, and it sat in a row whose other controls had settled on unifiedSize md. Making it a Button meant SupabaseIcon had to satisfy IconType, so it now takes `size` (deriving height/width from it) alongside the string props its other callers pass. The manual resource form spaced every field 32px apart and WhitelistIp added another 16px of its own, which read as a gap rather than a rhythm. One gap of 16px, with the form itself given a little more separation from the description above it. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix(frontend): stop Supabase resources coming up modified when first opened Resource forms fill in every unset property from the schema as soon as they render, so a postgresql resource saved without region, root_certificate_pem and use_iam_auth was dirty -- and had saved a draft -- the first time anyone looked at it. Write them with the rest of the value. SupabaseConnect also rebuilt the resource shape by hand instead of using the shared helper, which is how the pooler host format ended up in two places. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * feat(backend): record where a data table came from and whether setup finished edit_datatable_config replaces the whole datatables map and DataTable does not deny unknown fields, so anything the request omits is dropped without a word. origin and setup_incomplete would have been erased by any unrelated save; preserve_unmanaged_datatable_fields carries them -- and migrations_enabled, which had the same problem inline -- forward for entries that already exist, following renames. setup_incomplete is what lets a row be recorded before the resource it points at exists, so the wizard can write nothing until the user finishes. There is deliberately no intermediate state: the setup runs entirely in the browser, so nothing server-side could advance one. datatable_health probes every data table at once for the settings page and skips the incomplete ones, whose resource_path resolves to nothing yet. set_datatable_setup patches a single entry instead of resending the map. test_datatable_connection_value checks a connection the caller has not saved anywhere, which the wizard needs before it has written a resource. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix(frontend): make destructive default and subtle buttons read red Both variants were neutral until the pointer arrived, then filled solid red: nothing marked the button as destructive until you were already on it. They now carry red text at rest, with a faded red border on default and a light red wash on hover, which is what the legacy red border style in the same file had always done. Three call sites passed color="red" alongside a design-system variant. getStyleClass returns before colour is read for accent, accent-secondary, default and subtle, so the delete-migration control, its modal confirm and the import-database button had all been rendering neutral. They pass destructive now. The dropdown variant strips the button's own border, and matched border-border-light literally -- a class the destructive style no longer contains. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * feat(frontend): rebuild data table setup around a read-only row The wizard gathers intent over two steps, reviews it on a third and writes nothing until Finish, so a billable Supabase project is created only once the user has seen what will happen. runSetup is also the retry: every step probes for its own result before doing anything, so running it again on a half-finished data table resumes instead of duplicating. Its steps are keyed rather than dispatched on their titles, where rewording one changed what it did. The settings row stops being an editable form with a dirty/save cycle. It carries the name, where the database came from, a health dot and two actions; everything rare moved into the gear panel, which also offers Finish setup for a data table whose wizard never completed. Manage is ExploreAssetButton, the control the ducklake list already uses, and the row and panel both link out to the underlying resource. supabaseResourceValue no longer assembles the pooler host from the region. aws-0-<region>.pooler.supabase.com is wrong for any project Supabase allocated elsewhere, so the host, user and port come from the pooler config endpoint. Two data tables sharing one database also share _wm_migrations, which is probed unqualified, so the review step warns when the database being connected is already behind another data table. SupabaseConnect is deleted. The resource drawer uses the shared project step restricted to existing projects: creating one is a billed action and belongs in the wizard, which has somewhere to report what it did. The kitchen_sink checklist playground goes with it. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix(frontend): fall back to a direct Supabase connection when the pooler cannot be read Reading a project's Supavisor config needs the database_pooling_config_read scope, which an instance's Supabase OAuth app may never have been granted. No retry recovers from that, and the wizard treated it as fatal: the user was left with an error and no way to finish connecting a project that was otherwise fine. resolveSupabaseConnection replaces the bare pooler read everywhere it happened. Asking for session pooling and failing now yields a direct connection plus the reason, which supabaseResourceValue already knew how to write. Nothing about the fallback is silent -- direct is IPv6-only, which is the whole reason session pooling is the default -- so the wizard warns on its review step and the resource drawer says so in its toast. The row is recorded before credentials are saved, so an origin claiming session pooling has to be corrected once a direct host is what gets written; the run patches it through set_datatable_setup rather than leaving the panel to report a mode nothing uses. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * feat(frontend): open the database behind a data table, and say when it cannot write Every database in the list now opens the surface that owns its credentials. A postgres one opens its resource in the editor drawer; a Windmill instance one opens the instance modal, which is where its setup checks, password rotation and drop already lived. Both are reachable from the row and from the panel's provenance list, and the provider icon moved inside the button so the whole thing is one target. CustomInstanceDbWizardModal targeted #content unconditionally, which put it underneath the panel drawer that now opens it. It takes a target, and the panel portals it to the body. The status column gains a third state. The probe reports privileges but nothing gated the dot on them, so a data table whose role cannot create tables showed as Connected and only failed when someone ran a migration. It reads "Limited permissions" instead, and opens the panel on the report carrying the GRANTs that fix it -- the settings page has already probed, so the panel takes that report rather than asking the user to run Test connection over work already done. fullyPrivileged is exported from the report component so the dot and the report cannot disagree about what counts as healthy. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * revert(frontend): keep the data tables settings table as it was The settings table and the setup wizard are two changes that only shared a file. Splitting them makes each reviewable: this branch keeps the wizard, and the read-only row, gear panel, health probe and clickable databases move to their own branch. The rows go back to the editable form with its pickers and save footer, still opening the wizard from Add a database. DataTableSettingsPanel, dataTableHealth and dataTableOrigin had no other consumers and go with them; the connection report stays, because the wizard shows it too. DataTableSettingsType keeps `origin`: the wizard writes it, and the review step reads it back to warn when two data tables would share one database and therefore one _wm_migrations table. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix(frontend): confirm before dismissing the data table wizard mid-setup Closing was guarded while a run was in flight and unguarded before one, which is backwards: a run leaves a row to resume from, whereas a backdrop click on the review step threw away the project, the pasted password and the folder with nothing to recover them from. Backdrop, Escape and the close button now go through one path that asks first. It only asks when there is something to lose -- no provider chosen yet, or a run that already produced a result, closes immediately -- so the dialog does not become something to click through. Continue in the background still leaves in one click; that exit was always the deliberate one. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix(frontend): stop the wizard claiming the resource folder controls who can use a data table "Who can use this database" was wrong. Every path that resolves a datatable:// reference -- both executors and the agent-worker endpoint -- reads the resource unchecked, by workspace and name. A resource in u/admin is usable by everyone's scripts. The folder governs who can see and edit the connection, and who can reference the resource directly in a SQL step; neither is who can use the data table. The wizard was contradicting the tab's own description two screens later. The folder select and name field become one Path picker, the same one the resource, variable and script forms use, so the review step reads as a resource path rather than a permission choice. Its initialPath is snapshotted when the step opens: Path seeds itself from it, and a live value fights the typing. Finish now also gates on Path's error, so a taken or malformed path stops the run before it writes anything. The button that opens all this says "Add a data table" -- the data table is what you get; the database is a detail chosen along the way. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * revert(frontend): move the destructive button restyle out of the wizard PR This reverts 3881e4d8ea. Making default and subtle destructive buttons red at rest changes every existing caller of the prop -- the workspace integrations, AI skills, workspace creation and the instance database drop -- so it is a design-system change, and the call sites it fixed are the migrations list and the database manager. None of that is the setup wizard. Nothing on this branch passes destructive any more, so it leaves with no loose ends. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix(frontend): make the wizard stepper navigate the steps it already offers Stepper dispatches a click and paints cursor-pointer on every reached step, but the wizard never listened, so the breadcrumbs invited a click and did nothing. They now reach any step already passed, in either direction: going back to check something should not cost the progress, which means tracking the furthest step reached rather than the current one. Forward movement still only happens through the primary action, so a step is never reachable without having been validated -- and changing the intent revokes the steps ahead of it, or Finish could run against a review built from something the user has since edited. The five places that cleared the probe on an edit now do both through one call. During a run nothing is reachable, and the stepper says so rather than showing a pointer over steps that will not respond. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix(frontend): restore the data tables description lost in the branch split The rewritten description went into DataTableSettings.svelte shortly before that file was restored wholesale to its pre-rebuild state, so it left with the row rework it had nothing to do with. The tab went back to describing the plumbing -- a fully managed PostgreSQL database, reachable from the SDK -- which never answered the question a new user actually has: why this rather than a Postgres resource. It leads with what a data table is, then the two things a resource cannot do -- nobody needs the credentials to query it, and the name can be pointed at another database without editing anything that uses it -- and closes with what Windmill runs on top. Both middle claims are the ones every resolution path backs up: datatable:// resolves by workspace and name, unchecked. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix(backend): say what is missing when a $res: or $var: reference does not resolve Both interpolations fetched with fetch_one and mapped the error through to_anyhow, so a reference to something deleted surfaced as "no rows returned by a query that expected to return at least one row @workspaces.rs:2169". It names neither the kind of thing that was missing nor its path, and it is what a data table pointing at a deleted resource reports. They now fetch_optional and return NotFound naming the path, and datatable resolution adds the data table on the way out: the caller asked for one by name, and a bare "resource f/x/y does not exist" leaves them to work out which of them points at it. The health probe is new, so this string had only just become something users read. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * feat(frontend): gate the data table wizard behind a dev flag The wizard only appears with `dataTableWizard` set in localStorage; without it the settings page keeps the inline-row flow it had before this branch, down to the empty-state copy and the "New Data Table" button, and the wizard component is not mounted at all. The existing e2e suite drives that button, so the default-off flag is also what keeps it green. Step 2 of "your own database" becomes one list rather than a segmented control: the workspace's Postgres resources, then a New resource card that expands in place. A connection string is not an alternative to a resource, it is how one is written, and the old layout taught otherwise. The card holds the same connection as a string or as fields and carries values across when you switch, so `parse` and `compose` have to be inverses -- hence the percent-encoding on both sides, which also fixes a password containing `@` silently corrupting in the resource form. The Supabase step now uses the same shape. Names and paths are checked as they are typed rather than at the end of a run that may have created a billed project first: the data table name against the charset `edit_datatable_config` enforces, the instance database name against what `setup_custom_instance_db` will accept, and the resource path against both the resource and variable namespaces, since the run writes to both and both writes upsert. `test_datatable_connection_value` refuses `$var:`/`$res:` in its body. It feeds `transform_json_value_unchecked`, which resolves references with no permission check of its own, so an admin could otherwise have had the API server decrypt any workspace secret and hand it to a host the same request chose -- without the audit trail a variable read leaves. Callers testing something unsaved hold the literal value already. Alert, SetupChecklist and postgresConnectionString change for everyone, not just behind the flag: body-only alerts no longer reserve an empty title row, the checklist can nest the checks a step is made of, and the connection-string parser is shared with the resource form. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * chore: pin ee-repo-ref to the EE branch merged with EE main The Supabase proxies the wizard calls are still unmerged, so the ref cannot be an EE main commit yet; it now names that branch merged with EE main rather than the branch alone, which was nine commits behind and would have been built against a CE main it never saw. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * feat(frontend): gate the supabase resource path behind the dev flag * test(frontend): pin connection string parsing to libpq behaviour * fix(frontend): keep the supabase resource link off the popup callback path * refactor(frontend): load the supabase resource dialog only behind the flag * fix(frontend): refuse a resource path the wizard run does not own * fix(frontend): let a failed data table setup be corrected without losing what it made * fix(frontend): let a failed setup reuse the resource path it claimed Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix(backend): record the two data table connection tests in the audit log Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix(frontend): use Section for the data table wizard advanced group Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix(frontend): read connection strings the way libpq does Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix(backend): pin the ee ref back to a commit this branch can build Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix(frontend): keep a failed setup's claims across the redirect and rollback Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix(backend): probe a data table with the auth mode the worker will use Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix(frontend): keep every part of a connection string through the round trip Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * refactor(frontend): give a setup run one record of what it created Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix(frontend): mark a resource claim by edited_at, not its creator Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix(frontend): mark every claim by revision, and keep an unconfirmed project's secret Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix(frontend): refuse to test or save behind a connection string that will not parse Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix(frontend): refuse a connection string carrying options the resource cannot hold Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix(frontend): allowlist the connection-string parameters a resource can honour Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix(frontend): guard every created Supabase project, not just the last one Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix(frontend): do not warn about renaming an item that does not exist yet Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix(frontend): make the review step read as one list of what will exist Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix(frontend): keep the picked Supabase project across the redirect, reject connect_timeout Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix: check the data table connection from a worker, not the API server The wizard's connection check ran on the API server through two endpoints added for it. That server is a different machine with a different identity, so the answer was about the API server rather than about the worker that will run the queries: a host reachable from one is not necessarily reachable from the other, and IAM RDS and Azure workload identity authenticate as whichever process opens the connection. Run the privilege query as a preview job instead. A job goes through the worker's Postgres executor, which is where `PgAuthMode::of` already picks the authentication mode, and it takes either a resource value or a `$res:` path exactly as a Postgres step does. Postgres composes the suggested GRANT statements through `format('%I')`, so identifier quoting stays where it is already implemented. Removes `test_datatable_resource_connection` and `test_datatable_connection_value`, and `connect_as_the_worker_would` with them. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * refactor: fold check_datatable_connection back into its only caller The helper was split out so the two connection-test endpoints could share a body. Those endpoints are gone, leaving one caller. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * revert: keep the data table connection check schema inline It was lifted into components so three endpoints could share it. Two of those are gone, so it is back to one user and the extraction changes nothing. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix: restore openapi.yaml to the branch point The previous commit restored main's tip rather than the merge base, which carried three unrelated main-only changes into this branch: the resource mcp_tools truncation fields, the execution_mode description, and a version bump. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * refactor(frontend): drop four effects from the data table wizard Each was doing work a derived, a load callback or a real entry point does better. - The name conflict is kept with the name it was raised for and derived from it. As an effect it was correct only because it never read what it wrote: the pre-flight sets the message and the effect does not re-trigger, so adding a read would have cleared it the instant it appeared. The message now also comes back if the taken name is retyped, which is what the server will say. - The default resource selection is seeded inside the fetcher that loads the list, where "has the fetch settled" cannot be asked wrong. - Reset-on-open becomes an exported open(), called by the settings page, so a fresh run is set up by the act of opening rather than by a flag emulating mount. - The OAuth connects and the folder list become resources; supabaseAvailable and folders are derived from them. defaultFolder takes the list rather than reading it, so the fetch can seed off its own result. Leaves the debounced path check, which is async with an out-of-order guard. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * refactor(frontend): drop three effects from the Supabase branch - useSupabaseOauth reports success as onAuthed, alongside the failures it already reported. SupabaseResourceConnect was watching `authed` to find out; it takes the callback instead, keeping the guard that stops an authorization started elsewhere on the page from opening its dialog. - SupabaseProjectStep loads its orgs and projects through a resource keyed on the token, so the `loaded` latch goes and re-authorizing reloads rather than keeping the lists from the expired session. - SetupChecklist records what the user toggled and derives the open state from it, a failed step defaulting to open. Recording the open state instead needed an effect to force it, and that effect re-ran on every progress update, so a description closed while anything was still ticking reopened. A close now holds for the life of the checklist, including across Try again. Leaves the message listener, which subscribes to another window. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix(frontend): confine the modal restyle to the wizard, and trim the comments The wider side padding and lighter dialog heading were changing all 17 Modal2 dialogs to suit this one flow. They move behind an opt-in `formStyling`, taken by the three dialogs this branch owns; every other Modal2 renders as it did. Also drops two comments that cited a design approval rather than a constraint, and shortens the blocks that had grown past the four lines AGENTS.md asks for. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix(frontend): use the accent token for the wizard's links `text-blue-500` is the marketing blue `#3B82F6`, which brand-guidelines.md rules out in the app interface. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * chore: point ee-repo-ref at the EE branch head Picks up EE main, which the branch now needs, and the Supabase proxy auth fix. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix(frontend): read sslmode by name, and stop decrypting a secret to date it - `sslmode` was found by searching the query text, so it also matched inside another parameter's value: `?application_name=sslmode=disable` passed the allowlist on the parameter name and then parsed as a request to turn TLS off, which both the wizard and the resource form saved and probed. Parsed with `URLSearchParams` by exact name, with a test. - `secretMark` read the variable with `decryptSecret` defaulted to true, so every write decrypted a secret nothing reads and recorded the decryption -- including someone else's on the retry about to refuse it. It wants only `edited_at`, which is returned either way. - The probe gave up at 15s while the worker allows its Postgres connect 20s, so a host that accepts the connection and never answers was cancelled and reported as a missing worker rather than a failed connection. - The create-mode region and project name did not report an intent change, so renaming a project after a name collision left the failure naming the old one. - Two comments described the code as it was before the claim mark became a revision, and a doc comment outlived the field it documented. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix(frontend): read connection parameters the way libpq does One reader for both the parser and the allowlist, since they disagreed about what a string says in two ways that both ended in a weaker connection than was pasted: - `URLSearchParams.get` takes the first of a repeated parameter and libpq takes the last, so `?sslmode=disable&sslmode=require` was read as `disable`. - The allowlist folded the parameter name and the parser did not, so `?SslMode=verify-full` was refused by neither and honoured by neither, and saved as the `require` default. The parked Supabase run is now handed to `open()` rather than read back off the `resume` prop it was just assigned to, so restoring it does not depend on when that prop reaches the component. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix(frontend): keep connection parameter names case-sensitive libpq does not fold them: `?SslMode=disable` is rejected as an invalid URI query parameter rather than read as `sslmode`, which a local server confirms. Folding made Windmill accept and honour a string Postgres itself refuses; naming the parameter instead tells the user why it cannot be stored. The last-value-wins rule for a repeated parameter is unchanged, and matches what the same server does. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix(frontend): seed the Supabase organization from the project it selects The loader took `orgs[0]` independently of the project it seeded, so an account whose first project sits outside its first organization had the review step name an organization the database does not belong to. Picking a project by hand already derives it; the seeding now does the same. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix(frontend): let the probe report an empty search_path instead of failing on it `format('%I', NULL)` raises rather than returning NULL, so a role whose search_path names no valid schema failed the whole privilege query and was reported as an unreachable database. That is the one case `fix_search_path` exists to name, and it never reached the user. Verified against a local server with `SET search_path = ''`. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix(frontend): say which of the two refusals a connection string hit Making parameter names case-sensitive gave `unsupportedConnectionParam` two reasons to refuse, and the single message explained only one. `?SslMode=` was answered with "Windmill cannot store SslMode on a Postgres resource", which is false twice over: sslmode is exactly what the resource stores, and the string asks for nothing because Postgres rejects the URI. It now names the spelling when the parameter is one we keep, and the storage limit otherwise. The folder-list guard also still read the `resume` prop that `open(parked)` was changed to stop trusting, so the resumed path now comes from whatever `reset` was handed. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix(frontend): leave the Supabase organization unset when the lookup misses Falling back to the first organization named one the seeded project is not in, since `supabaseSummary` prefers `intent.org` over the project's own. Unset, it falls through to the project's organization identifier — the right one, spelled as a slug rather than a name. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * test(frontend): pin which refusal a connection string gets The two messages differ in what they ask the user to do, and the condition choosing between them — whether the lowercased name is one the resource keeps — is not visible from either call site. `Connect_Timeout` is the case that keeps them honest: miscased *and* unstorable, so respelling it would not help and the message must not suggest it. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix(frontend): hand a failed Supabase leg back to the page holding its run Denial, a token error and a malformed callback all sent the user to /resources whether or not a run was parked. Nothing else consumes the park, so the run stayed in sessionStorage and sprang the wizard open on an unrelated later visit instead. A parked run now lands on the data tables tab, where the wizard resumes on the setup step and can authorize again. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix(frontend): let a run reuse the name of a row it could not take back out `removeRow` reports `kept` when the undo cannot reach the server, so the row this run wrote stays in the workspace config and comes back in `existingNames`. The client-side name check then refused the retry on the run's own name, with no way forward but a rename. The instance database name has carried the same exemption since it was written; this is the data table name catching up. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix(frontend): discard a variable check the wizard has moved on from The post-await guard compared only the path, and the path is built from the review step's fields -- so picking an existing resource stops the wizard minting one without changing it. A check already in flight then answered for a branch nobody was on, and a `true` disabled Finish over a path the run no longer writes. The cleanup cannot help: it cancels a pending timer, not a live request. Both sides of the await now ask the same question. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * chore: update ee-repo-ref to 483513b70979aa9497cab869837108d948449984 This commit updates the EE repository reference after PR #715 was merged in windmill-ee-private. Previous ee-repo-ref: 8604b30a740c5620069208801a7ae50937b61977 New ee-repo-ref: 483513b70979aa9497cab869837108d948449984 Automated by sync-ee-ref workflow. --------- Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com> Co-authored-by: windmill-internal-app[bot] <windmill-internal-app[bot]@users.noreply.github.com>	2026-08-19 20:03:14 +02:00
Ruben FiszelandClaude Opus 4.8	fa7fbd348d	fix(security): validate ansible git repository URLs before invoking git (#10759) ... The Ansible executor passed the user-controlled git repository `url` (from playbook YAML or a `git_repository` resource) straight into `git clone`, `git ls-remote` and `git remote add` on the worker host. A URL that git parses as an option — e.g. `--upload-pack=<cmd>` — turns `git ls-remote <url> HEAD` into arbitrary command execution on the host, outside any job sandbox. Non-http transports (`ext::`, `file://`, local paths) similarly run programs or read host files. Add `validate_git_repo_url` in windmill-common: reject a leading `-`, reject remote-helper `::` syntax, and allow only the `http(s)`, `ssh`, `git` and scp-like `[user@]host:path` transports. Also reject a `branch`/`commit` that starts with `-`. Validation runs at every ansible entry point that spawns git, covering both the inline-YAML and resource-provided URL paths. CWE-88 (argument injection) / CWE-78. Reported by Nitin Gavhane. Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>	2026-08-19 14:04:39 +02:00
4 people	53eb94659b	feat(telemetry): extend feature-usage tracking beyond AI features (#10681) ... * feat(telemetry): extend feature-usage tracking to long-tail features Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * docs: describe telemetry as product feature usage rather than AI usage Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * refactor(telemetry): trim disclosure copy and drop unused pick origin Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix(telemetry): count trigger fires per run and key hub picks from hub data Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix(telemetry): slugify hub keys and order both writers' upserts Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix(telemetry): key native trigger adoption by service so it matches fires Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * chore: update ee-repo-ref for native trigger adoption fix Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * refactor(telemetry): move feature-usage collection into the ee crate * docs: point feature-telemetry at the moved registry and rust writer * docs: correct the trigger-fire gate comment to match measured step counts * docs: put the private-build caveat on the verification step * chore: update ee-repo-ref to f079db9e7962a413b349c4ff8036080894f30771 This commit updates the EE repository reference after PR #725 was merged in windmill-ee-private. Previous ee-repo-ref: 055adb80416f9339c9a28ae7fbaeadad30d74959 New ee-repo-ref: f079db9e7962a413b349c4ff8036080894f30771 Automated by sync-ee-ref workflow. --------- Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com> Co-authored-by: Ruben Fiszel <ruben@windmill.dev> Co-authored-by: windmill-internal-app[bot] <windmill-internal-app[bot]@users.noreply.github.com>	2026-08-14 18:50:38 +02:00
hugocasaandClaude Opus 5	98bacab907	refactor: combine the per-minute counters onto one shared helper (#10687) ... * refactor: combine the per-minute counters onto one shared helper Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix: keep dashmap in windmill-store for the azure devops token cache Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * refactor: name the sweep counter for what it counts Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>	2026-08-14 18:45:20 +02:00
Ruben FiszelandClaude Opus 5	30f5d2e766	perf: declare a settings pass instead of reading one setting at a time (#10698) ... * perf: read global_settings once per settings-load pass `initial_load` reads several dozen settings back to back, one `SELECT value FROM global_settings WHERE name = $1` each: 50 serialized round trips before a worker is ready, 32 before a server is. On localhost that is ~20ms and invisible; against a real database it is 50x the RTT per process start, which `EXIT_AFTER_N_JOBS` turns into a per-job cost. `with_global_settings_snapshot` reads the whole table (12 rows on a typical instance) into a tokio task-local, and `load_value_from_global_settings` serves from it. Scoping it to the task is what keeps the single-setting reload paths correct: a `notify_global_setting_change` event for one key runs outside any scope and still reads the database, so a live settings change reaches a running worker as before. Agent workers hold an HTTP connection with no snapshot to take and are unchanged. `load_smtp_config` and `reload_custom_tags_setting` had their own inline copies of the same query; they go through the shared loader so they land in the snapshot too. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * docs: state the snapshot contract on the reader and the query `load_value_from_global_settings` is called from ~10 crates and one of them writes a setting then immediately re-reads it through `reload_custom_tags_setting`; say on the function itself that a scope, when one is installed, serves the read and leaves `db` unused. The query comment claimed the table is a handful of rows. It is not bounded that way: `workspace_dependencies_map_rebuilt:<workspace_id>` adds a row per workspace and never removes it. Those dynamically named rows are also why the snapshot fetches the whole table instead of the wanted names, so state that as the reason. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix: bound the settings snapshot and keep it out of two reads Three review findings, all real: The snapshot fetched the whole table, which is not bounded by the settings that exist: `workspace_dependencies_map_rebuilt:<workspace_id>` adds a row per workspace with no cleanup path, and no settings pass reads one. It now fetches only statically named rows, and reads of a `<prefix>:<id>` name skip the snapshot and go to the database. Correctness does not rest on that naming convention — a colon-free dynamic name would simply be in the snapshot and still answered correctly — only the bound does. A snapshot query that failed inside an enclosing snapshot awaited the body bare, so its reads were served by the outer snapshot rather than falling through as documented. The task-local carries an explicit bypass state and the failure path scopes it. `reload_jwt_secret_setting` decided whether to generate-and-upsert the JWT secret from a snapshot-served read, so a replica booting alongside another could overwrite the secret it had just generated and invalidate its tokens. That read goes through the new `load_value_from_global_settings_fresh`. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix: keep the snapshot query on the primary-key index `name NOT LIKE '%:%'` bounded the rows returned but not the work: a leading wildcard cannot use the index, so Postgres read every row anyway. Against 50k dynamically named rows it plans as a seq scan of 516 buffers whether or not seqscans are enabled — and worker connections disable them, so the plan was one the query shape forbade rather than one the planner chose. `name = ANY($1)` over an explicit list plans as a bitmap index scan, 7 buffers, bounded by the listed names rather than by table size. That list is also exactly the set the snapshot may answer from, so a name outside it falls through to the database instead of reading as unset: listing a setting is a performance choice, never a correctness one, which is what keeps the list safe to maintain by hand. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * refactor: declare a settings pass instead of reading one setting at a time Replaces the prefetch-list snapshot with a pass the call sites build themselves. `SettingsPass` collects the reads `initial_load` will make as `(name, applier)` pairs, fetches them together, then replays the appliers in declaration order. Declaring is what makes the batch exact. The same `if server_mode` / `if *CLOUD_HOSTED` / `cfg` branches that used to guard a read now guard a declaration, so the fetch asks for what this process needs and nothing else, and there is no list of setting names to keep in sync with anything. Ordering is preserved end to end: appliers run in the order they were declared, and non-setting work in the middle of the sequence keeps its place as a step, so nothing moves and nothing runs twice. Steps that need several settings at once take them together. The batch distinguishes three states where a per-setting read only ever produced two at a given call site: - a value, - genuinely unset, which several settings must see in order to restore a default when the setting is cleared, - could not be read, which must leave the in-memory value alone. Collapsing this into "unset" would let one failed query reset workspace fairness and the queue caps across a cluster. Over HTTP the reads go out together rather than sequentially, so an agent worker's settings load costs one round instead of ~36, with no new endpoint. A setting an agent may not request still resolves to unset, as the per-setting call returned for it. `reload_*` keeps working per setting for the notify path, sharing its apply half with the pass. The wrappers no caller was left using are dropped. worker startup: 50 queries -> 2 (the batch, and jwt_secret which stays its own read so the pass cannot sit between reading it absent and upserting a replacement over another replica's). Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix: run the pass's non-setting steps in declaration order too Review round found the settings pass had a gap: the reads were declared but the work interleaved between them still awaited inline, so it all ran before `pass.run` applied anything. `manage_audit_partitions` therefore saw `AUDIT_LOG_RETENTION_DAYS` at its compile-time default rather than the configured value, and dropped every partition past that default. An instance keeping 30 days on CE lost the 14-to-30-day band on startup and on every full-reload tick. The `STORE_AUDIT_LOGS_S3` export anchor had the same cause: the gate read `false` before the setting applied, so an env-var-enabled export never anchored and its first tick skipped the rows committed before it. `action` exists so a step keeps its place in the sequence; every remaining inline await is now one, which fixes both and leaves no phase where a read can observe a value the pass has not applied yet. Two more from the same round: A batch that fails as a whole now falls back to per-setting reads. Skipping every applier preserves known-good state on a reload tick, but a starting process has none, and would have run on compile-time defaults until the next full reload twelve hours later. `FORCE_RUBY_REPOS` is honored again: the batched url-list path parsed without the `FORCE_` check its per-setting counterpart applied, so the override was silently dropped. `load_setting_value` never had one, so the third helper was never affected. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix: declare the object-store and worker-config steps in the pass too Two awaits were left running ahead of `pass.run`, so the settings they read were still at their compile-time defaults. The object-store reload is the one that matters: an AWS OIDC store mints its first token against an issuer built from `BASE_URL` (`oidc_ee.rs`), and with `OTEL_ENVIRONMENT` set nothing loads that before this pass does, so the store signed with the unset default, left `OBJECT_STORE_SETTINGS` empty and fell back to the ten-second retry while startup carried on. `reload_worker_config` calls `store_pull_query`, which reads the workspace fairness knobs. It happened to converge because the enabled flag re-stores the query when it changes, but it was reading defaults on the way there. Both are steps now, which is also what the earlier fix should have covered: the only await left outside a step is `pass.run` itself. Also from the same round: `fetch_settings_batch`'s doc comment had been stranded on the helper inserted above it, and the batch-failure fallback re-ran the same reads on an agent worker, where the batch already is the per-setting read. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * docs: point the setting-loader docs at functions that still exist `reload_setting` went with the other wrappers no caller was left using, but two doc links still referenced it. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix: decide the jwt secret in sql so the read can be batched `reload_jwt_secret_setting` generated a secret whenever its read came back absent or unparseable, and upserted it unconditionally. Two replicas booting against an empty row therefore each installed their own and rejected each other's tokens, and the same happened on a running cluster whenever the row was deleted or set to a non-string. Keeping the read next to the write kept the window narrow but never closed it, and it was the reason this one setting could not go through the settings pass. `get_or_create_jwt_secret` puts the decision in the statement instead: INSERT ... ON CONFLICT (name) DO UPDATE SET value = EXCLUDED.value WHERE jsonb_typeof(global_settings.value) <> 'string' RETURNING value First writer wins, a usable secret is never overwritten, and an empty RETURNING is how a caller learns another process's secret stands. The `WHERE` also keeps a normal startup from writing at all, which matters because `notify_global_setting_change` fires on every write to this table and an unconditional upsert would have made each start trigger a cluster-wide reload. Because the statement decides rather than the caller's read, a stale value is harmless and `jwt_secret` is now an ordinary declaration. Worker startup is a single batch round. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix: keep a failed read from dropping a FORCE_ override or clearing a setting Two ways a read that did not succeed was being treated as an answer. A `FORCE_` override used to be checked before the read, so a failed read could not affect it. Moving that check into the parser put it behind a value arriving, and a failed read skips its applier, so a forced private registry fell back to the public index and a forced `settings.xml` was deleted from disk by the Maven step that follows it. Forced settings are declared as steps with no read now: the override outranks the database, so there is nothing to fetch and nothing to lose when a fetch fails. The setting loaders were passing `v.ok().flatten()` to their appliers, which turns a database error into "unset". Most appliers ignore `None`, but `apply_tag_per_workspace_workspaces` clears the workspace whitelist with it, making every workspace eligible for per-workspace tags, and `apply_fork_workspace_tag_append_fork_suffix` stores `false`. Both are also reached from the notify handlers, so a blip during a reload changed routing for the cluster. They take `?` now, as the code they replaced did by leaving the error arm empty, and the other five are converted with them so an applier that later grows a `None` branch cannot inherit the problem. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix: route hub_api_secret through the FORCE-aware declaration `HUB_API_SECRET` lives in an `ArcSwap` rather than an `Arc<RwLock<_>>`, so it could not use `option_setting` and was declared by hand with a bare `setting` plus `parse_option_setting_value` — which is exactly the path that skips the `FORCE_` handling, so a failed read still dropped `FORCE_HUB_API_SECRET`. The rule now lives in `option_setting_with`, which takes the store closure and leaves `option_setting` a wrapper over it, so a setting held in something other than an `RwLock` reaches it too rather than having to reimplement it. The three remaining hand-written parses are `parse_setting_value`, which has no `FORCE_` handling to miss: `load_setting_value` never had the check either. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>	2026-08-14 18:03:33 +02:00
Ruben FiszelandClaude Opus 5	633d7bcb2e	feat: add trigger_history table with source tracking (#10696) ... * feat: add trigger_history table with source tracking Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix: gate trigger history reads on scopes and harden its writers Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix: filter trigger history scopes in SQL and match the cleared-handler diff Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix: record a trigger restore from the trashbin in its history Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix: record bulk http trigger creates and document the recording boundary Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix: lock the trigger row when capturing its history preimage Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix: only record an auto-disable that actually flipped the schedule Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * chore: state the auto-disable invariant once instead of at four call sites Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * feat: render trigger history changes as a structured field diff Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix: make a server-initiated disable atomic with its history row Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * docs: note that the auto-disable savepoint takes no pool connection Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * docs: note the flow fallback is the last chance to disable Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix: never leave a trigger enabled because its history row failed Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix: retry the disable history row instead of dropping it on first failure Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix: use the design-system Button for the change-value expander Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix: hold the trigger row lock across its disable history row Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix: keep the history-loss alert out of the listener cancellation race Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix: read the history workspace through the trigger-workspace seam Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>	2026-08-14 17:57:11 +02:00
Ruben FiszelandClaude Opus 5	22eadab67d	perf: resolve the worker external IP in the background (#10697) ... * perf: resolve the worker external IP in the background `run_workers` awaited `external_ip::get_ip()` — an HTTPS GET to hub.windmill.dev — before spawning any worker, so every worker process paid that round trip before its first job pull. Measured on a CE debug build it was 120-450 ms of a ~200-500 ms startup, and behind a firewall the call does not fail fast: it burns its whole 5 s connect timeout, on every process start. That cost is per-job under EXIT_AFTER_N_JOBS. The value is informational (it is only written to `worker_ping.ip`, which the workers list displays so users can whitelist the address), so nothing needs to wait on it. It now resolves into a process-wide cache off the startup path, and `WORKER_EXTERNAL_IP` supplies it explicitly for deployments that know their egress address or have no egress at all. Until it resolves the ping carries no IP, which `insert_ping_query` now COALESCEs so a reclaimed row keeps the address the previous process wrote instead of being blanked. The main loop reports the IP as soon as it lands rather than on the next periodic tick, so a short-lived process still records it. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix: keep unknown worker IPs out of the whitelist alert Review follow-ups: - `WhitelistIp` filtered only the `'unretrievable IP'` sentinel, so the `'NO IP'` one a pending or failed lookup now leaves in the row would be offered as an address to whitelist. It filters both. - Register `WORKER_EXTERNAL_IP` in `ENV_SETTINGS` so operators can confirm from the instance settings view that it took effect. - The worker tracked whether it had reported the IP by re-reading the cache after each ping rather than remembering what the ping carried, so a lookup landing mid-ping marked it reported without it reaching the row. The value is read once and threaded through `insert_ping` / `update_worker_ping_full`. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix: report a sentinel IP once the lookup has definitively failed Keeping the previous process's address on a reclaimed `worker_ping` row is right while the lookup is still in flight, but not once it has failed: the row would advertise an address nothing has confirmed, and the whitelist alert would offer it. A failed lookup now reports `UNKNOWN_IP`, leaving NULL to mean "in flight". `WORKER_EXTERNAL_IP` is rejected when longer than the `varchar(50)` column rather than panicking the worker on its initial ping, which is a hard failure. Adds the regression guard for the `ON CONFLICT` semantics: reverting to `ip = EXCLUDED.ip` would compile and blank every reclaimed row. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix: keep the agent initial ping acceptable to older servers An agent worker routinely runs against a server of a different version, and one predating the background lookup rejects an initial ping carrying no IP — which `run_worker` turns into a panic, so a newly upgraded agent would crash-loop against it. The not-resolved-yet case goes over the wire as the sentinel instead, and the server maps it back so a reclaimed row still keeps its address while resolution is pending. Also documents `ip` as the one conditional exception to `insert_ping_query`'s "only `started_at` and `jobs_executed` survive a restart", and adds `WORKER_EXTERNAL_IP` to the README env-var table. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix: deliver the resolved IP to servers that only take it at registration A server predating the background lookup applies `ip` from the initial ping only, and ignores it on the periodic ones. An agent registering before its lookup resolves would therefore keep the sentinel forever on such a server, where it used to report its real address. It registers a second time once the address is known, skipping that when the address is still unknown, when the server is reached over SQL and needs no second registration, or once a job has run, since registering clears the row's current job. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix: re-register the resolved IP even after a job has run Gating the second registration on "this process has not run a job yet" meant an agent that pulled queued work before its lookup resolved never delivered the address to a server that only takes one at registration. No job of the worker is in flight where that runs, so the gate bought nothing beyond the last job's id, which the next job refills. Documents the two cases where WORKER_EXTERNAL_IP stops being an optimisation and becomes the only way to report an address: an agent against such a server, and a process shorter-lived than the lookup. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * revert: drop the WORKER_EXTERNAL_IP escape hatch Supplying the address by hand skips the hub lookup, which is not something to make easy. Resolving it in the background is what keeps it off the startup path; opting out of it is a separate decision this does not need to take. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix: distinguish an IP never established from one that could not be retrieved `NO IP` was doing double duty: the column default for a row whose lookup has not resolved, and the marker for one that failed. An operator reading the workers list could not tell "not resolved yet" from "this instance cannot reach the hub", and the latter is the actionable one. A failed lookup now reports `unretrievable IP`, which is also what it reported before the lookup moved off the startup path. That leaves `NO IP` meaning only "no address established", which is what an agent sends while its lookup is in flight and what the server maps back to "unresolved" — so the wire sentinel no longer collides with the failure marker, and an agent delivers the failure to a server that only reads an IP at registration. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>	2026-08-14 14:27:21 +02:00
hugocasa	a91d55769d	chore: pin git-sync scripts to hub 28903/28904 (cli 1.787.0) (#10682)	2026-08-13 11:04:12 +00:00
Ruben FiszelandClaude Opus 5	71b9989daa	feat: auto-build binaries to object storage on deployment (#10673) ... * feat: auto-build binaries to object storage on deployment Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix: queue the auto-build from pre-locked deploys and off the lock slot Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix: materialize companion modules before a deploy-time build Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix: keep a build job from stamping lock_error_logs on a healthy script Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * test: de-flake test_flow_lock_all and surface the lock error it hides Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * test: trim drafting history from the flow-lock fixture comments Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix: stop a binary build from restarting dedicated workers Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix: keep the build-job marker off the agent wire and out of user args Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>	2026-08-13 07:50:28 +02:00
Ruben FiszelandClaude Opus 5	2fcce4526a	feat: add EXIT_AFTER_N_JOBS worker mode for environment cleanup (#10671) ... * feat: add EXIT_AFTER_N_JOBS worker mode for environment cleanup Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix: address review findings on the EXIT_AFTER_N_JOBS worker mode Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix: address round-2 review findings on EXIT_AFTER_N_JOBS Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix: address round-3 review findings on EXIT_AFTER_N_JOBS Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix: bound WORKER_SUFFIX length and document the same-worker drain Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix: validate the assembled worker name length Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>	2026-08-13 06:14:55 +02:00
Ruben FiszelandClaude Opus 5	4cb51cf7bc	feat: add memory limits to the go build subprocess (#10666) ... * feat: bound go compilation memory with GOMEMLIMIT Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix: bound the whole go build tree, not each toolchain process Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix: keep the go build memlimit and parallelism atomic Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix: log the go limits actually installed and stop serializing small workers Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix: make go build parallelism authoritative over persisted GOFLAGS Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix: canonicalize the go build -p value and floor the module-step budget Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix: parse GOMAXPROCS for -p the way the go runtime does Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix: read GOMAXPROCS with go's own grammar and report limits neutrally Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix: derive go build parallelism from the cgroup quota over its own period Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix: keep go's minimum build parallelism under sub-CPU quotas Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix: keep the windows 1CU cap out of go's two-compiler floor Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * docs: record that a worker runs one job at a time Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * docs: scope the one-job-at-a-time rule away from native workers Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>	2026-08-13 06:11:34 +02:00
hugocasaandClaude Opus 4.8	84f3b0094d	fix: harden custom env var name handling in the nativets/bun prologue (#10634) ... * fix: escape and validate custom env var names in the nativets prologue Custom workspace environment variable names were spliced verbatim into the generated NativeTS/Bun JS prologue (both the `const {name}` binding and the `process.env['{name}']` assignment), while only the value was escaped. A non-identifier name could therefore alter the generated program. - Add `escape_js_single_quoted` / `is_valid_js_identifier` helpers. - worker.rs and bun_executor.rs: escape the name as a string literal, and only emit the `const {name}` binding for valid identifiers. - set_environment_variable: reject non-identifier names on write (deletion stays unrestricted so existing rows remain removable). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * fix: address review — reserved-word const gate, grandfathered-name editability - Gate the `const {name}` prologue binding on `can_bind_as_prologue_const`, which additionally excludes JS reserved words and the prologue's own bindings (`process`, `BASE_URL`, `BASE_INTERNAL_URL`); such names would otherwise emit a SyntaxError that breaks every NativeTS run. They are still exposed via `process.env['{name}']`. - set_environment_variable: only enforce the identifier check for names that don't already exist, so editing the value of a pre-existing non-identifier name (the edit UI resubmits the name) isn't rejected with no in-product fix. - Document the name constraint on the endpoint in openapi.yaml. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * fix: exclude eval/arguments from const gate; skip existence query on valid names - Strict-mode ES modules forbid `eval` and `arguments` as binding names, so add them to the non-bindable set — otherwise an env var named `eval`/`arguments` emits `const eval = ...`, a SyntaxError that breaks every NativeTS run. - set_environment_variable: run the existence check only when the name isn't a valid identifier, so the common (valid-name) path skips the extra query; trim the rationale comment. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * fix: allow `async` as a prologue const binding; note reserved-bindings coupling `async` is a contextual keyword, not a reserved word — `const async = ...` is valid, so it needn't be excluded from the const binding. Also cross-reference the prologue head from PROLOGUE_RESERVED_BINDINGS so the two stay in sync. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>	2026-08-12 21:09:30 +02:00
3 people	5b0a159a01	fix(smtp): explain why a test email failed instead of 'deadline has elapsed' (#10620) ... * fix(smtp): explain why a test email failed instead of 'deadline has elapsed' Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix(smtp): keep non-SMTP error codes and retire a stale test alert Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * chore: update ee-repo-ref to f0df8b82c4c089d384423ed64b8504506084820d This commit updates the EE repository reference after PR #721 was merged in windmill-ee-private. Previous ee-repo-ref: 1ffaf3dea81e007c6c11146c1e12e97e83f5b938 New ee-repo-ref: f0df8b82c4c089d384423ed64b8504506084820d Automated by sync-ee-ref workflow. --------- Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com> Co-authored-by: windmill-internal-app[bot] <windmill-internal-app[bot]@users.noreply.github.com>	2026-08-10 21:21:14 +02:00
3 people	bf1b2cdcf9	fix(duckdb): cast list columns in quicksearch so tables containing them can be previewed (#10614) ... * fix(duckdb): cast columns in quicksearch so nested types can be previewed DuckDB's `CONCAT` implicitly casts scalars but rejects nested types: D SELECT CONCAT(' ', ['a','b']); Binder Error: Cannot concatenate types VARCHAR and VARCHAR[] - an explicit cast is required Quicksearch concatenates every visible column, so one LIST, STRUCT or MAP column makes a table impossible to preview — both the grid and its row count fail: Binder Error: Cannot concatenate types VARCHAR, VARCHAR, BIGINT, ..., VARCHAR[], ... and TIMESTAMP WITH TIME ZONE - an explicit cast is required LINE 1: ... FROM "raw"."accounts" WHERE ($1 = '' OR CONCAT(' ', "id", ... Every scalar in that list concatenates fine on its own — VARCHAR, BIGINT, DOUBLE, BOOLEAN, DATE and TIMESTAMPTZ were each checked individually — so the array column is the entire cause. Cast each column in the predicate. The comparison is textual either way, so no result changes, and the projection is untouched: casting there would change the types the caller reads back. This follows the shape already used for MSSQL in `mssql_needs_cast_for_eq`. Both DuckDB quicksearch sites are covered, SELECT and COUNT. Fixing one leaves the grid rendering while the row count still errors. Tests include the live path: the Database Manager sends a `-- WM_INTERNAL_DB_SELECT {...}` marker and the backend expands it, so the new test drives that expansion with the real 27-column definition captured from a failing job, `sync_id VARCHAR[]` included. It fails without the fix and passes with it. * fix(frontend): cast columns in the DuckDB quicksearch Same defect as the Rust query builders, in the implementation that actually runs. `make_select_query` / `make_count_query` in windmill-common have no callers anywhere in the repo; the query the browser sends is built here. DuckDB's CONCAT implicitly casts scalars but rejects nested types, and quicksearch concatenates every visible column, so one LIST column makes a table impossible to preview — both the page and its row count fail with Binder Error: Cannot concatenate types VARCHAR, ..., VARCHAR[], ... and TIMESTAMP WITH TIME ZONE - an explicit cast is required The helper lives in select.ts and is imported by count.ts so the two cannot drift, and both call sites are fixed: fixing only SELECT leaves the grid rendering while the row count still errors. * fix(duckdb): cast only list columns in quicksearch, leaving other SQL byte-identical Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * test(frontend): pin the DuckDB quicksearch column list byte-for-byte Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Ruben Fiszel <ruben@windmill.dev> Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>	2026-08-10 20:48:19 +02:00
Ruben Fiszel	b7d2052b03	block the whole 0.0.0.0/8 range in the SSRF filters (#10597)	2026-08-10 19:57:29 +02:00
hugocasa	330175f83b	Revert "fix: scope a fork's cloned app policy and custom path to its creator …" (#10592) ... This reverts commit 8e95bfe615.	2026-08-07 12:39:44 +02:00
hugocasaandClaude Opus 5	8e95bfe615	fix: scope a fork's cloned app policy and custom path to its creator (#10589) ... * fix: scope cloned app policy and custom path to the fork's creator Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * refactor: share the app custom-path scoping rule across its call sites Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * docs: tighten the cloned-app-policy comments Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * docs: correct the execution_mode and custom-path scoping rationale Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>	2026-08-07 12:37:04 +02:00
Ruben FiszelandClaude Opus 5	8c6211c277	feat: offer more dev workspace environment labels (#10570) ... * feat: allow custom dev workspace environment labels Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix: reject dev labels that shadow a tracked branch's namespace Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix: guard dev labels against a repo's assumed default branch Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * chore: state the badge-cap rationale once and drop unenforceable openapi constraints Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * refactor: offer a fixed list of environment labels instead of free text Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix: match the accepted label set to the openapi enum exactly Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * docs: stop describing the label set as dev/staging only Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>	2026-08-06 18:32:19 +00:00
3 people	c61404a0f4	feat: preview merge result in git-sync PR diff check (#10542) ... * feat: preview PR merge result in git-sync diff check * chore: update ee-repo-ref * fix: match pr diff sentinels as structured field, tighten comments * chore: update ee-repo-ref * fix: neutral verdict for unfetchable pr head, testable sentinel parse * chore: bump git-sync pull script pin to hub/28889 * chore: bump git-sync pull script pin to hub/28890 * fix: cover failed history deepening in unavailable-head check text * chore: update ee-repo-ref to 181fa0c206d7f84a289b4396a7f7764bc815d284 This commit updates the EE repository reference after PR #712 was merged in windmill-ee-private. Previous ee-repo-ref: 36f5c0e9d147f9eed63ebc316f2aef9f86b500af New ee-repo-ref: 181fa0c206d7f84a289b4396a7f7764bc815d284 Automated by sync-ee-ref workflow. --------- Co-authored-by: windmill-internal-app[bot] <windmill-internal-app[bot]@users.noreply.github.com> Co-authored-by: Ruben Fiszel <ruben@windmill.dev>	2026-08-06 17:27:49 +02:00
Ruben FiszelandClaude Opus 5	5ce29b3436	feat: add public sharing option for job pages (#10573) ... * feat: add public sharing option for job pages Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix: gate public run sharing and address review findings Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix: address review nits on public run sharing Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix: key public run view on workspace, job and token Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>	2026-08-06 17:18:29 +02:00
Ruben FiszelandClaude Opus 5	e203ab087a	feat: allow a dev workspace to have its own dev workspace (#10534) ... * feat: allow a dev workspace to have its own dev workspace Fixes WIN-2324 Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix: keep every dev workspace in a chain on a distinct deploy branch Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix: count a dev workspace the caller has no seat in as holding its label Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * refactor: keep the attach form standing when a candidate takes the last label Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix: keep the label toggle visible when a candidate's dev workspace clashes Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * docs: describe the cycle guard by what holds, not by what changed Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix: refuse to archive a fork-backed dev workspace that owns a nested dev Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix: put the deploy target and item filters under the pairing they configure Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix: refuse to archive any dev workspace that owns a nested dev Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * docs: fix the fixture family count Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix: put the deploy target with the pairing line it restates, above protections Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix: name the same family head in the workspace menu and the scope picker Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix: stop offering to delete a dev workspace from the sidebar settings menu Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * docs: state the visibility boundary the lineage root actually resolves to Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix: serialize dev-pairing creation against teardown of the same workspace Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix: lock both sides of an attach so adjacent pairings cannot share a label Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix: serialize dev pairings on one key, the invariant being chain-wide Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix: scope the pairing lock to the chains an operation reads Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix: hold the pairing lock across renames and re-check the cycle under it Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix: hide the fork-delete action until the workspace entry has loaded Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix: lock archive before it reads the pairing state it acts on Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * docs: describe the archive lock test by what it pins, and drop an unused fixture row Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>	2026-08-05 19:07:12 +00:00
Ruben Fiszel	340d3cd565	feat(dbt): reach any dbt adapter through a dbt_profile resource, and constrain the warehouse picker (#10525) ... * feat(dbt): reach any dbt adapter through a dbt_profile resource, and constrain the warehouse picker The workspace dbt warehouse picker listed every resource in the workspace, so a slack or github resource was an offerable answer to a field that can only be a warehouse. Constraining it exposed that the set of resource types that actually work is both smaller than the docs claim and too small to be useful: - `render_profile` translates only six adapters from a Windmill resource; the rest (clickhouse, duckdb, salesforce, mssql, oracle) refused one outright. - `redshift` and `duckdb` name no resource type anywhere, so two of the adapters the quickstart advertises were unreachable. - the `databricks` resource carries `workspace_url`, while the renderer demanded `host`, so that warehouse could never render at all. So the picker gets a constraint and dbt gets an escape hatch wide enough to make it honest. `dbt_profile` is a resource whose value IS a `profiles.yml` target — `{ type, target }` — passed to dbt unchanged, so any adapter and any key it documents works. `DbtAdapter` is now open: it carries dbt's own `type:` spelling plus an optional `KnownAdapter` (the eleven Windmill has facts about — a field mapping, a pip package, the license gate). Anything else is carried by name and installed as `dbt-<name>`, the convention every adapter on PyPI follows, so "whatever dbt supports" no longer means "whatever this enum lists". The license gate is unaffected: `sqlserver`/`oracle` still resolve to their `KnownAdapter` and are still gated. The name is confined to `[a-z0-9_-]` starting alphanumeric because it reaches a pip requirement and a venv path on the host. Two adjacent fixes fall out: the project's own `profiles.yml` and the descriptor's `profile.type` now accept any adapter instead of the closed list, and a databricks resource renders its `host` from `workspace_url`. The picker is constrained to `dbt_profile` plus the translated types, so nothing it offers can fail for want of a mapping. Fixes WIN-2320 * fix: drop the unused DbtAdapter::from_resource_type wrapper Nothing calls it: a Windmill resource type maps through KnownAdapter::from_resource_type, and the executor resolves an adapter from the resource's own dbt spelling or by inference. CI builds with -D warnings, so the dead wrapper failed every backend check. * fix(dbt): make dbt_profile the block itself, and address the review findings **A `dbt_profile`'s value IS a `profiles.yml` output block**, `type` included. It was `{ type, output }`, which asked the user to restructure their block before pasting it — a translation step, in the one type that exists to avoid translation. The schema now declares no properties, so the resource form renders a single JSON editor over the value. That means the value's shape can no longer say what it is: a `dbt_profile` and Windmill's bigquery resource are both objects with a `type` (the latter says `type: service_account`). So the warehouse carries its resource's type (`DbtWarehouseConnection.resource_type`), and detection is exact. It also makes decision 9's "the resource type name is the authority" true at runtime for the translated path, which until now resolved its adapter by sniffing fields. Review findings, all three reviewers: - **[P0] an author-chosen adapter became an unsandboxed PyPI install.** `dbt-` is not a reserved prefix, and `provision_core_1x` installs through `run_tool`, outside the nsjail ordinary dependency installation uses — so `dbt-<name>` from a script author's `type` could run a PEP 517 build backend as the worker. Now gated on a list of published adapters plus `DBT_EXTRA_ADAPTERS`, so trust stays the admin's call. The open set survives: the engines that ship their adapters install nothing and take any type. - **[P1] `type: fabric` rendered as `sqlserver`.** dbt's `type:` was resolved through the resource-type table, where `fabric` is a Windmill alias for SQL Server — so a Fabric profile installed dbt-sqlserver, was enterprise-gated, and failed on an ODBC driver without ever naming Fabric. dbt types now have their own table. - **[P1] two spellings of one adapter compared unequal.** `PartialEq` covers the carried name, so `postgres` != `postgresql` even resolving to one adapter, and the descriptor/resource check rejected valid configs with a message naming the same adapter twice. The name is normalised to the adapter's dbt spelling. - **[P2] identity keys.** `database_key` is what a Windmill resource spells it, and only translated adapters have one; the rest read dbt's `database`. - **[P2] duplicate `sslrootcert`** when a block carried both a PEM and a path. Verified with three real dbt builds: a flat `dbt_profile` postgres block, the same with `type: postgresql` under a `profile.type: postgres` descriptor (the alias case, which failed before), and trino for the unknown-adapter path. * docs(dbt): say that installing an adapter is gated, not just using one The open-adapter text promised every future adapter is installed as dbt-<name>, which ensure_adapter_installable refuses outside PUBLISHED_ADAPTERS and DBT_EXTRA_ADAPTERS. Separates the two: rendering, licensing and identity are open to any adapter, and only the dbt-core 1.x PyPI install is gated, because that is the step that runs outside the sandbox. * fix(dbt): keep a dbt_profile's own sslrootcert when Windmill writes none The previous round skipped the block's sslrootcert unconditionally to avoid emitting the key twice, which drops a path-only CA reference — a certificate baked into the image or mounted on the worker, which is the block's own trust source. Skipped now only when a root_certificate_pem is present, which is when Windmill writes a replacement. * fix(frontend): let a resource type declare no properties A schema without `properties` is a JSON-edited resource type, not a broken one - `dbt_profile` is a profiles.yml block whose keys belong to its adapter, so there is nothing for Windmill to declare. Both editors assumed properties exist: - ResourceEditor threw on Object.keys(undefined) while deriving the field order, which left the drawer on its loading skeleton forever, so the resource could not be viewed or edited at all. - ApiConnectForm caught the same throw and reported the type as missing from the workspace, offering to sync a type it already had. Both now fall back to the raw JSON editor, which is what usesRawEditor already intended for a schema with no properties. * chore: cut the new comments to AGENTS.md's four-line cap Each still states its constraint once; the long-form rationale belongs in docs/dbt-runtime.md and the PR, not beside the code. * fix(dbt): keep a dbt_profile's empty and nested collections intact A block with no children reads back as null, so `extensions: []` reached the adapter as a missing value rather than the empty list dbt was handed, and a nested array went through the scalar path and arrived as a quoted JSON string. Both are keys dbt passes to the adapter as it finds them, so the type has to survive: empty collections are emitted inline, and the value half of an entry recurses instead of bottoming out at a scalar. The test parses the rendered YAML back rather than string-matching it, since what matters is what a YAML reader sees. Also cuts DbtWarehouseConnection.resource_type's comment to the four-line cap.	2026-08-05 00:34:26 +02:00