59 Commits

Author	SHA1	Message	Date
Ruben Fiszel	71b9989daa	feat: auto-build binaries to object storage on deployment (#10673) ... * feat: auto-build binaries to object storage on deployment Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix: queue the auto-build from pre-locked deploys and off the lock slot Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix: materialize companion modules before a deploy-time build Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix: keep a build job from stamping lock_error_logs on a healthy script Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * test: de-flake test_flow_lock_all and surface the lock error it hides Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * test: trim drafting history from the flow-lock fixture comments Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix: stop a binary build from restarting dedicated workers Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix: keep the build-job marker off the agent wire and out of user args Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>	2026-08-13 07:50:28 +02:00
Ruben Fiszel	fb82748296	fix: make on_behalf_of control permissions for scripts and flows (#10438) ... * fix: make on_behalf_of control permissions for scripts and flows Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix: inherit the recorded on-behalf-of identity when a preserving deploy omits it Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix: keep an omitted permissioned_as from re-versioning an unchanged script Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix: derive the on-behalf-of principal from the email and reject mismatched pairs Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix: stop workspace deploys from carrying a source-workspace principal Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * docs: correct the onBehalfOfPermissionedAs param doc Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * test: pin that workspace deploys never carry a source-workspace principal Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * docs: correct the omitted-principal contract and refresh generated prompts Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix: keep external-superadmin principals on email-only redeploys Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix: scope the recorded principal to its workspace and prefer real accounts Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix: carry the recorded principal correctly through drafts and set-permissioned-as Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix: sweep draft identity pairs on email change and offboarding Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix: leave group identities alone when sweeping a user's email Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix: treat only g/ without an email as a group, and match the offboard preview Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix: stop the group guard from skipping rows with no recorded principal Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * docs: state the group guard once instead of restating it Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * refactor: make the permissioned_as the only stored on-behalf-of identity Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * perf: skip resolving the on-behalf-of address for sync clients that discard it Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix: address the local review of the identity refactor Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix: resolve the on-behalf-of identity coherently across clones, offboarding and no-op deploys * test: pin that a fork keeps only the on-behalf-of identities that resolve in it * fix: decide a principal prefix-first everywhere and canonicalize bare addresses * fix: prefix a slash-containing address so a reader cannot take it for a group * fix: read an address as a username before the group- convention * fix: rewrite the canonical principal when an account's address moves * fix: keep the address form of a principal to accounts without a usr row * fix: reject an identity a job row cannot carry and read it uncached at dispatch * fix: count characters against the job identity width and cap the backfill * refactor: name the script/flow principal on_behalf_of, as apps do * docs: state the caller-must-authorize contract on the identity resolvers * fix: keep writing on_behalf_of_email until every worker reads the principal * fix: err high on the compatibility version and document the last resolver * fix: keep the compatibility address current through identity mutations * fix: carry the compatibility address with the principal on every copy path * chore: re-pin the EE ref to the companion branch merged with EE main * fix: key the dbt retry lookup on the stored principal * fix: keep a mixed-version address recoverable through a fork * fix: read a round-tripped address uncached so a redeploy is not rejected * fix: refuse an email change that would make a principal unenqueueable * chore: update ee-repo-ref to ac3d7d015296f041ae44ab6bc4953485f44d36e4 This commit updates the EE repository reference after PR #704 was merged in windmill-ee-private. Previous ee-repo-ref: 219b0b03905a1a0028054b3a4985724e77d09036 New ee-repo-ref: ac3d7d015296f041ae44ab6bc4953485f44d36e4 Automated by sync-ee-ref workflow. --------- Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com> Co-authored-by: windmill-internal-app[bot] <windmill-internal-app[bot]@users.noreply.github.com>	2026-08-01 20:37:21 +02:00
Ruben Fiszel	032300e28e	feat: run dbt projects as a first-class Windmill runtime (#10326) ... * fix: mount only the engine in the dbt jail, reject shadowed and malformed args Review round 42. The jail mounted the whole dbt cache directory, whose siblings of the engine are `repos/` and `packages/` — other workspaces' private checkouts and package trees, kept apart by cache key rather than by permissions. A jailed project could read them. It now mounts the engine's own directory, which the provisioner names; verified from inside the jail that `repos/`, `packages/` and `state/` are invisible while the engine stays usable. A `{{ placeholder }}` may no longer take the name of a run argument this runtime defines. It was silently dropped from the signature, so a descriptor like `value: "{{ select }}"` deployed and then could not be run at all: the built-in `select` is an array and the interpolation needs a scalar. Refused at parse, so the deploy says so. A `vars` override that is not an object is refused rather than ignored. Argument-schema validation is opt-in, so a string or an array silently ran the descriptor's own vars — against a different schema or alias than the caller asked for. `select` and `exclude` already refused theirs. * feat(dbt): the project is the script's module bundle, not a git checkout A dbt script now carries its whole dbt project as its module bundle. The descriptor is the script content; `<script>__dbt/` holds the project verbatim, so importing an existing project is `cp -r` plus `wmill sync push`, and the worker materialises the bundle into the job directory instead of cloning. Backend - `prepare_project` writes the script's modules and requires `dbt_project.yml` at the bundle root. `checkout`, the git-ssh command, the clone cache and the repository resource are gone, along with `repo`, `project`, `ref` and `git_ssh_identity` on the descriptor. - Run identity and the package cache key take a `project_digest` (sorted SHA256 over the bundle) where the commit used to sit, so an edited project cannot resume a previous run's `run_results.json` or reuse its `dbt_packages`. - The per-run graph re-ingest is now gated on `vars` placeholders and `$var:` env alone. - `capture_dependency_job` takes the script's modules so a dependency job, which has no generic module-writing step, materialises them itself. - `dbt deps` caching strips the git remote from every package it cached, not just the tree root: `packages.yml` can render a token into a `git:` URL. - `git_clone.rs` is dropped and `ansible_executor.rs` returns to its own copy of the clone helpers. CLI - `wmill sync pull` keeps a dbt script's lock beside its folder rather than inside it, so the folder holds nothing but the project. - Directories dbt generates (`target-path`, `packages-install-path`, `clean-targets` and the usual defaults, read from `dbt_project.yml`) are excluded from the bundle, from the sync diff and from staleness hashing. - A module-only edit now pushes its parent dbt script and is reported as a changed module rather than passing unnoticed. * fix: keep a script's modules in the worker's file-system cache The first fetch of a script version reads the database and carries its modules; every later fetch imports from the worker's cache directory, whose `RawScript::import` hard-coded `modules: None` and whose `export` never wrote them. A worker restart therefore started running the script without its own files, silently — for a dbt script, without its project, which fails with "carries no project"; for any other script with a module bundle, with the imports missing. `modules.json` is now written on every export and required on import, so an entry written by an older version fails to import and is refetched rather than serving a stripped script for as long as the directory lives. Also derives a dbt run's `project_digest` from the bundle the run actually carries: `handle_dbt_job` was passing `None`, which collapsed every project in a workspace onto one digest and let `dbt retry` resume a different project's `run_results.json`. * fix(dbt): give every phase the script's environment, bound the cache copies `dbt deps` ran without the script's environment variables on an unsandboxed worker, so a `packages.yml` resolving a private package URL through `env_var()` could not see them while the package cache key was still built on their digest. `with_invocation_env`, applied at three of the four call sites, is folded into `dbt_command` so no phase can be added without it, and `DBT_TARGET_PATH` is set after both environments rather than before. The package cache copies ran through a bare `Command::output()`: the tree is the project's, so a cancelled or timed-out job held its worker slot until `cp` finished. Both the restore and the publish now run under the job poller like every other phase. * fix(dbt): only offer commands whose writes match the graph, honour packages-install-path `dbt_command: run` is dropped from the allowed overrides. Asset dispatch fires a script's deploy-time writes on any successful job, and `dbt run` covers models only, so a project with seeds or snapshots notified consumers of relations the invocation left stale. That is the same reason `test` was already excluded. Narrowing what a run touches is `select`/`exclude`, which scope the graph too. `dbt deps` writes to the project's `packages-install-path`, so a project that moved it got no package cache at all: the publish found nothing at `dbt_packages` and every job resolved its dependencies over the network again. The path is read from `dbt_project.yml` and validated as project-relative, since both cache copies are rooted at it. Also states the sidecar's mutator contract at the module level: the dbt manifest tables carry no RLS and grant `windmill_user` full access, so a user-scoped transaction is not enforcement and every caller must have verified write access to the script itself. CLI: a module file is now grouped with its parent script for the push. Left in a group of its own it got its own `alreadySynced`, so a push touching several files of one bundle deployed the script once per file; the resulting versions raced, and the asset graph could end up describing none of them. * fix(dbt): seed a project for browser-created scripts, refuse a no-op retry A dbt script created in the browser only got a descriptor, and the runtime refuses a script whose bundle has no `dbt_project.yml`, so the advertised Create → Deploy → Run path always failed its dependency job. New dbt scripts now start with a project that builds: pointing `profile.resource` at a warehouse is the one edit, and growing it is `wmill sync pull` plus a local editor, which is where dbt development happens. `dbt retry` builds its graph from the previous run's error, fail and skipped nodes alone, so retrying an all-green run selected nothing and wrote nothing — and a job that succeeds having written nothing still dispatches every deploy-time write, waking every downstream consumer for relations no one touched. Refused, with the reason. CLI: a configured `target-path` or `packages-install-path` may be nested (`build/target`), and `clean-targets` has a block form as well as an inline one. Both are now parsed, and the exclusion compares the project-relative path rather than the top-level segment, so a nested generated tree no longer lands in the bundle and no longer makes a local `dbt run` look like a project change. * fix(dbt): lock a project once, find the parent on either path separator A dbt script's modules are its dbt project, not helper code with dependencies of its own, so the generic per-module lock loop is skipped for it: the parent lock already ran `dbt deps` and `dbt parse` over the whole project. Locking each file separately re-materialised the bundle and re-invoked dbt once per file, so a project of N files paid N project-sized passes and a large one timed the deploy out. The 13-file fixture went from 14 relock passes to 1. `pushParentScriptForModule` searched the raw path for `__dbt/`, so on Windows, where the folder is spelled `__dbt\`, a module-only edit returned without deploying its parent while the caller still recorded the file as synced. It now goes through `getScriptBasePathFromModulePath`, which normalizes separators. Also drops the last of the external-repository wording from the descriptor's module docs and from the `codebase` rejection a user can hit. * feat(dbt): infer the run form locally, keep test-only retries from cascading `windmill-parser-wasm-yaml` 1.770.0 carries `parse_dbt`, so the browser and the CLI derive a dbt script's run arguments from its descriptor instead of waiting for the deploy to hand back a schema. Pins bumped in both. `generate-metadata` was rewriting a dbt script's `lock` field on every run: a dbt lock comes from the dependency job on a worker, so nothing generates it locally and the resolved `!inline` reference was left inlined into the metadata or blanked. It is restored instead, and a push straight after `generate-metadata` is a no-op again. A retry now needs a failed node that materialises something. `dbt retry` builds its graph from error, fail and skipped nodes, and with `test_behavior: after_all` a failing test is what `run_results.json` ends up describing — so the retry reran tests, wrote nothing, succeeded, and still dispatched every deploy-time write. The dbt badge's destination is deterministic: writers outrank readers, and among several writers of one relation (which the backend permits) the smallest id wins, rather than whichever write edge arrived last. * feat(dbt): browse the project and read a run's per-node result Two views a dbt user expects and that the generic script surfaces do not give. **The project.** A dbt script's editor gains a Project tab beside its descriptor: the module bundle as the tree dbt itself expects, each file read-only with syntax highlighting. The existing module tab strip is a flat row built for a couple of helper files and does not survive a real project; a 13-file fixture already overflows it. Directories sort before files so it reads like the checkout on disk, and an empty bundle explains the `cp -r` instead of showing a blank pane. **The run.** `DisplayResult` renders a dbt invocation's per-node breakdown above the raw payload: totals, then a table of node, kind, target relation, rows and time, with failures and warnings sorted first and carrying their message. The data was already structured; it was being shown as JSON to scroll and PASS/WARN counts to find in the log. On a failed run the same JSON rides in the error message after the exit-status line, so it is parsed back out — that is the case worth rendering, since the failing node is what the user came for. * docs(dbt): say that profile.resource is what buys the asset graph The starter descriptor described `profile.resource` as the thing rendered into profiles.yml, with the project's own file as an equal alternative. It is not equal: the resource PATH is the warehouse's identity in the asset graph, so a project bringing its own profiles.yml runs fine and silently gets no assets, no lineage and no cascade. The deploy already says so in its log; now the descriptor a user starts from says it too, before they choose. * fix(dbt): authorize a resource used only for asset identity, clean up after failed installs A descriptor setting both `profile.profiles_yml` and `profile.resource` took its connection from the project's file but returned the resource path as the graph's warehouse identity without ever reading it. A script editor could therefore publish `table://<any resource>/...` writes, and wake that warehouse's subscribers, while connecting somewhere else. The resource is now read on that path too — reading is what authorizes it — so the combination keeps working for the case that wants it (keep your own profiles.yml, still get lineage) and fails closed otherwise. Provisioning cleaned up its staging directory only on the paths someone remembered, so a run of failed or cancelled first-use installs accumulated venvs, tarballs and installer scripts until the worker's disk was gone. All three engines now hold their scratch paths in a guard that removes them on drop, which is the one exit every path takes, cancellation included. Frontend: `partial success` is dbt's word for a node that built but whose tests failed, counted in `totals.error` and redone by a retry, so it ranks with the failures instead of rendering green with its message hidden. And the run panel now keys off the worker's engine discriminator rather than `{nodes, totals}`, which is a shape an ordinary script can return. Both pinned by unit tests on the extracted `parseDbtRun` helpers. * feat(dbt): show a run's models on the run page The run page is where you land on a running job, and until now it showed a dbt run as streaming text: the per-node table only renders once the job has produced a result, and the graph that moves per model lived on the pipeline page you had to navigate to. A Models section now sits above the result, scoped to the running script's own relations and its `ref()` lineage, polling while the job is in flight so nodes move as dbt walks the DAG. No `resolveGraph`: that merges drafts and live editor buffers into the persisted graph, and a run page has neither. * feat(dbt): retry failed nodes automatically, and from any worker **Node-level retry, in the job.** `retry_failed_nodes: {attempts, delay_seconds}` rebuilds only what a failed build left failed or skipped, before the job reports failure. dbt confines a failure to its own subtree and `dbt retry` resumes exactly that set, so a transient warehouse error costs those nodes rather than the project. Doing it in-job is what keeps the state question out of it: the previous attempt's `run_results.json` is still in the job directory, so there is nothing to persist and no worker to land back on. This is the granularity astronomer-cosmos gets from one Airflow task per model, without the ~6x that per-model tasks measured. A retry's `run_results.json` names only the nodes it redid, so it overlays the accumulated results rather than replacing them: the job's result has to be every node the job touched, or the nodes that succeeded before the retry settle no materializations. Pinned by a test. **Durable retry state.** `run_results.json` is now saved to `dbt_run_state` as well as the worker's local cache, so an explicit `dbt_command: retry` works from any worker of the group rather than only the one that failed. Only the results are stored: `dbt retry` also needs `manifest.json`, roughly sixty times larger and growing with the project (732 KB against 12 KB on the six-node fixture), but the manifest is a pure function of the project files, vars and env, all of which the stored identity already pins, so a worker restoring from the database re-derives it with a `dbt parse` of about a second. * fix(dbt): restore the sqlx cache, make retries cancellable and path-aware **SQLx cache.** A `cargo sqlx prepare` deleted 750 entries, including the enterprise queries CI needs under `SQLX_OFFLINE=true`, and the check that was supposed to catch it reported zero losses because it was run from `backend/` with a `backend/`-prefixed path, so its baseline was empty and it failed open. All 750 are restored; the branch now adds 19 and deletes none, and `SQLX_OFFLINE=true cargo check` passes. **Retry backoff observes cancellation.** `canceled_by` is only written by the job poller, which does not run between attempts, so re-reading it reported the state as of the failed attempt and missed every cancel issued during the wait — the whole window the check exists to cover. The wait now reads `v2_job_queue.canceled_by` each second, and the job's deadline is honoured before starting another dbt process. **Retry state follows its script.** `dbt_run_state` is path-keyed like the manifest sidecar but, unlike it, nothing regenerates it: a rename moves the row so a resumable failure survives, while archive and delete clear it, so a script later created at that path cannot inherit a stranger's failure and its arguments. **CLI.** `table` joins ducklake and s3object in the local graph's auto-trigger kinds, matching `is_auto_trigger_kind` and the frontend's set; without it a local graph and the generated docs omitted a cascade edge the deploy has. * fix(dbt): carry only the project files a bundle can hold, and say what it drops Exploring real and edge-case projects surfaced three frictions, all in the import path a user hits first. **A binary file broke the push, opaquely.** dbt projects carry images under `docs/`, stray `.DS_Store` files and occasionally a parquet seed. Read as text they become mojibake, and a NUL among them is rejected by Postgres with `unsupported Unicode escape sequence` — which `wmill sync push` then reported as success, exiting 0 with the script never created. Binary files are now detected the way `git` detects them, by a NUL in the first 8000 bytes rather than by extension, and skipped with the reason. **The size guard the docs promised did not exist.** Now it does: 5 MB per file, which only ever catches a committed dataset. Real dbt code is about 500 bytes median and 1.9 KB at p90. **Skipped files became a permanent phantom diff.** The push dropped them while the sync diff still offered them, so every push reported changes no push could resolve. One predicate now answers for the push, the staleness hash and the diff. Verified on a project with unicode filenames and content, CRLF endings, an empty model, an ephemeral model, a disabled model, a `.md` docs block, an extensionless README, six levels of nesting, a 7.6 MB seed and a PNG: it pushes, round-trips byte-for-byte through pull, deploys to 7 dbt nodes and 6 `table://` assets (ephemeral and disabled correctly absent), and runs green. * fix(dbt): resolve dbt-core against the adapter, settle partial success, unify status **Adapters could not be provisioned.** The 1.x engine pinned `dbt-core` to a fixed version independent of the adapter, but several adapters cap below it: `dbt-mysql` at `~=1.7`, `dbt-oracle` and `dbt-databricks` below 1.12, and `dbt-salesforce` has no package at all (it exists only inside Fusion). Those projects failed at provisioning with a uv resolver dump. The install now asks for a range and lets the adapter choose, and records what the resolver picked so the lock pins a version that adapter can take. The floor is the CLI this runtime invokes: resolving down to dbt-core 1.7 produced a working venv that then failed with `No such option '--target'`, which is worse than not resolving. An adapter with no release in range now fails naming itself and pointing at `dbt-core-2x` or `fusion`, instead of a resolver dump. Salesforce is refused up front with the reason. **`partial success` left a model stuck on `Running`.** It is dbt's word for a node that built and then failed its tests, and it was already treated as a failure when counting totals and deciding a retry — but the two sites that settle the RELATION fell through to "says nothing", so the tailer's `Running` was never replaced and a finished job showed a model still building. Six status comparisons had drifted apart, two folding case and four not, while dbt-core 1.x echoes the author's casing and 2.x uppercases; they are now one classifier. **Agent workers.** The durable retry state and the cancellation poll both need a database, which an agent worker reaches only through the API. The automatic node retry is refused there rather than running a wait it could not interrupt, and the docs say "any worker with a database connection" instead of overclaiming. Also clears `dbt_run_state` when a path stops being a dbt script, and moves `run_identity`'s contract onto `run_identity` from the digest helper below it. * feat(dbt): show the transform behind a model on the run graph The run page's graph carried a node for the script itself and drew every relation as a bare table. Both were wrong for that page: the graph there is already scoped to one script, so a node standing for it distinguishes nothing (on the pipeline page it separates one project from another, which is why it exists), and dbt's own DAG node is the model — the SQL and the relation it writes are one thing, so a graph of relations alone leaves out what a reader came to see. The script node is dropped, and selecting a model now shows its SQL underneath the canvas with its file path and materialization, read-only, the same view the pipeline details pane gives. * feat(dbt): move the graph with the run The worker has always recorded a state per relation as dbt walks the DAG — `running` when a model starts, `materialized` or `failed` when it ends — but nothing rendered it: the graph response carries what a relation IS, not what a particular run is doing to it, so the canvas had nothing to show and a running job looked identical to a finished one. `assets/run_progress/{job_id}` returns that state for one job, the run page polls it beside the graph, and the asset node carries a spinner or its outcome. Errors and retries need nothing extra: a failed node writes `failed`, and an in-job retry rewrites the same row, so the node returns to `running` and on to its new outcome by itself. `materialized_partition` holds a relation's CURRENT state keyed by relation, so filtering on `job_id` returns exactly what this run last touched — which is the question a run page asks, and why a superseded older run shows nothing. * feat(dbt): a dbt project is not a data pipeline Deploying a dbt script marked it `auto_kind = 'pipeline'`, which enrolled it in pipeline membership: the folder became a Pipeline entry on the home page, the script folded into it, and `/pipeline/<folder>` opened a canvas holding the project's whole model DAG next to the pipeline's own scripts. A folder holding both then read as two projects in one editor, and the pipeline editor offered to author transforms that are in fact authored in a local `dbt run` loop and pushed as the script's bundle. A dbt script is now never a pipeline member, and the pipeline canvas drops the dbt script node. Its models stay, with their `ref()` lineage: the relations are what a downstream pipeline script reads, and dropping them would break the cascade from a dbt run — the point of giving dbt models `table://` identity. Also drops a screenshot committed to this branch by accident. * fix(dbt): authorize run_progress through the job, drop dbt from the local graph `run_progress` read `materialized_partition` through `user_db` on the assumption that RLS would scope the rows. That table has RLS disabled and no policies, so any workspace member could pass a job id and read that run's relation paths, row counts and error text. It now joins `v2_job`, which does carry per-user policies, so a caller who cannot see the job sees nothing — the same pattern `v2_job_completed` reads need. Verified as a plain member: the old query returned 6 rows for another user's run, the new one returns 0, while the job's owner still sees all 6. The CLI's local graph still forced `in_pipeline` on every dbt script, so `pipeline docs --local` and `pipeline dev` kept presenting a dbt project as a pipeline the deploy no longer enrolls. It now skips them, matching the server. A dbt descriptor has no asset parser locally, so nothing is lost: its models come from the manifest the deploy derives. Declares `run_progress` in openapi.yaml so the frontend uses the generated client instead of a handwritten fetch; the generated `status` union also replaces a hand-rolled string mapping. * fix(dbt): drop the dbt node from the CLI's deployed pipeline views too `pipeline dev` and `pipeline docs` (without `--local`) read `/assets/graph` directly. That endpoint is asset-usage driven rather than membership driven, so it returns a dbt script like any producer — and both commands render every runnable, so a dbt project still showed up as a pipeline script there after the local builder stopped emitting one. `hideDbtRunnables` mirrors the frontend's projection of the same payload. It is generic over the graph shape so the bounded-cascade view (`BCGraph`, a narrower type over identical JSON) passes through without a cast. The relations stay: they are what a downstream pipeline script reads, and the node is what attributes them to a producer for every other consumer of the endpoint, so the filter belongs in the views rather than the query. * fix(dbt): narrow a selective run's cascade, settle the finished run graph Review-round fixes. A `select`/`exclude` run builds part of the project, but asset dispatch reads the deploy-time write set for the whole script, so a run selecting one model woke the subscribers of every other. Dispatch now intersects that set with the relations the run actually recorded as materialized, scoped to dbt because it is the only producer whose write set is decided per run. A run that recorded nothing still dispatches everything, so an agent worker whose reconciliation failed cascades as before. Verified both ways: `select: [extra_model]` no longer wakes the `fct_orders` subscriber, and a full run still does. `hideDbtRunnables` keyed its removal set on path alone while the graph keys runnables by `(usage_kind, path)`, so a flow sharing a path with a dbt script lost its node, edges and triggers too. Both copies now key on the pair. The run graph never took a final reading when a job finished, so the last state shown was whatever the tick before completion saw. Only `dbt-core-1x` streams node events; the other engines record every relation during end-of-run reconciliation, so their finished graph showed nothing until a reload. `DbtNodeOutcome::Inconclusive` collapsed statuses the tally has to tell apart, so two sites re-lowercased the status beside the classifier and `no-op` landed in `totals.error` — a clean run reporting an error in its own result. Split into Warn / Skipped / NoOp / Unknown so every site falls out of one match; `no-op` is kept out of the retry set, which dbt spells as error / fail / skipped. Also: reattach two doc comments to the items they describe, and correct the engine-distribution table — only dbt-core-2x is baked into the images, 1.x is a per-adapter venv provisioned on first use, and the default is compiled in rather than an instance setting. * fix(dbt): make the model chip inert where its project node is not on the graph The canvas passed `onDbtSelect` unconditionally, so the chip always rendered `cursor-pointer` and hover-highlighted — but the owner map is empty on both graphs this feature added, since the run page carries no runnables and the pipeline page hides the dbt node. The chip advertised a click that resolved to nothing. It now takes its handlers only when the relation has an owner on this graph, so it stays live on the surfaces that do show the project node. `classify_status` and `DbtNodeOutcome` were `pub` in a private module with no caller outside the file, unlike every neighbour. * fix(dbt): take the cascade's write set from the run's own result The previous narrowing read `materialized_partition`, which was wrong twice. That table keeps one row per relation and the newest writer takes `job_id`, so two overlapping runs over the same model erase each other's claim to it: the earlier job would dispatch a subset of what it built, or none of it. And an empty row set was read as "recording failed, dispatch everything" when it is also a real answer. A `select` matching no model, or one resolving to tests only, exits 0 having built nothing — and then woke every consumer of every model in the project, which is the opposite of what the narrowing exists to do and is reachable by a typo in a run argument. The run now reports the relations it materialized in its own result, which is immutable and per job. Absent means the producer said nothing (a job from before the field, a non-dbt producer) and the whole deploy-time set dispatches as before; present-and-empty means it built nothing and dispatches nothing. Verified on all three: an unmatched selector builds nothing and wakes nobody, a selector naming one unsubscribed model wakes nobody, and a full run wakes the subscriber. Also indexes `materialized_partition (workspace_id, job_id)` -- the run page polls that shape every 2s and no existing index leads with `job_id` -- corrects the selective-cascade section of the design doc, which still described the old deploy-time behavior, and reattaches `buildLocalPipelineGraph`'s doc comment. * docs(dbt): attach the CLI JSDoc to its function, correct the index rationale The `hideDbtRunnables` JSDoc ended up documenting the type declared beneath it — made while fixing the same mistake one function down. The migration's comment credited the cascade with a `job_id` lookup that the same commit replaced with a read of the job's own result. The run page's poll is the only reader keyed on that column. * fix(dbt): refuse graph publication for a removed script, allow test-only retries An archived or hard-deleted script could still republish its graph: the publication guard filtered `deleted` but not `archived`, and treated a missing row as "nothing newer exists" rather than "nothing left to publish for". A dependency job or dynamic run finishing after the removal put the asset, provenance and subscription rows back with nothing left to clear them. `dbt_command: retry` refused a run whose only failures were tests, which is precisely what `test_behavior: after_all` produces. That restriction existed because a successful job dispatched its whole deploy-time write set, so a test-only retry would have woken every consumer for relations no one touched — the cascade now dispatches what the run reports materializing, so it wakes nobody and the restriction only blocked a legitimate retry. * fix(dbt): gate run progress behind the job-read check, not RLS alone The endpoint joined `v2_job` so RLS would decide visibility, which it does — but `require_job_read_access` adds two things RLS does not: a scoped token's `if_jobs:filter_tags` restriction, and the app-embed cutoff that stops untrusted app JS from inheriting the viewer's broader job access. A scoped or embed token could therefore read relation names, statuses, row counts and errors for jobs the ordinary job endpoints deny it. That helper is private to `windmill-api`, which depends on `windmill-api-assets` rather than the reverse, so the endpoint moves to the job routes instead of the check being duplicated. It is job-scoped anyway: `/w/{ws}/assets/run_progress/{job_id}` becomes `/w/{ws}/jobs/run_progress/{id}`, and the frontend follows the generated client. * feat(dbt): a dbt run does not trigger downstream runs dbt orders its own DAG, so a cascade only ever adds one thing: waking a Windmill script that reads a mart. That edge is narrow, and only half of it can even be expressed — nothing outside dbt can declare a `table://` write, since `// materialize` accepts DuckLake targets only, so an ingestion script cannot wake a dbt project. Against that, dispatching correctly is not cheap. A run's `select` can build any subset of the project, so the deploy-time write set is not what ran; using it wakes consumers of relations the run never touched, and narrowing it needs a per-job record of what was built. The per-relation state table cannot supply one (it keeps a single row per relation stamped with the last writer), and the result field added for it made a run's own output carry the cascade's bookkeeping. So `asset_dispatch` returns early for `ScriptLang::Dbt`, before the producer gate. dbt still materializes, records per-model state and publishes its graph: models, `ref()` lineage and live run progress are unchanged, and a `# on table://<mart>` reader still renders beside the model it reads. It simply does not fire. Wiring it up later means deciding what a selective run should notify, which is the actual work. Verified: a full run of a 6-model project succeeds and starts nothing, where it previously triggered its subscriber; the run page still reports all 6 relations and the folder graph still carries 14 tables and 8 ref() edges. * fix(dbt): remove the cascade surface, settle stranded models, fix nested __mod Stopping dispatch left its surface behind. `table://` was still an auto-trigger kind, `persist_ingest` still derived subscriptions from a manifest's reads, and the deploy still accepted `# on table://` — so the canvas drew cascade arrows into scripts nothing could wake. All three are gone: the kind no longer derives, the ingest only deletes rows earlier versions wrote, and the deploy refuses the annotation with a message saying why rather than persisting a silent no-op. `DescriptorTriggers` went with them; every field it parsed was cascade config. A model marked `running` by the live tailer was never settled when the run did not finish: reconciliation only revisits nodes `run_results.json` names, and a cancelled or timed-out run has none for the model in flight, so the finished job showed a relation building forever. It is now settled on every exit path. Verified by cancelling a run mid-flight: 3 models `running` before, 3 `failed` after, none stranded. `getScriptBasePathFromModulePath` took the first matching suffix rather than the outermost boundary, so `proj__dbt/models/legacy__mod/a.sql` resolved to `proj__dbt/models/legacy`. dbt owns its directory names verbatim, so a folder ending `__mod` is legal inside a project, and a module-only sync would have looked for a descriptor that is not there and skipped the deploy. * fix(dbt): colour a finished run's models from its own result `materialized_partition` keeps one row per relation stamped with whichever job wrote it last, so reopening a run showed only the models no later run had touched since — down to none for an old run, which reads as a broken page rather than as stale data. Reproduced: a 6-model run reported 6 relations, then a second run rebuilt one shared model and the first reported 5. A finished run already carries the answer. Its result lists every node with a status, and the graph carries each asset's dbt `unique_id`, so the two join directly — no path derivation, nothing stored twice, and nothing a later run can overwrite. The endpoint stays for the live window, where the result does not exist yet, and as the fallback for a run that never produced one (cancelled or killed, whose relations the worker settles in the table instead). `relationOutcome` mirrors the worker's `classify_status` so the colour drawn over a record agrees with the record: `warn`, `skipped` and `no-op` leave the relation untouched and stay uncoloured, as do tests and analyses, which match no asset. Verified in the browser on the run whose model had been stolen: all six relations green again, both sources correctly uncoloured. * docs(dbt): record why only dbt-core 1.x has live per-model progress `emits_node_events()` reads as "the Rust engines produce no node events", which is false and would close off the option. They produce exactly the same events; they put them on the console and ignore `--log-format-file json`, which both accept. Measured on 2.0.0-alpha.5 and fusion 2.0.0-preview.202: 15 node events each on stdout, 0 in the file log, for a three-model project. Taking them means owning the job log's presentation to work around a flag that is documented and simply unimplemented, so the note records the measurement, the sample event, and that flipping the predicate is the whole change once either engine honours it. * fix(dbt): give HighlightCode a dialect-agnostic sql language `npm run check` had three errors the fast check does not reach: `"sql"` is not a value `HighlightCode` accepts. Every SQL dialect it knows maps to one grammar, but a dbt model is compiled by whichever adapter the project targets, so naming a dialect would be a guess — `sql` is now a value in its own right. `langOf` was typed `string` and returned `markdown`, `python` and `text`, none of which the component accepts either, so a dbt project's YAML and Python files rendered unhighlighted. It now returns the component's own prop type, which is what caught them, and `undefined` for what has no grammar rather than a name that silently means the same thing. Verified in the project panel: SQL 22 tokens, YAML 27, where YAML was plain. * fix(dbt): stop failing no-op models, drop table triggers client-side, keep cross-selection edges The sweep that settles a run's stranded relations was marking `no-op`, `warn` and `skipped` models FAILED on successful runs: reconciliation reports those nodes without settling their record, so they were indistinguishable from a model the run never reached. It now excludes every relation the run accounted for, so only the genuinely abandoned ones are settled. `table` was removed from the backend's auto-trigger kinds but left in both client mirrors, so the editor and `pipeline dev`/`docs` kept drawing cascade arrows the deploy will not create. `isModuleEntryPoint` scanned for the first `__mod/`, the same bug its sibling just had: a `legacy__mod/script.ts` nested in a dbt project — dbt owns those names verbatim — read as that script's entry point. Both now anchor on the outermost boundary. A script selecting a model whose parent another script builds dropped the parent entirely, so no `dbt_edge` could reach it and the two relations sat on the graph unconnected. The parent is now kept as an endpoint and recorded as a READ, since this script does not build it — splitting a project across selections only composes if the seam still draws. * fix(dbt): don't double-run after-all tests, count only models a script builds An `after_all` run whose test phase failed saves a `run_results.json` holding tests alone. Retrying it reran exactly those tests — and then the test phase ran the whole suite again, appending a second copy of every result: duplicate ids in the run table, doubled totals. A retry whose saved results are tests alone IS the test phase, so the suite is not run after it, and the two phases now merge by node id rather than concatenating. Keeping a selection's unselected parents as nodes made them count toward the `×N` badge, whose tooltip says "materializes N models" — a script selecting one mart claimed the staging models upstream of it, and the number grew with the seam. The count now comes from the relations the script writes. That change also made the cross-selection read block dead, with a comment asserting the inverse of what now happens; it is removed, and the test that covered it still passes on the new arm. The test I added landed between a neighbouring test's comment and its `#[test]`, orphaning the attribute so that test stopped running. Two display fixes: the run page no longer shows a relation's SQL when the provenance belongs to another project that materializes the same relation, and the editor no longer draws an explicit `# on table://` arrow the deploy refuses. The starter descriptor no longer promises the removed cascade. * fix(dbt): clear untouched models, reject unknown descriptor fields A `no-op` model was left `running` forever on a successful run. The previous attempt at this stopped the sweep marking such models FAILED but gave them no terminal state instead, so they simply never settled. Reconciliation now returns what it settled and what the run reported but did not build, and the two get opposite treatment: a relation the run left untouched has its row DELETED, which is what the finished run's own result says about it (`relationOutcome` colours a `no-op` nothing), so the live and settled views agree; only a relation the run never reached at all is failed. The descriptor accepted unknown fields, so `selcet:` was ignored and left an empty selection — building the whole project — and a misspelled `target` fell back to the profile's default. It rejects them now. That immediately caught two of our own test fixtures still passing `repo:`, a field removed with the git path, which is exactly the class of mistake it exists to stop. `isDbtModulePath` matched `__dbt/` anywhere in a path, the third site with that bug: `foo__mod/vendor/x__dbt/a.ts` read as a dbt project file, and the push then looked for `foo.script.yaml` and could skip the edit. A verbatim dbt bundle dropped any file named `*.lock` before it reached the module map, so an authored `uv.lock` never deployed and the unmodified-project round trip quietly lost it. The exclusion now applies only to `__mod` bundles, where `.lock` really is the script's own lockfile — in the walker that hashes modules too, or a change to such a file would not register as one. Also: `langOf` fell back to `undefined`, which HighlightCode resolves to TypeScript rather than to no highlighting, so seeds and Markdown were coloured as code; and four comments still gave the removed cascade as the reason for sharing an asset node, which is now lineage. * fix(dbt): retry failed tests too, anchor the last __dbt path check `retry_failed_nodes` only ran after the model phase, which fails before the `after_all` test phase exists — so a project whose models built and whose tests failed got no retry at all, exempting exactly the failure mode that separate phase produces. The loop is now a function, called after both phases. `isDbtGeneratedPath` matched `__dbt/` anywhere, the fourth site with that bug: `foo__mod/vendor/x__dbt/target/a.ts` counted as generated dbt output, so `ignoreF` excluded an ordinary module file and a module-only edit never deployed its parent script. `wmill sync push` still dropped an ADDED or DELETED `.lock` three branches before the module arm, so the earlier fix only covered a first push: adding a `uv.lock` to a deployed project was reported as a change forever and never applied, and deleting one left it deployed. Editing worked, which is what made the round trip look whole. Also removes a duplicate `#[test]` that was double-registering a test and detaching its neighbour's comment, and rewrites seven comments that still gave the cascade as the reason for behaviour that now serves lineage only. * fix(dbt): bound the excluded-file read, keep the retry budget job-wide `isBundledModuleFile` read a file in full before deciding it was too big or binary, so a project sitting next to a multi-gigabyte parquet seed loaded the whole thing only to reject it. It now takes the size from `stat` and reads at most the 8 KB the NUL check needs: a 191 MB file is rejected in 0.0ms at 82 MB RSS. Calling the retry helper after both phases gave each its own `attempts` budget, so a job could spend double what the descriptor asked for — the bound exists because every attempt is a real dbt invocation holding a worker slot. The budget is now the job's, spent across whichever phases fail, and the field says so. Extracting that helper had also placed it between `#[allow(clippy:: too_many_arguments)]` and `run_dbt`, taking the attribute off the 12-argument function it was written for. * fix(dbt): actually spend the retry budget `retry_failed_nodes` looped on `while *remaining > 0` and never decremented it, so a failing job reissued `dbt retry` — logging "attempt 1 of 3" each time — until the job's deadline instead of `attempts` times. The decrement existed briefly and was lost when the function was re-extracted by hand. Claiming and counting are now one operation, `claim_attempt`, because keeping them apart is exactly how the bound goes missing: the loop cannot iterate without spending the budget. Its test is bounded by its own `for` rather than by the function under test. An earlier version collected `std::iter::from_fn(|| claim_attempt(..))`, which against a non-spending `claim_attempt` is an infinite iterator — it allocated until the machine died. A test for a loop bound must fail an assertion when the bound regresses, not consume the host: it now reports `[1, 1, 1, …]` against `[1, 2, 3]` in 0.00s. * fix(dbt): ask before reading, not after Bounding `isBundledModuleFile` did nothing for the bundle builder, which read the whole file into memory and only then asked whether to keep it — so a multi-gigabyte seed beside a project was still loaded in full just to be skipped. The predicate is now consulted first, and the read happens only for files the bundle actually carries. * feat(dbt): animate the ref() edges feeding the model being built The nodes moved during a run but the edges did not, so the graph showed where dbt had got to without showing it flowing there. Reuses the canvas's existing rule rather than adding a second one: an edge animates when it touches what is happening. For a pipeline that is the running script; for dbt the unit of work is the model, so a `ref()` edge animates while its target builds. Same `animated` field, same visual language, no new styling. Verified mid-run on a 7-model project: of six `ref()` edges only the two feeding the model then building were animated, and none once the job finished. * feat(dbt): show what each model wrote, and say when its SQL is another project's Three things a reader wanted from the run graph and could not get. Row counts: the worker already records one per relation and `run_progress` already returned it, but the graph used only `status` and dropped the number. A model that built green having emitted zero rows is the failure that looks like a success, so the count is on the node. The relation's fully-qualified name, copyable: there is no table browser to open, so the next best affordance is the exact identifier to paste into a SQL client. It is parsed with `splitRelation`, which honours quoting the way the worker's `split_relation` does — splitting on every period renders `"wh"."analytics.v2"."orders"` as a relation `orders` in a schema `v2`, which does not exist. And when two projects materialize one relation, the graph keeps a single provenance winner, so the losing project's node carries the other's model. The SQL was already suppressed there — correctly, it is not this run's code — but silently, which reads as a dead click. It now says so. * fix(dbt): a finished run's graph is the models it built, not today's project `/assets/graph` is the current deploy, so an old run's graph drifted with the project: a model added after it appeared as though the run had built it, and the older the run the wronger the picture. A finished run's node set now comes from its own result, which named exactly what it touched. Sources survive the filter regardless — dbt never lists them in `run_results.json` because it does not build them, but they are the upstream the run read, and dropping them would leave the models hanging. The graph is still the current deploy's, so a model renamed or deleted since cannot be drawn at all. Rather than a silently shorter graph, the count is stated above it. Verified by adding a model after a run: the old run renders 7 models without it, a fresh run renders 8 with it. * feat(dbt): preview a model's rows with `dbt show` There was no way to see the data behind a node — only its SQL and its row count. `dbt show` selects from a model and returns rows, and every engine ships it, so the preview needs no adapter code of ours: no connection path, no dialect-correct quoting, no type coercion for ten warehouses. It runs against the profile the run already renders. It is a `dbt_command` rather than a new endpoint, so it inherits the whole job path — authorization, isolation, cancellation, logs, engine provisioning — and `limit` joins the run form beside it. That the allowlist can admit it at all is a consequence of dropping the cascade: while a successful job dispatched its deploy-time write set, a command that wrote nothing woke every consumer for relations nothing had touched. Read-only, and treated as such: no graph republish, no materialization records, no retry state, no test phase. Captured rather than streamed, like `dbt ls` — these rows are the result, not commentary, and the job-log writer is what `NO_LOGS_AT_ALL` discards. Verified: `{"dbt_command":"show","select":["stg_customers"],"limit":3}` returns three rows; a preview leaves `materialized_partition` untouched (62 → 62, 0 rows for the job); `clean` is still refused by the allowlist. * feat(dbt): preview a model's rows from the graph, and keep our locks out of dbt projects The run page could show a model's SQL and how many rows it wrote, but not the data. Selecting a model now offers "Preview rows", which runs the script with `dbt_command: show` and renders the result as a table. Explicit rather than on-select: a preview is a job, so it costs a worker slot and the engine's start-up, and previewing on every click would spend both on mere navigation. Sources are excluded — dbt shows what a model SELECTs, and a source is not one. Also: `updateModuleLocks` was the one module helper that never learned about verbatim bundles, so it walked a dbt project writing `foo.lock` beside `foo.sql`. None of those files is a Windmill script needing a lockfile, and the bundle promises to round-trip the project byte-for-byte — our artifacts have no business in it. Verified in the browser: selecting `stg_customers` and previewing returns the columns `id`/`src` and five rows from the warehouse. * fix(dbt): keep a run's models when another project owns their provenance Scoping a finished run's graph to the ids it named dropped relations whose provenance winner belongs to a different project — so a run of a project sharing a schema showed 3 of the 6 models it had built. An id that was never this run's package cannot be judged against its result, so it is kept: the relation IS one the run wrote, and hiding it understates the run. The same rule applies to the "no longer in the project" count, which otherwise reported deletions that were only provenance collisions. Previews are now cached per model and survive the selection moving. One was thrown away whenever the reader clicked elsewhere, which for a job costing a worker slot and an engine start-up meant re-running it to see it again — and the run continues in the background, so leaving and returning finds the rows there. The spinner also never span: `startIcon` takes the icon and its classes separately, so the animation has to be passed alongside. How long it took is shown with the rows. A preview is a job, and its cost should not be something the reader has to guess at. * fix(dbt): resolve argument references, clamp the show limit, flag renamed relations `handle_dbt_job` cloned `job.args` where every other executor calls `build_args_map`, so a `$var:` / `$res:` / `$encrypted:` argument reached dbt as the literal string. A placeholder holding a schema or an `enabled` flag would then build a different slice of the project than the caller asked for. `--limit` took any positive i64, and the worker buffers the whole of dbt's stdout to read the rows out of it — so a caller with only run permission could make it hold an unbounded allocation. It is clamped to a ceiling now, extracted as `show_limit` so the bound is pinned by a test rather than inline in an async function nothing can reach. And a model keeps its id when its alias or schema changes, so an old run's node showed today's relation while the run wrote another — the page asserting it had materialized a table that did not exist yet. The run's result carries the relation each node actually wrote, so the drift is detectable without a graph snapshot, and the count is stated above the graph. Rendering the run's own lineage still needs a per-job snapshot; this stops the page claiming otherwise. * fix(dbt): stop persisting resolved secrets, bound the preview by bytes Resolving `$var:` / `$res:` / `$encrypted:` for dbt — added in the previous commit — meant `save_run_state` wrote the resolved PLAINTEXT into `dbt_run_state.args` and the worker's `state.json`. The row outlives the job, so a secret stayed in the database and a later `dbt_command: retry` replayed it after the grant was revoked or the value rotated. The invocation now carries the args as submitted alongside the resolved ones, run state persists those, and the restore path resolves them again under whoever is retrying. Clamping `--limit` bounded the row COUNT, not the size: one column can hold a megabyte, so a thousand rows is a thousand megabytes, and `run_capturing` buffers all of it. The captured output has a byte ceiling now. `limit` became a built-in argument without joining `RESERVED_ARG_NAMES`, so a descriptor writing `{{ limit }}` was silently handed the preview control's default instead of being told the name is taken. Two display fixes: the relation-drift banner compared a canonicalized (lower case) asset path against the warehouse's own spelling, so it fired on every model of every finished Snowflake run; and caching a preview's failure left `Preview rows` dead for that model until reload. * feat(dbt): key the graph by script version so a run renders its own project The dbt graph was keyed by path alone, so a deploy overwrote the only copy and a run page could only ever show today's project — an older run rendered today's models, SQL and `ref()` lineage no matter what it had run. My previous attempt filtered that view to the ids the run named, which stopped it lying but could not show what was gone: the data no longer existed. `dbt_node` / `dbt_edge` now carry `script_hash` in their primary key, so each deployed version keeps its own graph, and the run page passes the version its job recorded. Per DEPLOY, not per run — ten thousand runs of one version share one graph — and a composite FK to `script (workspace_id, hash)` with ON DELETE CASCADE means a version's graph dies with it. Nothing pruned these before, because there was one copy per path; they would otherwise have accumulated with no sweep. Two deploys of one path now write disjoint rows, so the graph can no longer be lost to a race. `claim_graph_publication` remains only for what is still path-keyed — the `asset` usage rows — and an older deploy finishing late records its own graph before declining to touch those, where before it published nothing at all. A pinned request is scoped by the version's own nodes rather than by `asset`: that table describes the current deploy, so scoping through it would filter a model out of the very run that built it. Verified end to end: deployed v1 (8 models), ran it, deployed v2 with four models removed and one rewritten. The old run renders 8 models, 6 ref() edges and v1's SQL; a new run renders 4 and the v2 rewrite. * fix(dbt): scope graph cleanup to one version, bound the preview capture Archive and delete both act on a single `hash`, but the graph cleanup they called deleted every row for the path. Now that the graph is keyed per version, archiving an old version erased the live one's models, SQL and lineage, and nothing repaired it. Both callers have the path in hand, so the by-hash wrapper is gone and they use the version-scoped clear directly. `dbt show` checked its 8 MB ceiling after `wait_with_output` had already buffered everything, so the ceiling could not bound what the worker held. `run_capturing` now reads both pipes incrementally against a caller-supplied limit and kills the child on overflow. The read buffers are heap-allocated: as arrays they were baked into the future, which the job poller boxes several layers deep, and that overflowed the worker thread's stack — a `dbt show` run aborted the whole worker process. A retry's `dbt parse` ran on the arguments as submitted while the build ran on resolved ones, so a `$var:` shaping the graph parsed verbatim. The parse moves to the caller, after resolution. A run that names its own `select`/`exclude` now drops the descriptor's `selector`: dbt resolves `--selector` instead of `--select`, so passing both made a preview of one model return another's rows. Also: log instead of silently swallowing a `modules` column that fails to deserialize (pre-existing, but for dbt it means running with no project at all); keep the model SQL reachable once a preview has landed; render which node the rows came from; stringify object-valued cells; document `dbt_script_hash` in the OpenAPI spec. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * docs: record the per-worktree dev environment and the backend-run check Three mistakes this guidance would have prevented, each of which cost a cycle: A worktree has its own database and ports, but AGENTS.md stated the single-checkout defaults as facts. Pointing `DATABASE_URL` at another worktree's database makes `cargo sqlx prepare` fail on every query touching a table your migrations added — and it deletes `.sqlx/` before it fails, so the cache is gutted rather than merely stale. Starting a backend on the wrong port leaves the UI up with every call 502ing, which reads as an application bug. Both values are now discoverable with commands that work as written. `prepare` is also documented as the wrong tool for a removal-only change: the cache is already complete for CI, and the only residue is orphaned entries that can be found by text-matching against the sources without a database. Nothing told a reader that `cargo check` does not exercise a worker path. A read buffer declared as an array inside an async block is baked into the future, and once boxed by the job poller it overflows the worker thread's stack — compiling and unit-testing clean while aborting the whole worker process at runtime. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * fix(dbt): scope the remaining path-wide reads and clears to one version Four places still spoke for a whole path after the graph became per-version: The relation-root drift check read `dbt_node` by path with an unordered `LIMIT 1`, so with v1 at root A and v2 at root B it could answer with v1's row, suppress the refresh v2 needed, and leave v2's graph naming relations the run does not build. It now reads this job's version. `dbt_dep`'s no-resource branch cleared the path, so a descriptor edited to bring its own `profiles.yml` emptied every earlier version's graph and with it every finished run's page. The ownership being given up is the path-keyed `asset` usages cleared beside it; the graph clear is now this version's. The graph was inserted before the publication claim checked the version was still live. Archive and delete only soft-update `script`, so the foreign key still accepted an in-flight dependency job's rows and the failed claim committed them — and because pinned queries deliberately serve archived versions, deleted model SQL became readable again. The write is now gated on a `FOR UPDATE` liveness check. `clear_dbt_run_state_by_script_hash` resolved a hash to a path and deleted the path's saved run. `dbt_run_state` is keyed by path by design — one saved run per script — so archiving one version discarded the live version's resumable failure. It clears only once no live version of the path is left; `identity` already refuses a resume whose project, warehouse or engine moved. "Preview rows" ran `runScriptByPath` while the SQL beside it was pinned to a hash, so an old run showed its own SQL over today's rows. Verified end to end: with v3 deployed, the v2 run's preview runs v2's hash and returns v2's rows. Also: keep the TAIL of a captured stderr, since dbt prints its summary last; one `$derived` for the parsed result rather than five; collapse three near-identical argument accessors onto one generic; fold the single-use `copy_dir_command` into its caller; and give `parseDbtRun.ts` one status classifier instead of spelling dbt's failure vocabulary twice. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * refactor(dbt): one JobCtx down the executor, one table per adapter Two changes aimed at the operations this code will keep having: adding a phase, and adding a warehouse. `JobCtx` already bundled the five values every phase needs, and nine functions took it — but the top of the executor threaded the fields apart and rebuilt the struct at each call, so the same literal appeared eight times and each new phase meant five more parameters. It is now built once per entry point and reborrowed. `prepare_project` goes from 21 parameters to 17, `retry_failed_nodes` from 15 to 11, and `run_dbt` drops below the lint threshold. The two remaining constructions are the worker boundary, where the pieces genuinely arrive apart. `DbtAdapter` answered five questions with five parallel matches over the same eleven variants, plus a sixth list of adapters kept by hand in a test. The facts now live in one `AdapterSpec` per adapter, reached through one exhaustive match, so adding a warehouse states its name, driver, package, port, database key and licensing together and the compiler demands the arm. Each arm spreads from a Postgres base, which makes the inheritance visible per adapter instead of hidden in the `_ =>` defaults `default_port` and `database_key` used to carry. `DbtAdapter::ALL` replaces the list the test kept separately. Verified by dumping all seven facts for all eleven adapters before and after: byte-identical. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * fix(dbt): job-keyed run progress, and stop path-wide reads and clears Six findings from the last round, in the order they bite. The publication liveness gate refused on `archived`, but `create_script` archives the parent on every redeploy — so deploying v2 while v1's dependency job was still parsing left v1 without a graph, permanently, which is the exact case the unconditional write existed to serve. It gates on `deleted` alone now; an explicit archive is still covered by the `FOR UPDATE` ordering. A project-owned `profiles.yml` trusted `profile.type` instead of reading the file. The Rust engines carry every adapter, so a CE script could declare `postgres` over a target that is `sqlserver` and have dbt connect with the enterprise adapter. The file is read whichever way, and a descriptor that disagrees with it is refused. Renaming a dbt script, or editing one so its newest version is no longer dbt, cleared the graph for the whole path — every older version's models, SQL and lineage, which their own finished runs still render. Neither needs it: graph queries join on `(path, hash)` through a `language = 'dbt'` CTE, so an old version's rows cannot attach to whatever lives at that path next. Live progress read `materialized_partition`, whose key is the relation and whose `job_id` is only the last writer. Two runs of one project took rows from each other. Progress now has its own job-keyed table; the relation table is untouched, because one row per relation is right for the pipeline canvas and fork defer. Verified with two overlapping builds: both keep 6 rows in the new table, while the old one attributes 6 to one run and 0 to the other. `Scratch::drop` removed a half-installed virtualenv synchronously from inside the job future, blocking a runtime thread; it goes to `spawn_blocking`, with a direct call when there is no runtime to hand it to. The E2E list asked for a `# on table://` subscription the deploy now refuses. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * feat(dbt): snapshot a dynamic descriptor's graph per run A `{{ }}` placeholder in `vars` can enable a different set of models per run, so those runs re-ingest the graph. Keyed by version alone, each re-ingest overwrote the last: reopening an older run showed the newer run's project, and a model only the older run built was gone entirely — no SQL, no lineage, and nothing the saved result could colour, since it can only tint nodes that are there. `dbt_node` / `dbt_edge` gain `job_id`. A run of a dynamic descriptor writes its own snapshot under its job id and its page reads it back; a static descriptor writes the version's graph once, under a zero-UUID sentinel, and every run of it reads that. The sentinel is a value rather than NULL because `job_id` is part of the primary key and Postgres does not treat two NULLs as the same key, so each re-ingest would add a row set instead of replacing one. `/assets/graph` takes `dbt_job_id` and prefers a snapshot when one exists, falling back to the version's graph otherwise — so a run page passes it unconditionally and static descriptors are unaffected. Snapshots age out after 30 days, pruned by the runs that write them, so no background sweep has to learn about these tables. Verified end to end: one deploy, two runs of it with `extra=yes` and `extra=no` gating a model's `enabled`. The version's graph holds 6 models, run 1's snapshot 7 including `opt_extra`, run 2's 6 without it; the endpoint returns each run's own and falls back to the version's when the parameter is omitted. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * perf(dbt): only snapshot a run whose graph differs, and prune from every run Two costs the per-run snapshot carried, both found by measuring it rather than by reading it. A snapshot was written for every run of a dynamic descriptor, but marking one dynamic is conservative: `graph_is_per_run` is true whenever `vars` holds a `{{ }}` placeholder or `env` holds a `$var:`, which says the arguments reach dbt and not that they change which models exist. The usual case is a date var, whose graph is identical run after run, so the table filled with copies of an unchanging picture — around 1 KB per model per run, which is a gigabyte or so a month for a 200-model project on an hourly schedule. A row set now carries a digest of its nodes, edges and relation root, and a run whose digest matches the version's writes nothing; the read already falls back to the version's graph, so those pages are unchanged. Only a run whose model set really differs pays. The prune was hung off the progress reporter, which exists only for engines that emit node events — so a Fusion or dbt-core-2x instance accumulated snapshots and never deleted any. Retention that stops working because of an engine choice is not retention; it runs detached from every dbt run instead. Verified against a descriptor with a var-gated model: the version's graph holds 8 rows, a run that resolves to that same graph stores none at all, and a run that enables the extra model stores its own 9. Both pages still render their own project — 6 assets without the extra model, 7 with it. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * fix(dbt): scope every dbt_node join to the chosen snapshot `job_id` joined the key, but only the scoping CTE and `dbt_edge` were taught to filter on it. The outer node SELECT and the parent/child joins in the edge query were not, so each model came back once per retained snapshot plus once for the version's graph, and each edge matched every combination of the two — the model count multiplied and the edge join fanned out quadratically. Measured against one stored snapshot: 17 node rows where 8 are wanted, and 28 edge pairs where 7 are. The response dedup hid the edge blow-up from the payload, not from the plan, and the run page refetches the graph every two seconds. The progress table gained writers it was missing. `terminalize_running_relations` settled only the relation-keyed table, so a cancelled or killed run — the case that function exists for, since it leaves no `run_results.json` — showed every in-flight model still spinning on the run page for as long as the row lived. An agent worker cannot write the new table at all, having no database of its own, so the read falls back to the relation-keyed one when a job has no rows there. Also: a wrapped string literal missing its backslash put eighteen spaces in the middle of the profile-disagreement error; a comment still described concurrent runs of one dynamic version overwriting each other's graph, which is what keying by job removed; and the `materialized_partition` index justified itself by a run-page poll that has since moved to another table, though the closing sweep still earns it. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * fix(dbt): give a graph snapshot a marker row, and scope what reads it Five findings, four of which are the same mistake in different places: a snapshot's identity was inferred from its contents. Existence was inferred from a `dbt_node` row, so a dynamic run that disabled every model — a legitimately empty graph — read as "no snapshot" and its page showed the deployed models instead. The digest was a column repeated on every node and read back with a `LIMIT 1` carrying no `job_id`, so a run could compare itself against another run's digest and suppress a snapshot it needed. The relation-root drift check read the same rows unscoped, so after a drift it could find a previous run's root and conclude nothing had moved. `dbt_graph_snapshot` holds one row per stored graph — path, version, job, digest, timestamp. Existence is that row, the digest lives there once, the drift check reads the deployed row explicitly, and the retention sweep deletes markers first and then the rows no marker stands for. The digest is SHA-256 rather than `DefaultHasher`, whose output is documented as unstable across Rust releases: this value outlives the process that computed it, so a toolchain bump would have silently stopped every comparison matching and quietly reinstated the duplicate snapshots the digest exists to prevent. `/run_progress` ignored the view token, so a share-link viewer got the graph and was refused the progress that colours it. A preview sent only its own three arguments, so a descriptor with a required `{{ }}` var could not be previewed at all and an overridden one previewed a different relation than the page was showing. The run's arguments go first now, with the preview's three overriding. Verified on a project whose only model is var-gated: the deploy stores a marker with zero nodes, a run with the var set stores a marker with one, and the endpoint answers 0 and 1 respectively rather than showing the deployed models for both. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * refactor(dbt): squash the runtime's migrations into one Ten migrations reshaping the same three tables is a history no installation ever had. `dbt_node` gained `script_hash`, then `job_id`, then `ingested_at`, with its primary key rebuilt twice; `graph_digest` was added by one migration and dropped by the next after the digest moved to its own table. On a fresh database all of that replays to arrive at a shape the schema can simply state, and this feature has never shipped, so there is no upgrade path to preserve. One migration now creates `dbt_node`, `dbt_edge`, `dbt_graph_snapshot`, `dbt_run_state` and `dbt_run_progress` in their final shape, carrying forward the rationale each of the replaced migrations recorded. The enum additions stay in `add_dbt_lang`, since a value cannot be added and used in one transaction, and the `materialized_partition` index stays separate because it belongs to a table this feature did not introduce. Verified by rebuilding: dropped the five tables, replayed from the single migration, and confirmed the result is identical — same primary keys, the same two composite `script` foreign keys, the same seven indexes. Every `sqlx::query!` in the workspace then compiled against it, which checks each column's name, type and nullability, and a deploy plus run on the rebuilt schema produced 8 nodes, 7 edges, a snapshot marker and 6 progress rows. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * fix(dbt): authorize snapshot reads, bound the prune, give the marker a lifecycle `dbt_job_id` is caller-supplied and selected straight from `dbt_graph_snapshot`, which carries no RLS — so a caller who could see the script could read any run's model set and relation paths, which a dynamic alias or schema can encode. Both graph queries now require the job itself to be visible, in the authed transaction, the same gate `raw_code` already applies to the script that produced it. The drift check compared against the deployed graph alone, which misses the way back: a run at root B republishes the path-keyed `asset` usages at B, and returning the profile to A then matches the deploy and skips the refresh, leaving those usages at B while dbt builds A. It reads the most recent ingest for the version instead — the one that last wrote them — ordered rather than an arbitrary `LIMIT 1`. The prune anti-joined every non-deployed node and edge with no age predicate, so each run scanned the whole retained sidecar and concurrent runs duplicated it. All three deletes share one age bound again, with the sentinel spelled as a literal so the partial indexes apply — a bound parameter cannot be proven to match the index predicate. `dbt_graph_snapshot` was the one dbt table nothing in the script lifecycle deleted: no `script` foreign key and absent from both `clear_dbt_manifest*` sites. A marker outliving its rows is read as a snapshot with no nodes, and its digest still answers the suppression check, so an identical run would write nothing and then render an empty graph. It cascades like the rows now and both clears take it. Also: the preview cleared `exclude` rather than inheriting it, since previewing a model the run excluded reached dbt as `--select m --exclude m`; and `terminalize_running_relations` no longer claims to cover a killed worker, which never reaches it. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * fix(dbt): quote profile names, record where usages were published, stop polling the graph A profile name comes from the project's own `dbt_project.yml` and a target from the descriptor, and both were interpolated into `profiles.yml` as bare YAML — including as mapping keys. A name like `prod # hidden` truncates the mapping and a newline opens a sibling key of the author's choosing. Both are rendered as quoted scalars now, as are the BigQuery keyfile's keys, with a test that asserts the document still parses to exactly the keys we wrote. The drift check read the most recent ingest, which latches: a run that returns to the deployed root re-ingests but stores no snapshot (its digest matches the version's), so the moved run's rows stay newest and every later run pays an extra parse and ingest. The publisher now records the root it published the path-keyed usages at, which is the only thing that answers "where do the current usages point" — the deploy's own root goes stale as soon as a run republishes. The run page polled `/assets/graph` every two seconds alongside progress, so it re-sent every node's SQL for the length of a run — hundreds of KB a tick on a real project, for a graph that a dynamic descriptor re-ingests exactly once before the build. It fetches once more shortly after mount and then polls progress alone. Node results carry `outcome` beside `status`. `status` stays dbt's own word, but dbt owns that vocabulary — 1.x and 2.x differ on casing and `no-op` arrived in a minor release — so publishing only it would force a break or a lie the first time it moves. `outcome` is the stable half a downstream script branches on. Also: the worker's dbt entry points are `pub(crate)`, since nothing outside the crate calls them and they resolve secrets and launch processes; and the snapshot gate records that it is RLS-only where `/jobs/run_progress` also honours a share-link token, which is a gap in what a shared page shows rather than in what it protects. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * test(dbt): pin the graph storage invariants against a real database Every defect review found in this area was DB-shaped — which row set a read resolves to, which rows a clear takes, whether a snapshot exists at all — and none of it is reachable from a unit test on a pure function. Four rounds established these answers and nothing guarded them, which is why each round kept finding another. Six cases, on the harness the repo already uses for schema-shaped behaviour: an identical run stores no snapshot and leaves no marker; a differing run keeps its own while the version's is untouched; an empty run graph is still a snapshot rather than an absent one; clearing one version leaves the others whole; the path-wide clear takes the markers with it; and the sweep ages out run snapshots while never touching a version's own graph. `IngestedNode` gains `Default` so a test can state the two fields a case is about rather than the eighteen it is not. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * perf(dbt): bound a script's stored graphs by deploy count Run snapshots expire on a clock, but a VERSION's graph could not: its reader is every finished run of that version, and a run page is as old as its job. So nothing reclaimed them — a deploy graph went only when its `script` row was hard deleted, which Windmill does not routinely do. A CI deploying on every commit added a full model set with SQL bodies per commit, forever: roughly 200 KB a deploy for a 200-model project, which is gigabytes a year across an instance. Bounded by COUNT instead of age, since age is the thing that cannot be right here. The newest 50 deploys per path keep their graph and older ones are reclaimed, making growth `versions x models` rather than unbounded in time. Generous on purpose: reaching the bound empties that version's run pages, so it exists to stop unbounded growth rather than to be hit in normal use. Ordered by the script's own `created_at`, so a late-finishing job re-ingesting an old version cannot promote it. Pinned by a test that deploys past the bound and asserts both halves: the count holds, and the newest version is always among the survivors. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * fix(dbt): let the run page know when its snapshot has landed The one-shot graph refetch was wrong: a dynamic descriptor's ingest happens before the build but after cloning, dependency install and parse, so a fixed delay either fires too early — and the run page then shows the deployed models for the whole run, never that run's own — or keeps re-sending the whole graph for the length of it. Neither is a timing problem to tune; the page had no way to tell "the snapshot is not written yet" from "this run has none". `/assets/graph` answers that directly: `dbt_snapshot_job` is the job the dbt half resolved from, when one was asked for and found. The page polls the graph until that is its own job, and stops. A static descriptor never snapshots, so an attempt cap ends it there rather than polling for the run's duration. `dbt_node.relation_root` is gone. The drift check moved to the marker's `published_relation_root`, which left the column written on every node and read by nothing. `outcome` was published as the stable half of the result contract, but the in-tree consumer still ranked and coloured from dbt's own word — so the field existed and nothing used it. `statusRank` takes it, `DbtRunResult` passes it, and `classifyStatus` is documented as the fallback for results that predate it and for the live event stream, which carries dbt's word alone. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * docs(dbt): record what a share-link viewer actually sees The comment at the snapshot gate said the graph "falls back to the deployed set", which is only the rarer half of it. A share link is an extra grant for a logged-in user who lacks access to the job, so the usual case is no read on the script either — and then the `live` CTE matches nothing and the whole dbt half comes back empty. A blank Models panel over working progress rows, not a fallback. `docs/dbt-runtime.md` now carries the analysis a follow-up needs: that relaxing this leaks nothing, because `v2_job_completed.result` already gives that viewer every node's `unique_id` and `relation_name` — the graph's only incremental exposure is `raw_code`, which is gated separately on seeing the script. And the shape of the fix: `OptViewToken` and `validate_view_token` are self-contained enough to move into `windmill-api-auth`, which `windmill-api-assets` already depends on, after which the gate can honour a token for that job's snapshot alone while `raw_code` stays where it is. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * fix(dbt): keep model SQL behind the scripts:read scope, and unbreak CI `/assets/graph` is authorized as `assets:read`, and RLS decides whether the caller can see the script that produced a node — but RLS is not a scoped token's grants. A token deliberately narrowed to `assets:read` could therefore read model source and repository paths for scripts outside its `scripts:read` paths. The same `build_scope_path_predicate` the macro endpoint already applies now gates `raw_code` and `original_file_path`; the relation's shape is unaffected, only its body is withheld. `DbtAdapter::ALL` exists for the tests that must cover every adapter, so it is dead in a release build and `-D warnings` failed all four backend checks on it. It is `#[cfg(test)]` now. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * fix(dbt): stop the graph poll at the ingest, and type `limit` in the schema The poll's stop condition was a snapshot appearing, with a 40-attempt cap behind it — so a STATIC descriptor, which never snapshots, took the cap every time and re-fetched the whole graph forty times. That is most of what removing the poll was meant to save, and static is the common case. The ingest runs BEFORE the build, so the first model to report progress proves it has already happened: a snapshot absent by then is one this run never writes. Progress arriving is now the second exit, and the cap is only a backstop for a run that reports none at all. `limit` is declared `Typ::Int` but `dbt_arg_schema` had no integer arm, so the run form and the generated clients saw an untyped default and offered no numeric control for a value the worker clamps. Covered by the schema test. `relationOutcome` still re-derived from dbt's word while `statusRank` had moved to `outcome`; both read it now. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * fix(dbt): make a retry prove its arguments still resolve the same The saved arguments are the ones SUBMITTED, so a `$var:` in them is re-resolved on retry. The identity did not cover the resolved values, so a variable that changed between the failed run and the retry was accepted — and which graph the retry then used depended on WHERE it landed: a worker holding the local snapshot replays the saved manifest, while a database restore reparses with the new value. Placement decided whether the resumed failures described the relations being built. The identity gains a digest of the resolved arguments, and is compared in two halves because resolution happens between them. Project, warehouse, engine and env are checkable up front; the arguments are not, because a retry request carries only `dbt_command` and the ones to compare are the SAVED arguments after this caller has re-resolved them. Comparing the whole string up front would have refused every retry — which is what the obvious version of this fix does. A row written before the digest existed has no last segment, and still restores rather than being refused. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * fix(dbt): keep pre-upgrade retries working, and stop losing a late snapshot Splitting the identity on its last `|` read a pre-upgrade row's env digest as an arguments digest and left only `<run_identity>` as the prefix, so every saved failure on an upgraded instance became unretryable — a regression the previous commit's own test missed by using an identity with no `|` in it at all, which is not what an old one looks like. The digest is tagged (`|args=`) rather than positional, and the test now uses a real pre-upgrade identity. The graph poll gave up after a bounded number of tries, but provisioning and `dbt deps` precede the ingest and can outlast that on a cold worker — and the engines that emit no node events never produce the progress that ends it early. A finished run now reloads the graph unconditionally, and the poll's own exit issues one last load: progress proves the ingest happened, not that the previous tick saw it, and dbt's compile window is wider than one tick. A `dbt retry` restores the failed run's arguments inside the worker and they are never written back to the retry job, whose own args are just `{"dbt_command": "retry"}` — so previewing a row on a retry's page ran without the vars the run used. The result now carries the invocation's arguments as SUBMITTED, so a `$var:` stays a reference and no resolved value is published. The deploy-count sweep ran instance-wide on every dbt run: `FROM script WHERE language = 'dbt'` has no index to stand on, and both orphan deletes are the complement of every partial index here. It is scoped to the running script's `(workspace_id, path)` — which `index_script_on_path_created_at` serves — and the orphan deletes only run when a marker actually went. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * fix(dbt): hide dbt from module-less pickers, and stabilise the retry digests `processLangs` feeds every language picker, including flow steps and app inline scripts. Those are raw bodies with nowhere to carry a module bundle, and a dbt script IS its bundle — so choosing dbt there produced a job that could only fail once the worker looked for `dbt_project.yml`. Those two surfaces use `processInlineLangs`, which drops the languages that need modules; a flow still reaches dbt the way it reaches any script, by path to a deployed one. `graph_digest` moved to SHA-256 because it is persisted and compared by a later worker, and `DefaultHasher` is documented as unstable across Rust releases — but the retry identity's own digests were left on it, and they are persisted in `dbt_run_state.identity` for exactly the same comparison. A toolchain bump would have refused every saved failure as a different project. All three go through one `stable_digest`, length-prefixed so no split of the same bytes collides. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * fix(dbt): enforce the tag scope on snapshot reads, reset state between runs A tag scope is an orthogonal hard restriction: a token limited to some tags must not read a job outside them however else it is authorized. The snapshot lookup went through `v2_job` RLS alone, which knows nothing about tags, so such a token could still retrieve a run's model set and its dynamic relation paths. The same predicate `require_job_read_access` applies for the progress half of the page is applied here — `get_scope_tags` is already public in `windmill-api-auth`, and it is `None` for an unscoped caller, so a normal session pays nothing. SvelteKit reuses the run graph between run ids, and `graphTries`, `polled` and `raw` all describe the previous job: a spent retry count stopped the next run's snapshot poll before it began, and stale progress coloured its models with another run's statuses. All three reset when the graph key changes. Also a wrapped string literal missing its backslashes, which put two ~22-space runs in the middle of the retry-refusal message. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * docs(dbt): put each digest helper's rationale on its own function Inserting `stable_digest` above `split_identity` split that function's doc, so five lines describing where the identity divides ended up introducing the hasher. Each is back on the function it describes, stated once. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * fix(dbt): read the run-pinned graph through the job, not the asset graph Pinning the asset graph to one run is job-scoped data, but `dbt_job_id` sat on `/assets/graph`, authorized as `assets:read`. The job-read contract — `require_job_read_access` — is five parts that pull in opposite directions (tag scope restrictive, `created_by` permissive, app-embed restrictive-overriding, view token permissive, RLS underneath), so plain RLS is neither a stricter nor a looser approximation of it. Restating the parts near the graph query kept leaving one out: first the job check entirely, then the share-link asymmetry, then the tag scope, and the app-embed restriction was still missing and failing open. The helper cannot be called from `windmill-api-assets`, because `windmill-api` depends on that crate. So move the read instead of the check: the run-pinned graph is now `GET /w/{w_id}/jobs/dbt_graph/{id}` in `windmill-api`, on the same gate as the `run_progress` it colours, and `/assets/graph` has no `dbt_job_id` parameter at all. - `asset_graph_for` takes the job as an argument from an already-authorized caller; the route handler passes `None`. - Extract the graph response into a `AssetGraph` component schema, now that two paths return it. - The run page fetches the job route when it has a job id. * fix(dbt): charge assets:read on the run-graph route, trust the job gate in SQL Round 13 findings on the route moved last commit. The scope domain comes from the URL segment, so putting the read under `/jobs` asked a scoped token for `jobs:read` alone while returning asset-graph data that `/assets/graph` charges `assets:read` for. A token narrowed to polling run status could read workspace topology, and the missing-job fallback made it cheaper still — any random UUID skipped the job gate. Both scopes are now required: the job gate reaches this run, `assets:read` reaches asset data at all. The `chosen` CTE re-decided job visibility under plain RLS after the caller had already passed `require_job_read_access`. It could only disagree, and did so silently by falling back to the deployed graph — a share-link viewer entitled to the run was shown a different run's model set. Dropped; the contract is that a job reaching `asset_graph_for` is already authorized. Also: the flow editor's `+` insert menu still offered dbt (the third `processLangs` caller, missed when the other two moved to `processInlineLangs`), the docs still described the deleted `dbt_job_id` parameter, and the new handler had again been inserted between `get_run_progress`'s doc comment and its function. * fix(dbt): resolve a pinned run's version from the job row, not script RLS Local codex review of the branch. A share-link viewer is entitled to the run and usually has no grant on the project — that is what the link works around. The graph's `live` CTE resolved the version by selecting `script` inside the viewer's RLS transaction, so it answered for their access to the project rather than for the run they were given: the Models panel came back blank beneath working progress rows. A pinned run now takes its path and hash from the job row the handler already read after authorizing the job, so `live` does not consult `script` at all. `raw_code` keeps its own `EXISTS` against `script`, so the model bodies stay behind access to the project. Verified under RLS as an unprivileged role: the shape query goes 0 rows -> 1, the `raw_code` gate stays 0. Taking the version from the job also means a caller can no longer pin one project's version while naming another's run, since `dbt_script_hash` is ignored when a job is given. The run page's graph fetch is a raw `fetch`, which bypasses the interceptor that adds `X-View-Token` to generated-client calls, so a shared page was refused before any of this mattered; it goes through `appendViewToken` now. Also trims three comments to the AGENTS.md limit, dropping drafting-history rationale that belongs in docs/dbt-runtime.md. * fix(dbt): snapshot vars-overridden runs, carry the pinned version everywhere Second local codex pass. A `vars` run argument overrides the descriptor's, and vars drive `enabled`, alias, schema, database and materialization — so such a run builds relations the deployed graph does not describe. It now snapshots under its own job id, which per-job keying makes safe: the version's graph stays for runs that did not override. The old comment claimed gating on it would strand the override's graph for the next default run, which was true only when the write went to the deployed slot. Two sites still read the caller's `dbt_script_hash` instead of the version resolved from the job, so `/jobs/dbt_graph/{id}` without that redundant parameter dropped models the run's version had and a later deploy removed. The `dbt_snapshot_job` marker re-checked `v2_job` under RLS — the recheck the graph query itself drops. A share-link viewer got the right graph and a null marker, so the run page refetched it 40 times before giving up. `wmill script preview` read the bundle with the generic `__mod` suffix and script-module parsing, so previewing a `.dbt.yaml` omitted the project and failed on the missing `dbt_project.yml`. It uses the same suffix and verbatim read as deploy. * fix(dbt): key retry state by principal, not by script path alone `dbt_run_state` held one row per (workspace, script path), and a retry replaces the caller's arguments with the saved ones. Anyone able to run the script could therefore retry whoever ran it last, replaying that run's literal `select` and `vars` against the warehouse and publishing them as their own job's `invocation_args`. Running the script was already theirs to do; seeing another principal's arguments was not. `permissioned_as` joins the key, so a retry resumes only state written under the same authority. Two runs sharing an authority can already act for each other, so this is the boundary that matches the rest of the job model. * test(dbt): pin what a caller without access to the project sees of its run The share-link case had no regression guard, and every fix in this area touched one of its two halves: the graph's SHAPE has to survive a caller who cannot read the script, and the model SQL must not. Two cases against a real database, calling `asset_graph_for` as a member with no grant on the project's folder: pinned to a run, the models render and `raw_code` is withheld; unpinned, the same caller sees nothing of it, so making the first work did not relax the second. Both assertions were checked by mutation — reverting the `live` bypass empties the graph, and dropping the `raw_code` script gate leaks `select 1` — so neither passes on the code it is meant to catch. * fix(dbt): key the worker-local retry cache by principal too Keying `dbt_run_state` by `permissioned_as` left its worker-local twin keyed by workspace and script path alone, so the boundary held only where the database row was consulted. An agent worker never reads that table — `Connection::Http` leaves `latest_job` as `None` — so there the local cache was the whole boundary and it had none: the next principal to retry the script on that worker restored the previous one's `select` and `vars`. Also records the sqlx `--all-targets` trap in the update-sqlx skill: it is needed for queries inside tests, and in a CE checkout it aborts on `tests/otel.rs` (EE-only `otel_ee`) after having already emptied the cache. * fix(dbt): log a dropped retry-state save, correct the run-progress contract Saving retry state is best-effort — losing it costs a retry, not the run that just finished — but `.ok()` dropped the reason too. The only symptom was `dbt retry` reporting nothing to resume, which reads as a bug in retry rather than a failed write. Found by running a real failing build against a worker whose binary predated the `permissioned_as` column: the insert violated NOT NULL and said nothing. The run-progress endpoint's OpenAPI description promised an empty list for a caller who cannot see the job. It is refused instead; an empty list means the job recorded nothing yet or is unknown here. * fix(dbt): return the retry-state write failure the warning was added to report `save_run_state` discarded the insert result, so the caller's warning could never fire and a lost retry row stayed silent — the symptom being `dbt retry` finding nothing on another worker. The error is held rather than returned at once: the worker-local copy is what an agent worker resumes from, so a failed insert must not cost that too. Every exit after it surfaces it, including the ones that give up on the local save. * fix(dbt): decide a retry's graph from its restored args, keep local state in step Three from the seventh local review. A retry submits only `dbt_command`, so the vars-override check ran against an empty argument set and left `graph_is_per_run` false. The failed run's arguments are restored afterwards, and those are what the retry builds with — an overridden one wrote no snapshot for its own job and its page fell back to the deployed graph, showing the wrong enabled models, aliases and schemas. The decision is re-asked once the restore has happened. A failed durable write no longer publishes the worker-local generation either. `restore` accepts a local generation only when the database row names it, so publishing one the database never recorded made this worker reject its own newest state and resume the previous run's — its selection and vars, or "nothing to retry" if that one had succeeded. An agent worker attempts no durable write, so it keeps its local copy as before. A rename that also converts away from dbt moved the old path's retry state onto the new one, reinstating what the conversion had just cleared and leaving one user's arguments and results under a path no dbt script occupies. It moves only while the destination stays dbt, and clears the source otherwise. * fix(dbt): drop retry state when a run produced none, let module pushes fail loudly Three from the eighth local review. A run that never wrote `run_results.json` — cancelled, timed out, or dead before dbt got there — left the PREVIOUS run's state authoritative in both the database and the local pointer, so a later `dbt retry` resumed that older invocation's failed nodes. Producing nothing resumable now clears both copies, so neither can answer for the other. `wmill sync push` wrapped the descriptor lookup and its deployment in one try/catch meant for a missing parent. Any API failure or invalid descriptor was reported as "no parent found" and swallowed, so a module-only push exited zero with the remote project unchanged. Only the lookup is tolerated now. Also condenses a comment that narrated how earlier status comparisons behaved. * fix(dbt): forget retry state on pre-build exits too, drop cascade claims A dynamic run whose pre-build `dbt parse` or graph ingest fails returns before the save that clears stale state, so the previous run stayed authoritative in both the database and the local pointer and `dbt retry` resumed ITS failed nodes — writing relations the run that just failed never touched. Both exits now invalidate, through one helper shared with the no-artifact case. Two frontend comments described dbt producer rows as driving cascade dispatch. The executor returns before dispatch for every dbt job and deployment rejects `table://` subscriptions, so they promised behaviour that cannot occur; they describe the lineage and ownership that is actually retained. * fix(dbt): let the database decide retry state where it is reachable A SQL worker treated "no `dbt_run_state` row" as no opinion and accepted any worker-local generation. But no row is the authoritative answer that the last invocation left nothing resumable, so a local pointer that outlived it — an unlink that failed, a process killed between the delete and the removal, a stale cache — resurrected a replaced run and let `dbt retry` write relations it never touched. An agent worker keeps accepting its local copy: it has no authority to consult. Invalidation failures are logged rather than dropped, since a silent one is exactly what leaves the pointer behind. * docs(dbt): record how to run an agent worker locally, keep archived graphs Every step of standing one up fails as something else: a normal build cannot start one at all, the server's routes need a separate feature, and all three token mistakes surface as a bare 401 on the agent with the reason only in the server log. Written down with the error each produces. Also keeps a dbt script's graph when it is ARCHIVED rather than deleted. The pinned read resolves versions through a CTE that already skips archived rows, so clearing bought nothing and emptied the Models panel of every completed run of the project. Deletion still clears it. * feat(dbt): let an agent worker publish its graph, through one endpoint An agent worker was refused any dbt script whose profile comes from a Windmill resource — the common case — because it could neither read the stored relation root to check for drift nor re-ingest a corrected one. Those look like two needs but collapse into one: verification exists only to decide whether the stored graph still describes reality, so a worker that can PUBLISH never has to ask. It stores what it just parsed. `POST /api/agent_workers/dbt_graph/{workspace_id}` is the whole addition. It wraps the same `replace_dbt_manifest` the SQL path calls, so digest suppression, the marker write and retention cannot drift between the two transports, and it refuses a job the token's tags do not cover. `IngestedManifest`/`IngestedNode` gain Deserialize to cross the wire. Two guards go, both now false: the pre-build refusal, and the `Connection::Sql` gate added earlier to stop a `vars` override grounding an agent run. Live progress stays SQL-only — that is a per-model event stream, and routing it through the API would mean a round trip per node. * docs(dbt): warn that a differing cargo feature set swaps the shared binary * docs(dbt): record the verified agent-worker behaviour and the tmpfs quota trap An agent worker now runs a dbt job end to end, retries, and publishes its graph — confirmed with a dynamic descriptor whose per-run snapshot came back through the new endpoint. The doc said it was refused; that was true before the endpoint existed. Also `WINDMILL_DIR`: on a dev box the job dies with `Disk quota exceeded (os error 122)` writing the project's files while `df` shows free space AND free inodes, because /tmp is a tmpfs carrying a per-USER quota. Point the worker at a real disk rather than trying to clean up beneath it. * chore(dbt): pin the EE revision carrying the agent graph endpoint * fix(dbt): bind the published graph to the job, break the completed-page poll loop Five from the thirteenth local review. The EE endpoint took `script_path` and `script_hash` from the payload and checked only that the supplied job carried one of the agent's tags, so an agent holding any matching-tag job could name another script and replace its graph. Both are read from the verified queue row now and the request carries only the job id. A raw preview has no version, so it no-ops rather than 422ing before dbt runs. `IngestedManifest`/`IngestedNode` take `#[serde(default)]`: they were serialize-only, and a field the serializer skips made the whole manifest unparseable on the receiving side. A completed run page fetched the graph forever — `load()` assigns `raw`, which recomputes `settled`, which re-entered the same effect. The final fetch is keyed to the job by a plain (non-reactive) variable, and `settled` is read untracked. The pin now names a revision that compiles: the previous one still called `authed.tags()`, a method that does not exist, because both that fix and the JSON response landed after it was committed. * fix(dbt): keep a run snapshot out of the script's deployed ownership Everything `persist_ingest` writes after the manifest is keyed by PATH — one row set per script, describing what is deployed there. A run snapshot was still reaching it, so a one-off `vars` override republished that invocation's relations as the script's ownership and the workspace graph stayed on the override's schemas and aliases: an ordinary run of a static descriptor never ingests again to correct it, so only a redeploy would. A snapshot now stops after recording its own rows. The row preview also selected a bare model name, which dbt resolves across every installed package while `show` takes a single node — a project model sharing its name with a package's was previewed wrongly or refused. It selects the package-qualified FQN. * fix(dbt): forget stale retry state when preparation itself fails `prepare_project` runs before every path that could clear it, and it fails for reasons unrelated to the saved run — a profile that stopped resolving, a provision cancelled, packages that will not install. The invocation still left nothing resumable, so the previous one must not stay authoritative: a repaired project would otherwise let `dbt retry` rebuild an older run's selection and write relations the latest invocation never reached. A retry is exempt, since it is trying to use that state and failing to prepare says nothing about it. Also corrects the runtime doc, which still described agent workers as unable to run dynamic descriptors or Windmill-resolved profiles. They publish their graph through the API now; what they do not get is live progress and a durable retry row, and the doc says so. * fix(dbt): spell the whole FQN for preview, bound retained retry generations The FQN selector added last commit was `<package>.<name>`, but a dbt FQN is the resource's path within its package and the matcher must consume the selector and end on equal lengths — so it matched nothing for a model under `models/marts/`, which is the layout most projects use and the one this repo's own complex fixture has. The middle segments come from `original_file_path`, whose first element is the resource root the FQN excludes. Without a path it falls back to the bare name: ambiguous across packages, but a selector dbt resolves rather than rejects. Tested on a nested model, which is the input that separates the three spellings. Superseded retry generations were removed only when a later run published one, and never inside the hour-long grace period — so a burst left a manifest and a results copy per run with nothing afterwards to collect them. At most four now sit in the grace window, oldest evicted first. * fix(dbt): scope preview state to the run, seed the project on a language switch Previews are keyed by `unique_id`, which is the same string for the same model in every run, and the run-change effect reset only the graph and progress. Opening a second run of one project therefore showed the previous run's rows immediately, and `runPreview` treated them as cached and refused to fetch. A generation counter also drops a preview that resolves after navigation, which the reset alone cannot catch. The dbt project was seeded only by the empty-script bootstrap, but dbt is in the ordinary language picker: reaching it by switching a draft produced a script with no `dbt_project.yml`, which the runtime refuses to deploy or run. Both entry points seed now, and neither touches modules that already exist. * chore(dbt): cache the agent graph endpoint's query for the EE offline build * fix(dbt): publish the graph a moved profile built A run snapshot stopped before everything `persist_ingest` keys by PATH, which is right for a one-off `vars` override and wrong for the other two reasons a run re-ingests. `graph_is_per_run` was one bool for all of them, and the profile drift check both sets it and reads back what the publisher recorded: a profile moved A->B was detected by every run forever, each paying a `dbt parse` for a snapshot nobody reads while the asset rows went on naming schema A. The reason is carried now (`GraphRefresh`), and it decides both writes. Drift is the version's own move, so it rewrites the VERSION's graph and republishes the ownership that ends the drift; a dynamic descriptor snapshots under its job id and still publishes; anything the CALLER scoped — an overridden `vars`, a narrowed `select` — snapshots and publishes nothing, so one invocation's subset can neither stand as what the script owns nor drop the models it left out from the version's graph. Where they meet the caller wins, and the next ordinary run settles the drift. A restore also rebuilt `run_results.json` by copying the generation directory a second time, so a burst of saves pruning it mid-restore left `dbt retry` with nothing to resume and a job that reported success. It is written from the bytes the restore already read; a manifest that went the same way falls back to the parse a database restore pays anyway, and a generation that vanished before either read falls back to the database's row for that same run instead of reporting there is nothing to retry. * fix(dbt): select a row preview by package, not by file path The preview built dbt's FQN by dropping one segment of `original_file_path`, which assumes the model root is `models/`. A project setting `model-paths: ["src/models"]` turned `src/models/marts/orders.sql` into `pkg.models.marts.orders`, and dbt's matcher — equal lengths, compared from the front — resolves that to nothing: the preview came back empty for every model in the project. It selects `<name>,package:<pkg>` instead. The comma is dbt's intersection operator, so this names the node by its own name and the package it belongs to, which is what the FQN was reaching for and needs no knowledge of the resource root. Verified on dbt-core 1.12, dbt-core 2.0.0-alpha.5 and fusion 2.0.0-preview.202, including a package shipping a model whose name the root project also uses. * fix(dbt): refuse a lockfile version that is not one, keep a named selector Two things a preview reaches that a deploy does not vouch for. A raw preview submits its own `lock`, so `engine_version` arrives from the caller and was interpolated straight into the engine cache path — `../..` in it made the download, extraction and rename land anywhere the worker can write, and provisioning runs on the host rather than inside the dbt jail. Both it and `adapter_version` (a pip requirement) are now accepted only as a plain version token. `effective_selector` also read any submitted `select`/`exclude` as an override of the descriptor's named selector. The generated run form posts a default back for every field the caller left untouched, and a selector descriptor's `select` default is `[]` — so pressing Test, saving a schedule or firing a webhook built the WHOLE project instead of `--selector nightly`. An override is now one that DIFFERS from the descriptor's own value; a run that wants the whole project despite the selector asks with `["*"]`. * fix(dbt): let a moved profile settle, from the runs that actually happen Two ways the drift check could never come to rest, both verified against a real run of a real project on a normal worker. `add_caller_args` read any submitted `select`/`exclude` as a caller's narrowing. The generated run form posts a default back for every field left untouched, so every run from the UI, a schedule, a webhook or a flow step carried them and was marked caller-scoped: with the profile moved A->B, each one stored its models under its own job id and left the workspace graph — and the root the check reads back — at A. Since no UI run omits the field, the "an ordinary run settles it" escape hatch was unreachable. Both this and `effective_selector` now ask one question, `selection_is_overridden`: DIFFERENT from the descriptor's, not merely submitted. The root was also recorded beside the path-keyed publication rather than beside the graph it describes, so a version that cannot claim the path — an older one run by hash, a deploy overtaken by a newer one — rewrote its graph at the moved root and recorded nothing. Its next run then compared against a root that was absent or two moves stale and skipped the refresh its own run page needed. It is written wherever the deployed row's graph is. Verified end to end: same UI-shaped arguments before and after, the moved profile now republishes (asset rows and version graph both move to the new schema), a second run detects nothing and re-parses nothing, and moving the profile back settles it again. `prune_dbt_run_graphs` also ran from runs alone, while a deploy writes a whole node set of its own, `raw_code` per model included: a project redeployed on every push by CI and run nightly kept one full graph per push until the next run, and one deployed but never run kept them for good. * fix(dbt): drop a self-dependent effect in the run graph `previewGen` was `$state` written by the effect that also reads it, three lines under a `finalLoadFor` that is a plain `let` for exactly that reason. Nothing reactive reads it — the only reads are inside `runPreview`, a plain async function — so it becomes a plain `let` too. * fix(dbt): pin a retry to the engine versions it resolved `run_identity` carried the engine KIND but not the version it resolved, nor the dbt-core 1.x adapter's. Redeploy an unchanged project after a release and it locks a newer dbt or adapter while the saved `run_results.json` still passes the check, so `dbt retry` feeds one version's artifacts to another — the exact reproducibility the lockfile exists to hold. Both resolved versions are in the identity now; a real failure and retry still resumes. Also drops three comments that outlived what they describe: two said `[]` clears a descriptor's selector, which `selection_is_overridden` reversed, and one pointed at an agent-worker guard that no longer exists — the agent path reaches the ingest deliberately and publishes through the API. * fix(dbt): discard a run graph the page has already navigated away from The component is reused across runs, so a slow `/jobs/dbt_graph` or progress response could land after the reset and put the previous run's models, statuses and failure state on the current run's page, where nothing would fetch again to correct it. Every response is now checked against the generation it was requested under — the counter the preview path already used, renamed for what it means. The graph poll also backs off. Neither of its stops is reachable for a whole class of runs — `dbt_snapshot_job` never matches a static descriptor, and `polled` stays empty for the engines that emit no node events — so an ordinary run walked to the cap, re-sending every model's SQL 40 times in two minutes. * fix(dbt): forget the previous run when the durable save fails, keep quoting `save_run_state` returns the database error when its upsert fails, which leaves run N-1's row and local generation in place: same project, same arguments, so a `dbt retry` matches them and resumes an older attempt's failed nodes against this checkout — the outcome the no-results branch twelve lines above calls `invalidate_run_state` to prevent, reached by another door. It now goes through the same call. Best effort, since the delete goes to the database that just refused a write, but the local pointer is what a retry landing back here reads. The run page also rejoined a relation's parts after `splitRelation` stripped their quotes, so the one name the button exists to paste — `"wh"."analytics.v2"."Order Items"` — was copied as something no client resolves. It copies `relation_name` verbatim. And a source on a finished run was called another project's: the check that guards against two projects claiming one relation asks whether this run executed the node, and a run executes no sources — they appear in no `run_results.json`. Nothing materializes a source, so that warning could never be true of one. Docs: the `vars`-override paragraph still said such a run does not refresh the graph, which the table above it contradicts — it refreshes under its job id and publishes nothing. * fix(dbt): keep the asset rows and the version's models describing one graph The workspace graph takes an asset's relations from the path-keyed `asset` rows and its models, SQL, tests and lineage from the version's `dbt_node`/`dbt_edge`. A dynamic descriptor published the former while storing the latter under its own job id, so a placeholder that moved an alias or a schema left the current graph with assets no model stands behind — nothing dbt contributes to them survives. Ownership is published exactly when the VERSION's graph was written now, which is the only state in which the two agree. Two cases are settled elsewhere by design: an override's relations are a one-off, and a dynamic descriptor at a moved profile keeps the deploy's ownership until a redeploy — its runs each show their own models and it re-parses regardless, so the undetected drift costs it nothing it was not already paying. An agent worker has no durable row, so its local `current` pointer is the whole of what a retry reads — and every local publication failure returned success with the PREVIOUS run's pointer still in place. Where a row exists that is harmless (`restore` takes a local generation only when the row names it), so the abandonment is scoped to the agent case. A failed `dbt show` also cleared the retry state: the preparation-failure exempts `retry` but not a read-only command, and the run page's row preview is exactly that, run as the principal the state is keyed by — so a preview that could not provision took the retry away from the run being looked at. Frontend: the run-change reset left `loading` and `failed` behind, so the gap before the next run's answer rendered "no models in the asset graph" — a claim about the descriptor — over a project that is fine. And three derivations argued from "the graph is the current deploy", which the pinned endpoint made untrue; each is still needed, for the version-graph rewrite and retention reasons now written down. * fix(cli): let --skip-scripts cover a script's module files The module shortcut in `elementsToMap` maps the file and `continue`s before every skip filter, and a module is deployed as part of its parent script — so `wmill sync push --skip-scripts` still pushed the script whenever one of its modules changed, and pull still overwrote them locally. Harmless while a module was a rare helper file; every file of a dbt project is one of these now. * docs(dbt): a dynamic descriptor's ownership stays the deploy's * fix(dbt): read the run out of a failure whose message has braces of its own `parseDbtRun` anchored on the FIRST `{` in the error message and parsed everything after it. The worker appends the structured result after the error text, and dbt's errors carry braces — a Jinja template, the compiled SQL, an adapter's own JSON — so the failures most worth reading were the ones whose summary and per-node outcomes the run page dropped. Every brace is tried now, bounded, and the first that parses as a run wins. Pins the EE revision that gives the agent publish endpoint the deleted-version guard the SQL path takes: deletion is soft, the foreign key still accepts graph rows, and the pinned graph query serves non-live versions, so an agent finishing during a delete put a deleted project's model SQL back on screen. The query is byte-identical to `persist_ingest`'s, so the offline cache already covers it — verified with a full-EE `SQLX_OFFLINE=true` check. * fix(dbt): seed a project when a modular draft switches to dbt `seedDbtProject` returned whenever the draft carried any module at all, so a modular script holding a `helper.ts` reached dbt with none of what dbt needs: the project view is read-only, and the worker refuses a bundle without `dbt_project.yml`, so that draft could neither run nor deploy. Keyed on the project file now, and the seed goes in under whatever is already there — the previous language's helpers are inert to dbt and the user's to remove. Also records this runtime's schema in `backend/summarized_schema.txt`: the `table` asset kind, the `dbt` script language, the five dbt tables and the `materialization_status` enum the progress table uses. * docs(dbt): move the pipeline-membership rationale out of the deploy path * fix(dbt): gate a pinned run's model SQL on the version it belongs to The `EXISTS` against `script` is the only thing standing between a share-link viewer and the project's source, and it matched the workspace and path alone. `extra_perms` is a grant on a ROW: archive a version that granted someone access, recreate the path with narrower permissions, and that stale grant satisfied the probe while the query returned the NEW version's `raw_code`. Both probes name the hash now. The regression test drives exactly that shape and fails without it, returning `select 2` to a caller granted only on the archived version. * fix(dbt): keep a delimiter an identifier escaped by doubling Every dialect these relations come from escapes its own delimiter by doubling it, and both split functions closed the quoted section on the first half and reopened on the second: `"schema"."a""b"` came out as `a.b`. The manifest keeps the real spelling, so the run wrote its per-model status and row counts under an asset path no graph node has — the node simply never moves, which is the failure mode this splitter exists to prevent. Fixed in the worker and in its frontend mirror, which have to agree, with a case per delimiter on both sides. * fix(dbt): key retry state by the caller, not only by the principal it runs as An `on_behalf_of` script executes every caller's job as its owner, so `permissioned_as` names one principal for all of them and the retry state — the durable row and the worker-local generation both — collapsed onto a single entry. After one caller's run failed, the next could submit `dbt_command: retry` and resume it: their arguments replayed against the warehouse, and handed back through `invocation_args`. Nothing else separated them, and on an agent worker the local directory is the whole boundary. `created_by` joins the key in both places. For an ordinary script it changes nothing — `permissioned_as` is already that caller — and a run that was itself superseded was never resumable anyway. Includes the offline cache for the four changed queries and the three the pinned-graph regression test added last commit, which had none: `prepare` without `--all-targets` does not compile test targets, so CI's `SQLX_OFFLINE=true ... --all-targets` would have failed on them. * fix(dbt): compare the schema too when reporting a relation that moved `relationDrift` compared the leaf name alone, and the move it exists to report — a profile repointed at another schema, which a later run then writes into the version's graph — leaves every model's name exactly where it was. So the one case that reliably produces a graph naming relations this run did not write was the one case the notice stayed silent for. The schema segment joins the comparison, qualified against qualified: an unqualified one means the target's own database, which the relation names anyway, so comparing that would report a move on every node. * fix(dbt): bound the retry state now that it is keyed per caller Keying by `created_by` fixed one caller resuming another's run and created a growth problem doing it: a shared `on_behalf_of` script kept one row and one worker directory for everyone who had ever run it, and the generation prune only bounds files INSIDE a directory. Three bounds, none of them new machinery. A run with nothing failed or skipped saves nothing — `dbt retry` builds from those nodes alone, so that state could only ever be refused — while still clearing what the previous run left, since its failures are no longer what last happened here. The rows expire on the same 30-day clock as a run snapshot, swept per path by the prune every dbt job already spawns. And the worker-local directories are swept there too, by the age of the pointer a save rewrites, because their digest names neither the script nor the caller. * docs(dbt): the retry state is worker-affine only on an agent worker * fix(auth): only the server may set a token label that names a user `create_token_internal` wrote `NewToken.label` verbatim, and the auth layer reads some labels as an IDENTITY: `username_override_from_label` maps `ephemeral-script-end-user-<name>` to exactly `<name>`, which then becomes `created_by` on every job that token pushes. The label is free-form request input, so any member could mint a token that speaks as somebody else — the shape `require_job_read_access` already works around when it refuses to trust `username_override` and falls back to an RLS probe, and the one that made dbt's retry-state key (`created_by`) forgeable for an `on_behalf_of` script. The labels are refused where request input enters: the member-facing `tokens/create`, and `impersonate`, which names its subject in `impersonate_email` and has no business renaming the caller too. The legitimate producers are unaffected — a job's own token comes from `create_token_for_owner` in the worker, and native triggers and app-embed tokens build their labels themselves rather than accepting one. `Ephemeral lsp token` stays allowed: its override is the fixed sentinel `lsp`, not a name the caller chose, and the editor mints exactly that label through this endpoint for its language server. The test pins the two lists together, so an arm added to `username_override_from_label` that lets a label choose a name fails until it is reserved too. * Revert "fix(auth): only the server may set a token label that names a user" This reverts commit efaa498d82. * fix(auth): only the server may set a token label that becomes a bare username * revert(dbt): key retry state by the execution principal again Reverts the per-caller key and the retention it needed. `created_by` cannot carry an isolation boundary: it is `display_username()`, which a token LABEL supplies, so two callers can share one value and — before the guard two commits back — one could name a third person. GHSA-8x8x-88qc-qp4r settled that class by refusing to trust the name for authorization, and keying on it here was the same mistake in a new place. It also cost a migration, a PK column, two sqlx cycles and a retention sweep to defend. Back to `(workspace, script_path, permissioned_as)`, which is derived from the authenticated username and cannot be chosen by a request. What that boundary is, and the one case it does not cover, is now written down where the retry is specified rather than left to be re-derived: anyone entitled to run the script as that principal may resume its last failure — the same capability as re-running that job, since running it requires the read access that already shows them the run and its arguments — except for a run pushed `invisible_to_owner`, whose arguments a retry still returns. Closing that means having a retry NAME the job it resumes and authorizing it as a job read, which is a change to the run argument, not to the key. * fix(dbt): give a caller's own selection a graph, and load a finished run once A run that overrides `select`/`exclude` was marked caller-scoped but not per-run, so it ingested nothing and its page fell back to the deployed graph. That is only right when the override NARROWS the descriptor's selection — `["*"]`, or any model outside it, builds relations the deployed graph never had, and those are exactly the ones whose progress, SQL and lineage had nothing to draw. It ingests its own graph now, still caller-scoped, so the subset stays out of what the script owns. `relationDrift` also read a schema whose own name contains a period as a move: a segment holding one is ambiguous — an overridden database, or a schema really called `a.b` — so either spelling now counts as unmoved. And a finished run fetched the whole graph twice on mount, every model's SQL included: the reset effect and the finished-run effect both fired in the same tick. The reset only loads while the run is in flight; the other owns the finished case, because a snapshot can land after the run ends. * fix(dbt): classify a restored retry by its resolved arguments `add_caller_args` was handed `raw_args` on the retry path — the arguments as SUBMITTED, where a `select` spelled `$res:` is still a string. `arg_list` then refuses it as "must be a list of strings" and the retry dies before parsing, for a reference that resolves to the very list the failed run built with. The resolved map decides it now, matching the selection resolver and the build; `raw_args` stays what is persisted and published, so no resolved secret outlives the job. Verified against the shape it breaks on: a `select` from a resource, failed, then resumed. Two docs that outlived their code: `dbt_script_hash` is described as a fallback for a job naming no deployed script rather than as the pinning mechanism, since a script job's version comes from the job row and the query value is ignored (spec plus the generated client, which carries the same sentence); and `GraphRefresh`'s fields no longer claim a caller's selection only ever narrows, which is the premise the previous commit disproved. * fix(dbt): a hidden run keeps no retry state The state is keyed by the execution principal, which every caller of an `on_behalf_of` script shares, and a retry publishes the arguments it restored — so for a run the other callers cannot read, that retry was the one way to see them. The equivalence the key rests on ("resuming it is the same capability as re-running it") holds only while the run IS readable, and exactly one run is not: one pushed `invisible_to_owner`. Those now save nothing, and clear whatever the previous run left, since this invocation happened. A hidden run therefore cannot be resumed by anyone, its author included — the cheaper half of the trade, and the reason having a retry NAME its source job is the design that would give it back. Verified: a visible failure still saves and resumes; a hidden one leaves no row, and the retry after it refuses without returning the hidden `vars`. Also collapses `GraphRefresh::caller_scoped`, which stopped distinguishing anything once a selection override became per-run: both writers set both flags, so `snapshot_job` and `publishes_ownership` are now one field and its negation — which is what lets the agent payload carry a single `per_run` bit, recorded there. `is_reserved_token_label` says why `ephemeral-` is reserved (a client-chosen system label is a token its owner cannot list or revoke) rather than restating a username rule its second prefix does not follow. And the module-cache import comment states the invariant once, without the shape of the cache that preceded it. * fix(dbt): a pre-build failure leaves the saved run alone Three exits cleared the retry state before `dbt build` ever ran: a preparation failure, a failed `dbt parse`, a failed ingest. None of them touches a relation, so the warehouse is exactly what the previous run left and its failures are still the accurate description of it — clearing there just costs a resumable failure, and a resume that no longer fits is refused by `run_identity` and the arguments digest regardless. The reachable one is a cancellation during provisioning. Only an interrupted BUILD invalidates, which the save at the end of the job already decides. Verified: a failure saves state, a run whose selection matches no node fails in the ingest and leaves it, and the retry after that still resumes the original failure. Agent-published graphs also had no sweep — `Connection::Http` snapshots every run and spawns none — so the EE endpoint prunes too (pinned at f58bf06), and `prune_dbt_run_graphs` now names every writer rather than only the runs. Drops `ephemeral-webhook-` from the reserved token labels: it yields the label verbatim, not a bare username, so the check now matches the rule its doc and its test state, and the reason given for it was wrong about what `is_user_token` costs an owner. And the result contract lists `invocation_args` — the field most needing it, being another invocation's arguments on a retry. * fix(assets): decode a doubled delimiter when canonicalizing a table key `split_relation` decodes it on the worker side, so the canonicalizer had to as well: a relation whose identifier escapes its own delimiter — `"sales""east"` — was filed by the ingest under `sales"east` and by a hand-written `table://` annotation under `sales""east`. Two nodes for one table, no edge, which is the exact split this key exists to prevent (decision 11). Fixed for both the name and each half of a database-qualified schema segment, with a case per dialect. `invalidate_run_state`'s contract also still promised the pre-build behaviour the previous commit removed; it now names the cases that do invalidate — an interrupted build, and a run hidden from the script's owners. * fix(dbt): reclaim package trees no project asks for any more The cache key covers the whole project digest — a `local:` dependency's content is in no manifest, so nothing narrower is safe — which means every edit of a project that declares packages publishes another full dependency tree, and nothing ever removed one. A worker that lives through a hundred deploys held a hundred trees until an operator cleared the entire cache by hand, and the disk that fills fails every job on that worker, not only dbt's. Swept by last USE, not by publication: a hit dates the tree, so a project unchanged for months is not evicted from under the jobs still running it. The marker is a sibling rather than a file inside the tree, which the restore copies into the project. Staging directories go after a day — one belongs to a single job and is removed when it ends, so an older one is from a worker that died mid-publish. Verified by planting a 30-day-old tree and a stale staging dir: one run reclaimed both and left a fresh tree alone. Narrowing the key to the declared `local:` paths would fix the churn as well, and is deliberately not done here: getting it wrong runs a project against another revision's packages, which is worse than the disk it saves. * docs(api): declare the dbt half of the asset-graph response `AssetGraph` is the response of both `/assets/graph` and the new `/jobs/dbt_graph/{id}`, and the spec described neither `assets[].dbt`, `runnables[].dbt`, `dbt_edges` nor `dbt_snapshot_job` — so every client generated from it saw a graph with no dbt metadata, no `ref()` lineage and no snapshot marker, which is why the run page reaches the endpoint with a raw `fetch` and a hand-written cast. `frontend/src/lib/gen` is gitignored, so the spec is the only tracked description of this surface. Declared, and the client regenerated from it now carries all four (verified: the regen is otherwise a byte-for-byte no-op). * fix(dbt): stop the package sweep from racing the restore it protects Three faults in the sweep the previous commit added. The `.last_used` marker was written AFTER the copy, and the tree it matters for is the one at the retention edge — where the first use in a fortnight is also when the sweep fires. So the source looked stale for the length of the restore and a concurrent job's sweep could remove it mid-copy, failing a run that should merely have refetched. Marked before the copy, and a restore that loses the race anyway is treated as a cache MISS: `dbt deps` resolves the tree again, costing a fetch rather than the run. The sweep also sat inside `if let Connection::Sql`, though it is a walk of the worker's own disk that needs no database — so an agent worker, which has none, published a tree per project edit and reclaimed nothing. That is the state the doc claimed was fixed, and the same argument the graph sweep makes two lines above: retention that depends on the connection is not retention. And the canonicalizer's two halves disagreed after `5890436`. `unquote_identifier` requires the quote at both ends, so a lone `"` in an already-decoded name survives; the schema walk treated one as opening a quote and dropped it, filing the ingest's `sa"les` under `sales` while an annotation's `"sa""les"` decoded to `sa"les` — the split this canonicalization exists to prevent, in the function I had just touched. The halves share one rule now, and the test drives the decoded spelling against the quoted one rather than asserting only the annotation's. * fix(cli): track a dbt project's parent descriptor for every authored file `buildTracker` decides whose top hash `wmill-lock.yaml` refreshes, and it reached the module branch only for files matching a Windmill script extension. A dbt project is mostly files that are not: `dbt_project.yml`, `packages.yml`, schema YAML, seed CSVs. Editing any of them left the descriptor untracked and its module-inclusive hash stale, so the lock disagreed with the bundle that was pushed. Module paths are now handled ahead of that gate. The parent was also derived by searching the RAW path for `__dbt/`, which finds nothing on Windows, where the folder is spelled `__dbt\` — so even a model edit was skipped there. It goes through `getScriptBasePathFromModulePath`, which normalizes separators, and which the sibling helpers already used for this exact reason. The regression test drives all three shapes. With the old gate reinstated it fails on the non-script files and on the backslash path, and passes only for `.sql`. * fix(dbt): re-resolve unlocked dependencies, and reclaim retry state nobody writes A project with no checked-in `package-lock.yml` asked dbt to RESOLVE its ranges and mutable git revisions, and dbt does that on every run. A cache hit skipped `dbt deps` entirely and every hit refreshed the marker, so the first resolution was pinned for as long as the project stayed in use — a version range that moved was never picked up. An unlocked tree is now a miss once it is a day old; a locked project keeps its tree, since the lock is in the key and pinning is what it asked for. Retry state: `invalidate_run_state` removed the pointer and left the generations, and the whole state root was swept by nothing after `a236972` — so a script that stops running, or a principal who stops running it, kept its last generations (a `manifest.json` each) for good. Forgetting the state now prunes its generations, and a 30-day sweep of the state root runs beside the package sweep, for the reason that one already gives: retention that runs only for the script being run is not retention for the ones that are not. This restores one half of what `a236972` reverted — the local sweep, not the per-caller key or the row retention — because it never depended on that key. The graph orphan sweep also committed its marker delete separately from the two deletes it gates, so an error or a restart in the gap left graph rows whose marker was gone — and since the sweep only runs when a marker went, every later call skipped it and those rows were unreachable for good. One transaction now. `parseDbtRun` scans from the LAST brace: the payload is appended, so backwards finds it immediately where forwards parsed the whole message once per brace in the error text. And `reject_reserved_label` moved into `create_token_internal`, the one path every caller-supplied label reaches, so the invariant sits where a new route would break it. * fix: undo two of my own regressions, and agree with the backend everywhere `2e34dd48` hoisted the module check above the extension gate, which is right, but did not bring the entry-point rule with it: a folder-layout script's METADATA is `<base>__mod/script.yaml`, an entry-point path, and pushing it as a content file makes the metadata pass ask for the language of `.yaml` and abort the whole command. Reached by editing the summary of ANY modular script — pre-existing, nothing to do with dbt. Metadata resolves to its content file now, with a test. `d3716a09`'s unlocked-dependency refresh is reverted. It measured staleness against the last-USE marker, which every hit rewrites, so an actively used project never reached the threshold — and when it did fire, the re-publish `rename` cannot land on a populated directory, so the freshly resolved tree was discarded and the stale one re-stamped: a network resolve thrown away per idle day. Doing it properly needs a publication timestamp and a publish that can replace a tree; the limitation is recorded where the cache is keyed instead of half-implemented. `parseDbtRun` no longer counts braces in either direction. The payload is appended pretty-printed, so its `{` is the only one at column zero — forwards blew the cap on an error full of braces, backwards on one `{` per node, which a few hundred nodes reaches. Test drives a 400-node failure. And `parsePipelineAnnotations` was the third copy of the quote rule, still splitting a doubled delimiter: the live canvas keyed `"sales""east"` as `saleseast` where the deploy keys `sales"east`, so an annotation pointed at a node the deployed graph does not have. All three agree now, including on a lone delimiter in an already-decoded name. * chore(dbt): leave cache retention and the migration's comment out of this PR Two deliberate subtractions, so what ships is the set that has been verified rather than the set that was written. The package-tree and state-directory sweeps are gone (recoverable on `dbt-cache-retention-followup`). They delete directories on a worker, they were the newest code here, and one of them already needed a second pass for racing an active restore — while what they buy is disk hygiene, not correctness: today an operator's `cache_clear` reclaims these caches, exactly as it does for every other language. Both limitations they addressed are now recorded where the cache is keyed. `invalidate_run_state` still prunes the generations it orphans, since that reuses the existing pruner and its grace window rather than sweeping a root by age. And the migration is back to its pre-PR bytes. The only change left in it was a corrected comment, which every database that already applied this migration would have paid for with a checksum mismatch at startup — the explanation lives in `prepare_project` and in the design doc, which is where it is read. * fix(dbt): pin the resolved package tree to the deployed version A cache hit skipped `dbt deps`, so an unlocked range or mutable git revision stayed on whatever the first worker resolved, and a retry could feed one resolution's run_results.json to another. The deploy now records the digest of the generated package-lock.yml, the package cache is keyed by it, and it joins the run identity; a worker that resolves anything else is refused rather than run. Also carries the agent-wire round trip test for an ingested manifest, and the EE pin for the agent publish endpoint. * docs(dbt): state the dependency contract the deploy pins The design doc still described the package cache as keyed by the project digest alone, and carried an open question about refreshing unlocked dependencies that the deploy-time pin answers: resolution happens once, at deploy, and every run of that version installs it or is refused. Records what that costs and buys — committing package-lock.yml makes deploys cache-hit, deploying again is what picks up a newer range — and that these worker-local caches are reclaimed by cache_clear, as every other language's are. * fix(dbt): four findings from the review round - An interrupted retry republished the results it had only restored. A retry starts with the previous attempt's run_results.json in place; cancelled before dbt rewrites it, the save dated those failures to this job, so the next retry rebuilt nodes this one had already redone. Verified by A/B: without the check the row survives, carrying the cancelled retry's id. - The deploy refused a committed package-lock.yml that dbt itself updates, which it does whenever the sha1_hash it recorded for packages.yml no longer matches. Only a run has a resolution to be held to; the deploy establishes one. - A second graph load for the SAME run could land out of order, leaving a finished run showing the deployed fallback for good. - Directory nodes in the project tree carried no path, so two of one name at the same depth shared a collapse key and folded together. * fix(dbt): name every adapter, and stop guessing which project owns a model The invalid-adapter message listed four of the eleven accepted spellings. And a relation may have several script producers, none of which the provenance record identifies — prefixing the first one names a __dbt folder that does not exist, so an ambiguous relation now shows the path inside the project alone. * refactor(dbt): name the graph-snapshot column for when it is written `published_relation_root` described a design persist_ingest does not use: the root is recorded by every ingest that is not a run's own snapshot, including one that publishes no ownership — deliberately, since a version that cannot claim the path would otherwise record nothing and compare against a stale root forever. The column comment said the opposite and the name followed it. Safe to edit the migration in place because this PR introduces it: no database outside a checkout of this branch has ever applied it. One that has needs UPDATE _sqlx_migrations SET checksum = decode('<sha384 of the file>','hex') WHERE version = 20260725084314; alongside the column rename, or a fresh database. * style(dbt): bring comment blocks within the four-line guidance AGENTS.md asks for each invariant at the place someone would break it, in at most four lines. The new dbt files carried 60 inline blocks past that, several of them three rationales deep at one site. Nothing durable is dropped: what covered several constraints at once is split to the lines it constrains, and eight comments that had drifted above the wrong test — six stacked over one profile test, two over the wrong executor test — are reattached to the tests they describe. * docs(dbt): carry the snapshot-column rename into the summarized schema The rename reached the migration, both queries, the sqlx cache and the design doc, but not the compact schema summary — which is the file agents read instead of the migrations, so it was the one copy that could mislead silently. * docs(dbt): bound the retry-state and package-pin residuals precisely Two claims in the design doc were true but not precise enough to act on. Retry state: the equivalence between resuming a failure and re-running the job holds because folder read grants job read, which is also what grants execution. It does NOT hold for a u/<owner> script shared through extra_perms, where no folder policy applies and a grantee cannot read even their own on-behalf run. The package pin: the refusal is per worker. A ranged dependency with no committed lock reproduces its resolution only from a cache hit, so it keeps running where the tree is warm and fails on the first cold worker. * fix(dbt): refuse a retry of a run its caller cannot read Retry state is keyed by execution principal, so every caller of an on_behalf_of script shares one saved run. Under a folder that matches job visibility exactly, but a u/<owner> script shared through extra_perms has no folder policy: a grantee can read neither the run nor their own, while a retry published its arguments. The restore now applies require_job_read_access's rule — you can always read a job you launched, otherwise the row must be visible under your own RLS. Deliberately NOT job_perms, which carries the identity the job runs AS: for an on_behalf_of script that is the owner, and probing with it authorizes everyone. Verified with two members on one shared principal: bob is refused alice's run on the u/ path and her marker never reaches his result, alice resumes her own, and bob still resumes alice's run of the same script under a folder. * fix(dbt): close the retry check's own gaps The check landed with three holes and four rough edges, all found by review: - A pruned source run was allowed. dbt_run_state outlives job retention, so that handed the last failure's arguments to whoever asked next. It now fails closed, and says so rather than claiming the run was someone else's. - An agent worker skipped the check entirely: it reaches no database. The saved generation now records who launched it, and an agent authorizes the half it can prove — the launcher may resume. A generation written before this field is not resumable there. - The row was authorized on one read and restored on another, so a run completing in between was restored unauthorized. restore_from_db now takes the job it was authorized against. Also: restore_run_state's doc comment had drifted onto the new helper, the not-a-member comment described the saved run rather than the caller, and the refusal message carried a run of spaces from its line continuation. * fix(dbt): keep the descriptor in --json sync, and fail closed on a stateless row A dbt script's content is `<name>.dbt.yaml`, and `elementsToMap` drops every .yaml when metadata is JSON — so a --json workspace tracked the metadata, the lock and the whole project bundle but not the descriptor: a descriptor-only edit pushed nothing and a fresh pull wrote a script with no source. Pinned by a test that fails without the exemption. And the retry check read job_id flattened, which merged "no row" with "a row naming no job" and skipped authorization for the second. Unflattened, the row that cannot be authorized is refused. Also graph_digest: serde rather than {:?} for edges, and delimited parts. * docs(dbt): state that retry authorization is by identity, not token scope The section claimed the read-access equivalence was enforced. It is, for identity — but the worker never sees the submitting token, and neither v2_job nor job_perms records a scope, so a token denied jobs:read can still resume its own principal's last failure. What it gets is the arguments as submitted, so a reference is re-resolved under whoever retries rather than disclosed. Also names the cases the check refuses outright, which were spread across three commit messages and nowhere a reader would look. * chore(dbt): repin EE after rebasing the agent-graph branch onto EE main * fix(dbt): say why a test-only run has no models, instead of blaming the profile A selection of tests alone builds nothing, and a test is an assertion rather than a relation, so the ingest keeps nodes with no asset to hang on and the graph comes back empty. The run is correct and its results render — but the empty state named the one cause it is not, a project with no warehouse identity, and sent the reader to their profile. The graph itself is unchanged: retaining a selected test's attached model would have a test-only script claim a read on every model it asserts against, which is an asset-graph semantics change and wants its own review. * fix(dbt): authorize a retry by RLS alone, and eight review findings Retry authorization drops the created_by grant that mirrored require_job_read_access's "you launched it". That name is a display name derived from a token label, so a worker cannot tell a launcher from a collision — the objection Codex raised on four heads. Visibility under the caller's own RLS is the whole rule now. An on_behalf_of script under u/<owner> shared by extra_perms is resumable by the owner alone; under a folder every caller keeps the resume. Also, and each verified against the code first: - The local generation is revalidated after the file work, so a newer run publishing state mid-restore no longer lets the superseded one resume. - Both dbt zip lookups in sync normalize the OS separator, as the resource lookup beside them already did: on Windows a pulled dbt project was laid out as __mod, the one layout dbt cannot run from. - A resource storing port as a string no longer silently connects to the adapter default; a value that is not a port is refused. - An oversized but readable project file fails the deploy instead of shipping an incomplete project that compiles and fails at run time. - cache_clear does NOT reach the dbt caches: it clears cache/ while these live under cache_nomount/, as bun's do. The doc said the opposite, and that claim was the justification for dropping the retention sweeps. - A dbt unit test counts as a test for the empty-state message, as it already does for the worker's test-phase detection. - Per-node results keep the catalog segment when rows disagree on it. - The project panel copies through the app's clipboard helper, .dbt.yaml resolves back to dbt in EXTENSION_TO_LANGUAGE, and two test blocks clean up their temp directories. A comment describing a race against a sweep this PR does not ship is gone. * fix(cli): keep the oversized-file check to a bounded read The refusal statted the file and then read it whole to tell text from binary, which is the cost the two comments above it and isBundledModuleFile exist to avoid: a multi-gigabyte seed would be loaded just to be refused. Same 8 KB head read as that predicate. * revert(dbt): accept the retry residual instead of gating it Resuming grants no capability a caller lacks: they may already run the script as that principal, and a plain run builds a superset of what a retry rebuilds. The one thing a retry adds is information -- the result echoes the resumed run's submitted arguments, for the row preview -- and in almost every shape the caller could already read that run: a folder grants job read alongside execution, and an ordinary script's runs are keyed under the caller's own principal. It takes a user-path script AND extra_perms sharing AND on_behalf_of for the two to diverge. Gating it needed an identity the worker does not have. created_by is display_username(), which a token label supplies: trusting it authorizes a collision, and resolving it as a username denied every labelled token -- a CI token is `label-<name>`, no workspace member, so its own retry was refused. That regression cost more than the exposure. Kept, because none of it is about the caller: - a cancelled retry no longer republishes the results it only restored, - the restore is pinned to the row it chose rather than re-reading it, - a newer run publishing mid-restore no longer lets the superseded generation resume. The three sqlx entries the gate needed are gone with it. * fix(dbt): select unit tests, and say why the state key is the boundary `dbt ls` enumerated five resource types and omitted `unit_test`, which dbt treats as its own type rather than a flavour of `test`. A descriptor selecting `test_type:unit` or a unit test by name therefore resolved to the empty set, which the deploy refuses outright with "matched no dbt nodes". Verified both ways on a fresh path: the deploy fails without this and succeeds with it. All three engines list `unit_test` among the accepted values. The restore's state lookup now carries the reason it keys on the principal and not the caller. That decision was in the design doc, the commit messages and a PR thread — everywhere except the line someone would change to "fix" it, which is where AGENTS.md asks for it and where I twice went wrong myself. Also: the `GraphRefresh` doc claimed a dynamic descriptor publishes its graph as the script's ownership, while `per_run_models` snapshots it under the job id and `publishes_ownership()` returns false — only a moved profile republishes. And `fn n` in dbt_profiles.rs had no callers left after `port_of` replaced it. * feat(dbt): offer the resume on a failed run, where the choice is relevant "Run again" prefills the arguments the run just used, so the obvious action after a failure rebuilds the whole project while the cheap resume stays invisible unless the reader knows dbt_command has a retry value. A finished run with failed or skipped nodes now says how many, and links to its own run form with the command already switched. A link rather than a submission: the caller still presses Run, so nothing about permissions or arguments changes. Deliberately not automatic. A resume is not a re-run — models are usually rebuilt because upstream data moved, not because they failed — so a schedule that resumed after a failure would leave every model that succeeded carrying yesterday's data while reporting success. * feat(dbt): put the command first, and offer the resume where the rebuild is Three things made rebuilding the whole project the path of least resistance after a failure, when resuming its failed and skipped nodes is what dbt offers: - `dbt_command` was LAST in the generated signature, under `limit`. The schema's `order` drives the run form, so the argument that decides what the run does sat below the ones that only narrow it. It leads now, and the signature test asserts the order rather than the set, with the reason it is pinned. - "Run again" prefills the arguments of the run it came from, so for a failed run it reproduces the rebuild. It now carries `dbt_retry_hint`, and the form shows what resuming would do instead -- without choosing it, so the caller decides. - Its dropdown offers "dbt retry with same args" above the plain re-run, for a failed dbt job only: a run that succeeded has no saved failure to resume. Verified in the browser: both dropdown labels render unclipped, the alert appears only on that path and disappears once the command is switched, and the schema of a redeployed script lists dbt_command first. * feat(dbt): describe the run arguments, and hide the ones a command ignores The form offered six fields at once with no indication that most of them apply to one command each. `retry` reuses the arguments of the run it resumes, so every override is ignored; `limit` belongs to `show`; `full_refresh` to `build`. Each argument now carries what it is for, and a `showExpr` for when it applies, which SchemaForm honours by hiding the field AND dropping its value from the payload. Selecting `retry` leaves the command alone on the form. Not modelled as a `oneOf`, which is the more precise shape: Windmill renders that as a variant selector whose value is a nested object keyed by a discriminator, so the arguments would stop being flat -- and the worker reads them by name, as do schedules, webhooks, the CLI, saved inputs and the retry state's own copy of them. * fix(dbt): a retry may name the run it means to resume The retry actions I added an hour ago appear on any failed run, but the saved failure is keyed by script and principal rather than by job — so from an older run's page they resumed whatever failed most recently, under a label promising "same args". Both now pass `dbt_retry_job`, and a retry naming a run the state no longer holds is refused with the id it does hold. Verified: with failures A then B saved, a retry naming A is refused and names B in the message, while a retry naming B resumes. Also from the same review: the `select` help advertised `state:modified`, which needs a comparison manifest this runtime supplies no `--state` for, and the openflow note still described a dbt script as a descriptor naming a git repo rather than the module bundle this ships. * fix(dbt): make both retry actions actually name their run `dbt_retry_job` is not a form field, so the banner's link to the run form dropped it and the retry resumed whatever failed last — the bug the argument exists to prevent, reintroduced by the affordance meant to use it. The banner submits the run itself now, as the "Run again" dropdown item already did. And an agent worker never applied the check at all: its `latest_job` is always None, so the local generation was accepted unconditionally. Both reviewers found this independently. The decision moved into `chosen_generation`, which the agent path shares, with the refusal naming the run the worker does hold. Pinned by a test that fails without the check, on the connection kind where it was missing rather than on the one that already worked. * fix(dbt): let an unchanged push be a no-op, and stop advertising a retry the form cannot aim Two of a dbt script's fields are DERIVED by the server after the no-op check runs: the lock, which only a dependency job can produce, and the schema, which comes from the descriptor and which no client can derive (windmill-parser-wasm has no dbt arm). The check compared what arrived instead of what would be stored, so every unchanged `wmill sync push` created another version and another dependency job -- the sync churn skip_if_noop exists to prevent. Both are now compared as stored, with the parent required to hold a lock so a deploy whose dependency job failed can still be retried by pushing again. Verified by A/B on a faithful payload: without this the same unchanged push creates a version, with it the existing hash comes back and the count holds. The run form's retry hint is gone. `dbt_retry_job` is not a form field, so a retry started there resumes the last failure of the script rather than the run the message pointed at -- the two buttons name their run, that path could not. Better to not offer it than to offer it wrong. Also reunites a comment with the derivation it describes, 40 lines below where an earlier edit of mine left it. * docs(dbt): an unchanged push no longer re-resolves dependencies The section told the reader that a byte-identical deploy re-pins a ranged dependency, which was true only because no-op detection was broken for dbt. Now that an unchanged push is skipped, moving a pinned resolution takes an actual change. * feat(dbt): offer the resume only on the run it would actually reach * fix(dbt): a refusal names both the run asked for and the one held * docs(dbt): say that show previews one node when several are selected * feat(dbt): a retry names the run it resumes, and the form can fill it in * refactor(dbt): make the run's command a oneOf carrying its own overrides * fix(dbt): refuse a command block that names no command, and keep state off retired paths * fix(assets): let copilot and the assets filter see warehouse tables * fix(dbt): drop the unused db handle from the retry-principal lookup * fix(dbt): stop a preview poll when the run page moves on * fix(dbt): refuse an escaping packages-install-path, re-arm the graph poll on navigation * docs(dbt): concurrent runs of one script are the script's to serialize * fix(dbt): bound what the log tailer holds in the worker process * fix(dbt): a stale prefill answer must not aim another script's retry * fix(dbt): a rename leaves the old path archived, so state must not be rewritten there * fix(dbt): keep a retry's routing inputs, and stop an app-embed token probing resumability * docs(assets): state the authorization the graph helper expects of its callers * fix(dbt): merge the retry into the fetched arguments, not the too-big placeholder * fix(dbt): the command block's label decides the command, not map order * fix(cli): an oversized dbt project file must not read as generated output * fix(dbt): the prefill names a run only when the caller may read it * fix(cli): refuse an oversized dbt file before anything reads its body * fix(dbt): fold table paths as the deploy does, and preview with fetched args * fix(dbt): stream an engine archive to disk instead of holding it in the worker * refactor(assets): the dbt asset kind is dbt://, keyed on the relation * chore: regenerate the auto-generated prompts for the dbt asset kind * fix(assets): rename the kind at the callers that request it, and refuse an unknown one * feat(dbt): show the project's models under the run form on the script page * fix(dbt): the rename missed the dbt-edge membership keys, so the DAG lost every edge * fix(dbt): a preview must run the dbt writer, with the form's arguments * feat(dbt): the models graph takes the bottom of the script page, as a flow's does * fix(dbt): an unknown outcome is not a pass, and a preview keys on its arguments * feat(dbt): a warehouse is configured on the workspace, named like the lake * feat(dbt): asset identity keys on the warehouse name, resolved for agents too * feat(dbt): warehouses are configured in workspace settings * feat(dbt): the descriptor lives in the project and is optional * docs(dbt): the warehouse is a workspace setting and the descriptor is optional * fix(dbt): the warehouse route names its capture, and the editor stores the map * test(dbt): pin the warehouse setting round-trip and the job-scoped route * fix(dbt): a project-owned profile must authorize the warehouse it names * fix(dbt): the settings tab imports TextInput from where it lives * fix(dbt): a warehouse name is validated wherever one enters, and an absent descriptor is not a diff * fix(dbt): the settings inputs pass their placeholder the way TextInput takes it * fix(dbt): pushing a project that has no descriptor is not an error * chore(dbt): keep the module suffix private to its module * fix(dbt): warehouses survive a fork or rename, and an empty descriptor is never a file * fix(dbt): a workspace rename carries its dbt graph with it * fix(dbt): a rename takes the run state with it, not only the graph * chore(dbt): drop the query-cache entries the settings edits orphaned * docs(dbt): state the three rules that keep an absent descriptor absent * fix(dbt): an unreadable descriptor is an error, and an emptied one is a deletion * feat(dbt): the warehouse is unpermissioned, like the workspace bucket * feat(dbt): an agent worker's run reports its per-model state too * docs(dbt): how an agent worker reports what it ran * feat(dbt): a project-owned profile reports its database, so its models share nodes * style(dbt): rustfmt the files this branch touched * fix(dbt): drop the job id write_profiles no longer reads a resource with * fix(dbt): a preview reads the graph's edges and sends the model, not the cache key * fix(dbt): a deleted producer explains itself instead of a bare 404 * fix(dbt): one warehouse resolution path, interpolated against the job * fix(dbt): a project with no descriptor pulls and opens in dev mode * feat(dbt): one file tree, editable, and Test builds the model you have open * fix(dbt): a preview asks the writer's own graph, and lfs keeps its omission * fix(dbt): metadata stays beside the project, and only a model narrows a build * fix(dbt): a descriptor-less project previews, and dev reloads the whole bundle * fix(dbt): forks carry the graph, and a vanished project stops the push * fix(dbt): no-auth resolves warehouses, and a preview runs the version it read * fix(dbt): a retry recognizes its own run when the profile carries a job token * fix(dbt): the project marker is not deletable and the descriptor path is reserved * fix(dbt): a templated adapter type is refused, and a removed project archives * perf(dbt): the manifest lands in batches, and .py models are creatable * fix(dbt): only a 404 excuses a failed archive, and edges are covered * fix(dbt): a python model narrows the build, and a failed removal is not silence * fix(dbt): a settings name may be any word, and the reserved path is stated once * fix(dbt): an absent descriptor needs its project, and the form matches the run * fix(dbt): a bundle is dbt by language, and both job routes agree who may call * docs(dbt): every identity doc names the warehouse, as the code does * perf(dbt): an agent posts its run's progress once, and show cannot touch a seed * fix(dbt): bound what a warehouse may be named, and say who may ask * fix(dbt): a run's graph withholds what the project's author wrote, and dbt_project.yml is rendered before it is read * feat(dbt): withhold dbt from the language picker until it has its own editor * fix(dbt): refuse a path claimed by both a dbt project and an ordinary script * test(dbt): the bundle assertion is a set, not an order * fix(dbt): a project's generated dirs are re-read when its config changes * test(dbt): assert the synthesized descriptor without assuming a separator * fix(dbt): both push paths refuse a shared path, and every writer sweeps its progress rows * fix(dbt): preview only what dbt show can select, and a retry keeps its command block * fix(dbt): an identifier carrying a path separator gets no asset key * fix(dbt): no engine ships in an image, and show may name only one node --------- Co-authored-by: Claude Opus 5 <noreply@anthropic.com>	2026-08-01 11:58:05 +00:00
Ruben Fiszel	dda59767c2	feat: stamp webhook trigger_kind on token-driven job runs (#10431) ... * feat: stamp ui vs webhook trigger_kind on direct job runs * fix: gate ui trigger kind on min worker version and dedupe display names * docs: state that the ui trigger kind attributes rather than proves * refactor: fold the trigger fallback into one trigger_or_fallback helper * feat: hold trigger_kind as a tolerant label on the worker paths * chore: refresh the sqlx offline cache for the trigger_kind label queries * chore: update ee-repo-ref to 7de7daff5eed410e0c815ad6b292d2b4303f02f2 This commit updates the EE repository reference after PR #700 was merged in windmill-ee-private. Previous ee-repo-ref: 974ab910d9a30c5565e1198ee312acc6d11239f3 New ee-repo-ref: 7de7daff5eed410e0c815ad6b292d2b4303f02f2 Automated by sync-ee-ref workflow. * fix: keep the API job structs tolerant of unknown trigger kinds too * chore: point ee-repo-ref at the merged EE main --------- Co-authored-by: windmill-internal-app[bot] <windmill-internal-app[bot]@users.noreply.github.com>	2026-07-31 14:46:08 +00:00
Ruben Fiszel	8d684c0b23	fix(ai-agent): keep tool description through flow deployment (#10373) ... Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>	2026-07-28 13:09:30 +02:00
Ruben Fiszel	3b95a2d096	feat: reusable AI agent steps with rigid linking and edit/fork (#9825) ... * feat: reusable AI agent steps with hybrid linking and evals Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat: make linked AI agents rigid (read-only) with unlink-to-fork Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat: show inherited agent config read-only on linked step Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat: edit/update a saved agent in place via upsert Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat: bind linked AI agent tool inputs to host flow context Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat: rebind linked AI agent tool inputs via graph tool nodes Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix: linked AI agent tool nodes, step test, and read-only card Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * chore: remove ai_agent resource type migration, sync from hub instead Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * chore: remove AI agent eval suite and run endpoint, defer to later Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * chore: unwire eval routes, types and UI (completes eval removal) Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * docs: update reusable AI agents guide for eval removal and tool rebinding Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * chore: regenerate system prompts for AIAgent agent/tool_inputs schema Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix: strip brain transforms on link, avoid dirtying flow on tool open Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix: flow-local test form and linked-agent marker in read-only graph Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix: store linked tool overrides as diff from resource base Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat: resolve linked agent tools in read-only viewer with fallback Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix: use operating workspace, block non-static provider, warn on unbound tool inputs Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix: resolve linked parent's tools from resource for nested agent tool lookup Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix: scope linked-agent tools by flow path, thread workspace to path check and embedded viewer Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix: strip flow-context tool inputs on agent save, drop unbound-inputs warning Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix: persist agent edit mode across tool selection, show linked tool code read-only Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat: show linked agent resource path in node definition panel Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * refactor: edit linked tool inputs in step panel, make tool nodes display-only Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * refactor: wire step-panel tool bindings (completes display-only pivot) Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix: single scroll for linked card, agent path as node label, drop fill-inputs in tool cards Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * style: align linked-agent UI with design tokens and components Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix: separate linked tool select target from module id to unbreak agent clicks Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@aanthropic.com> * fix: save agent tool inputs verbatim, host flows override via tool_inputs Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix: scope agent edit state by flow path, require linked-tools scope at init Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix: block saving an agent whose static provider is incomplete Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix: type errors in agent tool bindings and save drawer input Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix: key agent edit state by workspace, resync tool bindings on external changes Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix: include workspace in linked-tools scope and tool schema fingerprint Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix: remove unused workspace prop from FlowModuleSchemaMap Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * refactor: drop linked-agent placeholder tool node, path label suffices Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix: workspace-qualified resource links, guard stale tool schema loads Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix: keep flow tool overrides out of the agent on edit, fold only on unlink Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix: fold preserved tool overrides into the step on edit cancel Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix: refuse overwriting non-agent resources on save, show memory kind on linked card Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix: consume picker value, invalidate edit state on undo/reinit, cap nested agent tools Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix: guard in-flight edit fork against restores, migrate edit state on rename Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * refactor: validate agent edit state by fork identity instead of path keys Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix: key agent edit entries by fork marker alone, immune to editor nesting Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix: keep agent edit state across structural graph edits and flow renames Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01B6kq9PYqNdc5q7ubidBYAs * fix: centralize agent edit reanchor, guard in-flight saves, seed rename scope from flow path Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01B6kq9PYqNdc5q7ubidBYAs * fix: ancestry-keyed edit reanchor and doc-scope sweep for republished linked tools Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01B6kq9PYqNdc5q7ubidBYAs * fix: guard stale linked-tool fetches and resolve while-loop nested linked agents Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01B6kq9PYqNdc5q7ubidBYAs * fix: drop empty tool override entries on revert and correct stale viewer comment Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01B6kq9PYqNdc5q7ubidBYAs * docs: drop stale eval mention from the linked-agent comment Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix: deploy linked agent resource, guard viewer fetches, align tools schema Address review findings on the reusable-agent branch: - Cross-workspace deploy never collected a linked step's `agent` resource, so the deployed flow failed at runtime unless the agent already existed there. - The read-only viewer published resolved tools without the generation guard flowState uses, letting a superseded link's tools win a race. Share one guarded publisher (`publishLinkedAgentTools`) between both call sites. - `tools` was still required in the OpenFlow AiAgent schema while the deserializer defaults it, rejecting hand-authored linked steps; make it optional and narrow the call sites. - Overlay `tool_inputs` in the non-linked branch too, so a flow persisted while a step sits in "Editing" mode still binds tools to this flow. - Cap the linked-tools store's scope map; nothing evicted it before. - Drop the orphaned `.sqlx` entry left by the eval removal, regenerate the copilot OpenFlow schema, and fix the generator's nested-`z.record` arity. - Move `refreshFlowStateStore` out of `agentEditStore` into its own module. - Document that linked agents' tool scripts are outside the lock pipeline. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * chore: regenerate system prompts for optional AIAgent tools Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix: follow saved-agent deps on deploy, accept the linked shape in the schema Round-18 review findings: - Deploying a linked flow queued only the outer ai_agent resource. Follow `$res:` refs inside a resource value (every UI-saved agent has a provider resource) and the agent's own tools, which reference scripts, flows, MCP resources and nested linked agents by bare path. - The AiAgent input_transforms schema still required provider/output_type, so it rejected the very shape linking persists (brain transforms stripped, flow-local inputs kept). Only user_message is always present. - dfs traversed `value.tools` unconditionally through a cast, which throws on a linked module that omits it now that the field is optional. - Trim the flow-refresh invariant comment to the 4-line limit. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix: recurse into inline nested agent tools on deploy, require provider when unlinked Round-19 review findings: - The deploy walk only inspected a saved agent's top-level tools, so an inline nested agent tool's own scripts, flows and MCP resources were skipped. Recurse into it; a linked one is still queued as a resource instead. - Normalize a `$res:`-prefixed MCP tool resource_path like other refs. - Dropping provider/output_type from the schema's required list also let a standalone providerless agent validate, which deploys clean and then fails on every run. The constraint can't go in the schema: an `anyOf` makes AiAgent a union, which breaks the FlowModuleValue discriminated union it belongs to (verified: zod throws "Invalid discriminated union option"). Enforce it in validateFlowModules instead, next to the other cross-module checks, via a shared collectProviderlessAgentIds. - Correct the deploy paragraph in the docs: provider resources are traversed now. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix: follow linked tool_inputs overrides on deploy, untrack vitest artifact Round-20 review findings: - A linked step's `tool_inputs` override replaces the resource tool's default at runtime, so a static `$res:`/`$var:` override is the dependency the flow actually uses. The deploy walk queued only the saved agent, leaving runs in an empty target workspace to fail on the missing override target. It also never scanned an aiagent module's own input_transforms, since the scan was gated to script/rawscript/flow. - Extract the pure walkers to deployDependencies.ts and cover them: three rounds have each found a further gap in this one function. - Untrack a vitest cache artifact committed by accident, and ignore a repo-root node_modules/ (only per-package paths were listed). Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix: collect inline agent provider and tool deps, correct tool_inputs docs Round-21 review findings: - An inline agent's provider credential sits inside an object-valued static transform, so the top-level string check missed it and such a flow deployed without its provider. Walk transform values instead of string-matching them. - An inline agent's own tools were only partly reachable: getAllModules drops MCP and websearch tools, so their resources were never queued. A standalone agent module now recurses through agentResourceDependencies, and the module's own input_transforms are scanned inside aiAgentModuleDependencies so one function owns the whole step rather than splitting it with the caller. - `tool_inputs` was documented as empty/absent for non-linked steps, which contradicts the runtime applying it when `agent` is unset so a flow persisted mid-Edit keeps its bindings. Describe that case in both the Rust doc and the OpenFlow description. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix: keep linked steps brain-free on load, gate stale agent fetches, log linked tools Round-22 review findings: - loadSchemaFromModule filled every AI agent schema key with a placeholder transform, re-adding provider/memory to a linked step that deliberately carries none — persisted on the next save and rejected by the generated Copilot schema. Fill only the flow-local keys when the step is linked. - The linked-resource fetch was neither aborted nor tagged, so switching a step from agent A to B could publish A's tools under B and show A's brain next to B's link. Tag each result with the (workspace, path) it was fetched for and drop the ones that no longer match. - "Test this step" passed no tools for a linked agent, and the log viewer drops tool_call entries it cannot resolve to a definition, so the agent's invocations vanished from the log. Pass the resolved resource tools. - Correct the cancel-edit comment: the runtime does apply tool_inputs on an unlinked step, and folding is what leaves nothing for it to overlay. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix: pin the edit session across saves, resolve linked tools in the run viewer Round-23 review findings: - Cancel stays enabled while a save awaits its requests, and it keeps the `tools` array identity, so the old guard passed and the completing save relinked the step and cleared the edits Cancel had just kept. It also accepted any replacement edit marker. Pin the path being saved and require the marker to still hold it, which still tolerates a content-preserving refresh re-anchoring the marker onto a clone. - Resolve linked agents' tools in the run/status viewer too: it reads module.value.tools straight from raw_flow, which is empty for a linked step, so AIAgentLogViewer dropped every tool_call it could not match and the graph drew the agent with no tool nodes. Same gap the previous commit closed for "Test this step" only. - Drop the overlay call-site comment: it claimed resource defaults are discarded and unmatched keys ignored, while overlay_tool_inputs preserves defaults and inserts new keys, as its own test asserts. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix: scope linked tools without the trigger-node path, keep the standalone save guard Round-24 review findings, both regressions from the previous commit: - Passing `path` to the run viewer's graph also switched on its Trigger node (`triggerNode ? path : undefined`), which reads a TriggerContext that /run/[...run] does not provide — the page threw "Cannot read properties of undefined (reading 'triggersCount')". Give the graph a separate `linkedToolsPath` for the tools bucket so the two stay independent. - The rewritten save guard tracked only the edit path, so a plain "Save as agent" no longer noticed the step being replaced mid-request (undo, session sync): the replacement has no edit path either, so the stale completion relinked it and stripped its brain. Keep the array-identity check when there is no edit session, and use path re-anchoring only when there is one. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix: keep recorded tool calls in run history, send tool_inputs from step previews Round-25 review findings: - The agent log viewer dropped any recorded tool_call whose definition it could not find among the supplied tools, so renaming or removing a tool — or losing read access to a linked agent's resource — erased calls that had actually run. Render the recorded call labelled by its function name; its args, logs and result come from the child job, not the definition. - "Test this step" sent tool_inputs only for a linked step, but a step forked for editing has no `agent` while still carrying the flow's bindings, which the runtime overlays. The preview ran resource-authored defaults instead of the bindings under test. Send them from both branches. - Polling a running flow replaces `job` every tick, so the run viewer re-read every linked agent's resource each time. Key the fetch on the set of linked steps instead. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix: never discard edits made during a save, isolate the run viewer tools bucket Round-26 review findings: - The agent editor stays live while a save is in flight, so edits made after the snapshot were not in the resource yet linking stripped them from the step too, losing them outright. Compare the config against the snapshot on completion and, if it moved, leave the step alone and tell the user to save again. - The run viewer published into the editor's `${ws}:${flow path}` bucket, so opening an older run in the preview pane could flip the edited flow's tool nodes to that run's agent. Key it by job instead. - Drop the now-unreachable undefined filter in the agent log viewer. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix: claim the linked-tools generation on direct publishes and clears Round-27 review findings: - The step editor wrote resolved tools (and cleared them on unlink) straight into the store, leaving the fetch generation untouched. An older in-flight load for the previous agent then still passed its own check and overwrote them, so the graph and binding editor could show agent A while the step links to B. Claim the generation before those writes. - Correct two comments that still described unmatched tool calls as dropped; they are kept and labelled by their recorded name. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix: retain the loaded linked agent, rebuild run logs when tools resolve Round-28 review findings: - Rejecting a superseded resource response left the card with nothing: a late reply for a previous agent replaces `linkedResource.current` and no refetch follows, so the linked step lost its brain, tools and provider warning until remount. Retain the last response that matched the current link instead. - The agent log viewer built its module list on mount only, so a linked agent's asynchronously resolved tools never replaced the placeholders, and switching between completed runs reused the first snapshot. Rebuild on a value key — callers rebuild the agentJob object each render, so tracking its identity would reload in a loop. - Refresh a linked-tools scope's recency when it is read, not only when it is published: a run viewer opens one bucket per nested job, which could otherwise evict the bucket a still-displayed run is using. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix: supersede stale log reloads and stale tools on a link change Round-29 review findings, both on the reloads added last round: - Every prop change starts another loadToolCalls, and it awaits child-job requests before writing the shared view, so a slower reload for a previous run could restore its logs and tool states over the run now selected — or replace newly resolved definitions with an earlier empty-tools snapshot. Build the states locally and let only the newest load publish, including the parent's index-keyed job cache. - While a newly linked agent resolves, the previous agent's tools stayed in the store, so its bindings were editable against a step already linked elsewhere, and a failed load left them indefinitely. Clear them once the link moves away from what this component published; tools resolved at flow load are untouched, so selecting a step still doesn't flicker. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix: resolve a run's linked agents in the run's own workspace Round-30 review finding: the run viewer fetched linked agent resources with the navigation workspace, but session and fork previews render it with `workspaceId` pointing elsewhere. Those runs resolved nothing — or an unrelated resource sharing the path — losing tool nodes and log definitions. Prefer the explicit override, then the job's own workspace. The store scope stays keyed on `workspace` so it still matches what FlowGraphV2 reads; the job id in the key already makes the bucket unique. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix: refetch a run viewer's linked tools if its scope is evicted Round-31 review nit: the viewer publishes one scope per mounted nested job, hidden ones included, so a loop with many loaded iterations can push a displayed scope past the store's cap. Nothing refetched it afterwards — the set of linked steps had not changed — leaving the run without tool nodes or log definitions. Track the store and republish when the bucket is gone; publishing always writes a key, so this settles instead of looping. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix: retain in-use linked-tool scopes instead of refetching evicted ones Round-32 review findings. Republishing an evicted scope settles for one scope but not against the cap: with more than 32 mounted nested jobs holding linked agents, restoring one necessarily evicts another, and that mutation reran every viewer's effect — an endless round of resource requests. Hold a scope for as long as a viewer is mounted and skip retained scopes when evicting, so buckets in use are never dropped and nothing has to refetch. The cap yields to correctness when everything mounted is in use. Dropping the publish key also restores refetching when the fetch workspace changes for an otherwise unchanged job and link. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix: guard non-static brain edits during save, retain every displayed scope Round-33 review findings: - The in-flight edit guard compared the saved config, which holds only static brain values. A computed system prompt, memory or temperature changed while the save was awaiting the API therefore compared equal, and linking stripped it with no warning. Compare what linking actually discards — every brain transform and the tools — leaving the flow-local inputs free to change. - Retaining run-viewer scopes made them fill the cap, and eviction then picked any unretained scope, including the editor bucket a user is looking at, with nothing to refetch it. Retain the scope each graph draws from for as long as it is mounted, so every displayed bucket is protected. - A failed agent job has no parseable action list; the loader returned early and left the previously selected step's tool tree under the new header. Clear the view instead. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix: resolve only flow modules in viewer scans, prune scopes on release Round-34 review findings: - Both viewer scans used the default dfs, which descends into agent tools, and published each linked agent under its bare id. Tool ids imported from a resource are not flow-global, so a nested linked agent sharing an id with a top-level step superseded that step's fetch and showed its tools instead. Scan flow modules only — the graph resolves the store per module node. - Scopes skipped while retained were never reconsidered, so closing views left the store over its cap for the tab's life. Prune on release too. - Correct two comments that still argued the premises the retain mechanism and the read-recency policy replaced. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix: don't report success when a save left the step unlinked Round-35 review nits: - persist warns that changes made during the save are not in the resource and leaves the step alone, but both callers then toasted success unconditionally, burying the only actionable message. Report whether the step was linked. - Condense the tool_inputs invariant to the four-line limit. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix: seed the published link at mount, keep run history for toolless agents Round-36 review findings: - `publishedFor` started unset, but initFlowState has already published for the step's link by then. A link change landing before this component's own request therefore skipped the clear, leaving the previous agent's tools under the new link — indefinitely if the new one fails. Seed it from the link at mount. - A standalone agent that omits `tools` kept `undefined` here, and the gate downstream then hid the AI message and tool-call history behind the generic result view. Default to an empty list like the other consumers. - A save that lands after the step was replaced writes the resource but leaves the step alone; say so instead of closing the drawer with no outcome. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix: qualify nested agent tool store keys, keep an empty tools identity stable Round-37 review findings: - The step editor keyed the linked-tools store by the bare module id for nested agent tools too. Those ids come from a resource and are not flow-global, so a nested linked agent sharing an id with a top-level step read that step's tools — then overwrote them once its own fetch landed. Qualify the key by the parent agent, as the edit store already does; flow modules keep the bare id the graph looks up. - The `tools` binding handed the editor a fresh [] on every read when the module omits the field — a shape this PR made valid — so the save guard's identity check never matched and such a step could never link. Read through one shared empty array instead. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix: accept the first tool on an agent module that omits tools Round-38 review nit: the graph's tool insert required an existing `tools` array, so a module authored without the field — valid since `tools` became optional — swallowed the insert while still pushing history and dispatching a change. Create the array on first use. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix: don't evict a scope on the write that created it, and cover the store Round-39 review findings: - A rename removed the retained old key from the order but the new one is not retained until readers re-run, so eviction deleted the fresh bucket immediately. Reorder without evicting; the next publish or release enforces the cap, by which point the new key is held. - Writing the test for that surfaced the same shape in touchScope: it evicts right after appending, so once every older scope is retained the scope just published was the only eligible victim and was dropped at once. Exclude the scope being written. Add the store's first test: retention, eviction past the cap, pruning on release, and the rename handoff — four rounds landed fixes here with nothing pinning the behaviour. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix: re-resolve linked agents when a wholesale edit changes the links Round-40 review findings: - Undo/redo, YAML apply, AI apply and session restore swap a step's `agent` without re-running initFlowState, and the step editor only watches the step it is mounted on — so an unselected step kept showing, and binding against, the previous agent's tools. Re-resolve from the editor whenever the set of links changes. - Document that linked resolution is live rather than pinned: an edit landing mid-run affects steps that have not started, and a nested agent tool looks its definition up by id when its own job starts, so it can run a changed definition. Pinning would mean carrying the resolved definition into the child job instead of its id; inline agents are unaffected because their tools are snapshotted with the flow value. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix: per-module empty tools identity, invalidate tools when a link is replaced Both findings are over-corrections in the two preceding commits: - The shared empty-tools array made identity stable, but stable everywhere: a wholesale edit that keeps the module id reuses the component, so when both the old and the replacement module omit tools the save guard saw no change and could link and clear the replacement. Hand out one empty array per module value, which a replacement always renews. - The editor's link watcher resolved the replacement agent without dropping the previous one's tools first, so a step selected before the fetch landed still showed agent A under link B — and the freshly mounted editor seeds itself from B, so it could not tell. Clear the entry when the link for a module changes, seeding the map from the graph so the first run doesn't refetch what initFlowState just resolved. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix: reserve graph space for linked tools, re-resolve only changed links Round-41 review nits: - The layout reservation read the module's own `tools`, which is empty for a linked agent, so its display-only tool nodes were drawn over the node above in read-only viewers. Count the resolved tools for a linked step. - The editor's link watcher refetched every linked agent on each run. Resolve only modules whose link actually changed, and skip the pass entirely on a rename, where the scope sweep has already carried the buckets over. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix: protect a renamed scope until it is retained, drop the phantom tool row Round-42 review nits: - Readers release the old scope before retaining the new one, so a migrated bucket is unretained in between and, over the cap with everything else held, was the only thing eviction could take. Protect a just-migrated scope until a reader retains it, and cover that release/retain order in the store test. - The layout reserved an add-tool row for linked agents, which have no add-tool node, leaving dead vertical space. Match computeAIToolNodes. - Re-resolving links no longer short-circuits on a rename: comparing each module still costs nothing when only the path changed, and a restore that renames and relinks in one tick now gets both. - Hoist the duplicated linked-tools lookup in the graph's store update. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix: kill a scope's in-flight fetches before migrating it Round-43 review finding: fetch generations are keyed by (scope, module), so a resolution still running against the pre-rename scope keeps a valid generation there. It publishes into the old bucket after the rename, and the doc-scope sweep — which gives the source precedence — carries it forward over a link resolved since under the new scope, leaving the graph and binding editor on the previous agent's tool ids with nothing to refetch them. Invalidate the source scope's fetches before each migration, and pin the behaviour: the new test fails without the invalidation. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix: re-resolve links a scope sweep cancelled, and only sweep a real bucket Round-44 review findings, both on the previous commit: - Invalidating the source scope killed fetches that were perfectly current — a link still loading when the rename landed — and nothing restarted them, because the watcher already records that link. Resolve again, in the destination, every link the migration left without tools. - The doc-scope sweep ran on every store version bump, so during a draft refresh the first completed fetch cancelled the others mid-flight. Skip the sweep entirely when the source scope holds nothing. - Condense a six-line invariant to the four-line limit. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix: split rename from doc sweep, hide brain fields of nested linked agents Round-45 review findings: - Two reviewers disagreed about invalidating a scope whose bucket is empty, because the two callers differ. A rename is a cut-off: every fetch still running against the old scope is stale whether or not anything resolved there, so it always invalidates. The doc-scope sweep has no cut-off — those fetches belong to the refresh in progress — so it still waits until that scope holds something. - Recording the swept links as published undid the rename+relink fix: a restore that renames and swaps a link in one tick would keep the previous agent's tools with nothing to refetch them. Leave that comparison to the watcher, which compares links rather than presence. - A nested agent that is itself linked was offered the whole agent schema in the tool bindings, but the runtime overlays only its flow-local inputs, so the rest were collected and dropped. Show what actually applies. - Condense the hybrid-linking comment to the constraint. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix: don't resolve a shared agent's tool defaults when loading it Round-46 review finding: the whole agent resource was interpolated before tool_inputs was overlaid, so each tool's default `$res:`/`$var:` resolved first. A host flow overriding a default that points at the author's resource still had to resolve that resource, and an unused tool whose default is unreadable in the consumer's permission context failed the agent outright — defeating the point of sharing an agent across contexts. Read the resource raw, overlay the host's overrides, and interpolate only the brain; each tool resolves its effective inputs when it executes. The nested tool lookup reads raw too, since it only needs definitions. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix: interpolate the brain before overlaying caller inputs Round-47 review findings, all on the previous commit: - user_message and user_attachments were inserted before interpolation, so they went through it a second time: a user message of `$WM_TOKEN` expanded to the job token and was sent to the model provider. Interpolate the resource first, then overlay the already-resolved flow-local inputs. - The relink watcher skips tool nodes, so a linked agent nested as a tool kept the previous agent's entry through undo, YAML/AI apply or a session restore, and the step editor seeds itself from the new link and cannot tell. Emit the ancestry-qualified key for those too. - Correct the guide, which still named the interpolation path this branch replaced, and condense two invariants to the four-line limit. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix: deploy $jsonvar deps, key run logs by tool identity, seed only top links Round-48 review findings: - The deploy walkers recognised `$res:` and `$var:` but not `$jsonvar:`, which the worker resolves too, so a secret referenced that way by an agent brain, a saved tool default or a host override never reached the target workspace. - The run log rebuilt only when a tool's name or the tool count changed, so a refreshed resource that altered a tool's path, code or id behind the same name kept showing the old definition. Key on the array identity instead: the store swaps it exactly when the contents differ. - Nested linked agents were seeded as already published, but initFlowState resolves only top-level links, so their tools never loaded until their editor was opened. Seed what initFlowState actually publishes. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix: let the watcher's fetch survive the step editor's stale-clear Round-49 review nits: - On a relink the step editor claimed the fetch generation before clearing the previous agent's tools, which discarded the watcher's already-running fetch for the new link. The tool nodes then only appeared if the step stayed selected until the editor's own refetch landed. Clear without claiming: the watcher superseded the old fetch when the link changed, so nothing stale can return. Unlink still claims, since no watcher fetch covers it. - Condense the store's opening invariant to the four-line limit. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * docs: condense the stale-clear invariant Round-50 review nit. Also records why the branch deliberately doesn't claim a fetch generation: a reviewer asked for the opposite this round, but writing `agent` re-runs the editor's watcher, which supersedes the old fetch and starts one for the new link — claiming here would discard it. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix: guard Edit/Unlink by step identity, not just the link path Round-51 review finding: forkFromResource compared only the agent path after its fetch, so a module replaced mid-request while keeping the same link passed the check — the stale continuation then wrote the fetched brain and tools into the replacement and unlinked it. Compare the step's own `tools` array too, which is one instance per module value and so identifies the step. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix: report an Edit or Unlink abandoned because the step changed Round-52 non-blocking note: forkFromResource returns undefined when the step was replaced mid-request, and both callers treated that as do-nothing, so the click looked ignored. Say what happened, as the save path already does. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Co-authored-by: hugocasa <hugo@casademont.ch> Co-authored-by: Claude Opus 4.8 (1M context) <noreply@aanthropic.com>	2026-07-28 01:38:16 +02:00
Ruben Fiszel	4d3ff0299f	feat: mark failed jobs as resolved so handled failures stop showing red (#10319) ... * feat: mark failed jobs as resolved so handled failures stop showing red Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix: constrain auto-resolve to the proven retry chain and honor resolved filter everywhere Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix: apply resolved filter to queue-union, concurrency and delete paths, bound note Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix: sweep resolutions on workspace delete, verify helper args, enforce UI limits Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix: count resolution note in characters on both sides of the API Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix: skip the queue lookup for cancel-all under the resolved-only filter Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix: converge retry auto-resolution from either commit order, keep notes on re-resolve Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * docs: correct the idempotency claim on the retry auto-resolve sweep Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * feat: gate resolution notes and attribution behind enterprise, add note popover Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix: hide resolution from operators, exclude flow steps, enforce EE licence at runtime Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * docs: add job_resolution.automatic to the summarized schema Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix: preserve stored attribution when re-resolving without a valid licence Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * docs: condense the attribution-preservation comment to four lines Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix: validate resolution notes by code point instead of a UTF-16 maxlength Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix: keep the resolution popover open when a note is rejected Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * feat: offer to resolve the original failure after a successful re-run Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix: verify supersession server-side and stop re-runs overwriting notes Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix: apply tag scope to the superseding run Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix: exclude obscured cross-workspace runs from resolution actions Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>	2026-07-26 09:36:27 +02:00
Ruben Fiszel	f02df7fc45	feat(monitor): make between-steps zombie flows hand-recoverable (#10287) ... * feat(monitor): make between-steps zombie flows hand-recoverable When a worker is OOM-killed mid state-transition, the flow is reaped as a between-steps zombie (children all success, module still InProgress). We do not auto-recover (a re-driven transition can OOM again), so instead: - Append actionable recovery guidance to the cancellation reason when the reaped step's state is derivable (every child a success completion): which step, iterations completed, raise memory then restart-from-step (UI + API). - Restart-from-step now reuses a zombie step verbatim (InProgress with all children successful) and restarts from the next step, so no completed child re-runs; downstream steps re-derive its result from flow_jobs on demand. - Cast flow_status ::text in the reaper query: reading the jsonb column as Box<str> included the binary version byte and silently failed FlowStatus parsing (disabling the restart-not-yet-started branch since the v2 migration). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(monitor): only reuse a between-steps zombie step that provably finished Address review findings on the zombie-restart reuse path: - Require structural completeness (FlowStatusModule::is_between_steps_complete): a serial for-loop / branch-all reaped mid-fan-out has an all-success prefix but unrun remaining iterations, so the cursor must sit on the last element; while-loops are never derivable (continuation is a post-iteration condition). Parallel containers preallocate all children, so success alone is conclusive. Shared by the monitor guidance and the restart resolution. - Decline reuse when the step carries stop_after_if / stop_after_all_iters_if: those predicates decide whether downstream steps run, and reuse would bypass them; such a step re-runs instead. - Decline reuse when the zombie step is the last module (advancing past it lands on the failure step); it falls back to the existing re-run path. - Unit tests for is_between_steps_complete and an integration test asserting a mid-iteration serial-loop zombie is re-run, not reused. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(monitor): align zombie recovery guidance with restart eligibility Address CI review findings: - Exclude skip_if / suspend / sleep (not just stop predicates) from reuse via FlowModule::allows_zombie_reuse, so a skipped/suspend-armed step is never synthesized as Success (which would strand a restart waiting on an approval it never armed). - The reaper does not load the flow definition, so it cannot know whether restart will reuse or re-run a given step; reword the guidance to state both outcomes (reuse where derivable, re-run for the flow's last step or one carrying a stop/skip condition, approval, or sleep) instead of promising "no re-run". - Make the mid-iteration regression test exercise the cursor-completeness guard: a downstream step makes the loop non-final, so reuse is prevented only by the guard; a truncated loop result would then fail the assertion. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(monitor): never let zombie reuse swallow a nested restart request A nested restart (RestartedFrom.nested) descends into the restart step's child to re-run an inner step. For an eligible zombie BranchOne/Subflow the outer branch_or_iteration_n is None, so reuse fired, skipped the container, and the explicitly requested inner step never re-ran. Thread the presence of a nested chain into restarted_flows_resolution and decline reuse when set. Regression test added (RED without the guard: the nested target is reused instead of re-run). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(monitor): don't auto-requeue preprocessor zombies as unstarted flows The ::text parse fix re-activated the "hasn't started yet, restart it" branch, but its `modules[0] == WaitingForPriorSteps` check also matches a flow whose preprocessor is still InProgress (step == -1, first module waiting). Requeuing such a flow re-runs the preprocessor, duplicating side effects / repeating the OOM. Gate the branch on FlowStatus::is_not_yet_started, which also requires the preprocessor (if any) to be WaitingForPriorSteps. Unit-tested. Also drop the numbered procedural narration from the happy-path test comments. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(monitor): only emit restart guidance for restartable (deployed, top-level) flows The recovery guidance points operators at the run page's "Re-start from" button and the restart API, but both require a top-level deployed flow: a preview has no flow path (the button is hidden, the API 400s) and a subflow child restarts via its root, not itself. Gate the guidance on runnable_path IS NOT NULL AND parent_job IS NULL so previews/subflows keep the existing wording instead of being told to use a button/endpoint that isn't there. Verified end-to-end: a reaped preview gets no RECOVERY block, a reaped deployed flow does. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(monitor): gate recovery guidance on kind='flow' to match the restart surface Addresses review nit: a pathful editor preview (kind='flowpreview' with a runnable_path) satisfied the previous runnable_path check but the run page only renders the "Re-start from" button for kind='flow'. Match that condition exactly so previews/singlestepflow keep the plain wording. Verified end-to-end: a reaped pathful preview now gets no RECOVERY block. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(monitor): disable zombie reuse for raw-flow (editor preview) restarts A JobPayload::RawFlow restart queues the request's current, possibly EDITED, definition, but restarted_flows_resolution validates reuse against the completed job's STORED definition. For an eligible preview zombie, editing the restart step and restarting from it would synthesize Success from the old children and skip the edit. Thread allow_zombie_reuse into the resolver (true only for JobPayload::RestartedFlow, which queues the stored definition) and decline reuse for raw-flow restarts. Regression test added (RED without the guard: the edited step is skipped and the old result is reused). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * chore(sqlx): add offline cache for zombie_flow_recovery test queries The integration test's UPDATE v2_job_completed queries had no .sqlx entry, so the CI SQLX_OFFLINE build of the test failed to compile. Regenerated with --all-targets --features deno_core,quickjs to capture the test-target queries. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * test(monitor): drop procedural narration from the raw-flow zombie test Per AGENTS.md (comments record constraints, not narration): remove the two step-describing comments the reviewer flagged; the test doc comment already carries the durable rationale. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(monitor): restrict zombie reuse to monitor-reaped flows The reuse predicate matched the InProgress/all-children-success shape without checking provenance, so an ordinary force-cancel at the same boundary (a child succeeded before its parent transition landed) would also be reused, dropping the usual restart-from-step re-run. Gate reuse on canceled_by = 'monitor' (the username the zombie reaper cancels with). Regression test added (RED without the guard: a user-cancelled flow reuses the child instead of re-running it). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(monitor): reuse zombie step on Some(0) too, so the run-page button works The run page's "Re-start from" button always sends branch_or_iteration_n = 0 (never omits it), but reuse only fired for None, so the exact UI path the recovery message points to would re-run the children instead of reusing them. Treat a whole-step restart (None or Some(0)) as reuse-eligible; Some(n>=1) keeps the explicit partial-container restart. Verified against the live EE restart API with branch_or_iteration_n=0: all loop-iteration child UUIDs are reused. Happy- path test now sends Some(0) to match the button. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>	2026-07-23 19:09:28 +02:00
Ruben Fiszel	8eb36ce008	fix: treat concurrent_limit/timeout <= 0 as unset instead of a zero cap (#10288) ... * fix: treat concurrent_limit/timeout <= 0 as unset instead of a zero cap Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix: flow-step timeout <= 0 inherits the script timeout, not the global default Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>	2026-07-23 19:05:09 +02:00
Ruben Fiszel	68debab877	feat(triggers): add AMQP (RabbitMQ) trigger via lapin (#10230) ... * feat(triggers): add AMQP (RabbitMQ) trigger using the lapin library Fixes WIN-2214 Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * chore(triggers): defer AMQP cross-workspace deploy pending utils-internal publish Revert the amqp_trigger additions to the shared windmill-utils-internal TriggerDeployKind and the frontend cross-workspace deploy adapter: the frontend installs the published npm package, which lacks the new kind until a release is cut. AMQP create/edit/delete/list/sync/capture are unaffected (they use local types); only cross-workspace deploy/merge of AMQP triggers waits on the package bump. Also document the at-most-once ack in the consumer loop. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(triggers): address AMQP review — at-least-once ack, workspace cascade, contracts - ack AMQP deliveries only after successful dispatch; nack+requeue on failure - add ON DELETE CASCADE workspace FK so amqp_trigger rows are cleaned on workspace deletion (and the listener stops) - fix the /amqp_triggers/test OpenAPI body and add amqp_trigger to WorkspaceDiffRow.kind - register AMQP in the generated workspace trigger tool (create_trigger) - drop banned $bindable defaults on optional props in the config section - add build_uri unit tests (encoding, ports, vhost) Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(triggers): stop AMQP poison-message loop and reconnect on transient drops Chaos testing against a live RabbitMQ broker showed the previous nack(requeue) + immediate re-poll spun a tight redelivery loop (~1000 critical-error reports/sec) on a poison message, and any connection blip permanently disabled the trigger (lapin has no built-in reconnect). - on dispatch failure: nack+requeue then stop consuming; the listener framework re-lists the trigger after its ping goes stale (~15s), backing redelivery off to that cadence instead of a tight loop (verified: rate dropped from ~1000/s to ~1 per ~26s, message preserved) - on connection/stream error: stop and let the framework reconnect instead of disabling; persistent failures are still disabled via get_consumer (verified: a forced connection close now auto-reconnects and resumes) - finish the AI create-trigger action wiring for AMQP: add amqp to CreatedResourceTriggerKind, the action-card registry, and the drawer registry so the result card renders and its "Open" action works Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(triggers): complete AMQP frontend registries and defer merge rows - add amqp to capturableTriggerTypes (so AmqpCapture mounts), the Runs jobTriggerKinds filter, and CLOUD_DISABLED_TRIGGER_TYPES - wire AMQP into global AI chat mode: TRIGGER_KINDS, the request union, writeTriggerSchema, triggerServices, and the draft adapter - stop emitting actionable AMQP fork-comparison rows (revert amqp_trigger from TRIGGER_OR_SCHEDULE_TABLES) since cross-workspace deploy is deferred until windmill-utils-internal is published — avoids a deploy that fails with "Unknown kind: amqp_trigger" - use design-system TextInput instead of raw <input> in the config section Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(triggers): complete AMQP session/draft registries and constrain prefetch - add amqp to the session-deploy, draft-compare, preview-router, and copilot workspace-item registries so AMQP drafts/deploys/nav/path resolution work - include amqp_count in the MoveDrawer attached-trigger rename warning - replace the raw prefetch <input> with a design-system TextInput bounded to an integer 1-65535 (backend u16) and block save on invalid values Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(triggers): make AMQP disconnect/reconnect consistent with the Kafka trigger lapin, like rdkafka, has no transparent reconnect, so the AMQP listener now mirrors the Kafka trigger's explicit reconnect loop instead of relying on the framework re-list (which disabled the trigger once get_consumer failed on a sustained outage): - get_consumer returns cheaply; consume owns a (re)connect loop that retries with a 30s backoff, reports a critical error every 10 failed attempts, and reports a recovered critical error once it reconnects — never disabling the trigger on a connectivity failure - a consumer/stream error breaks out to reconnect rather than disabling - dispatch failure still nacks+requeues (at-least-once) with a short backoff to avoid a tight poison-message loop, keeping the connection alive Verified against a live RabbitMQ broker: killing the broker keeps the trigger enabled and retrying (attempt N), and restarting it auto-reconnects (logs "reconnected after N attempts") and resumes dispatch. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(triggers): complete AMQP capture registries and constrain prefetch contract - add the 'amqp' case to triggerKindToTriggerType so opening the AMQP editor from a capture button no longer throws "Unknown TriggerKind: amqp" - register AmqpIcon in CaptureTable's icon map and add an AMQP entry to the script/flow CaptureButton menu - bound the OpenAPI prefetch_count to an integer 1-65535 (matches the Rust u16) and regenerate clients/prompts - require a non-empty exchange name when the exchange binding is enabled - build_uri: fall back to "/" on a blank vhost and bracket IPv6 hosts (+ tests) Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(triggers): wire AMQP into pipeline graph, git-sync, and preprocessor types - asset_graph: discover attached amqp_trigger rows and emit an AMQP TriggerEdge so AMQP triggers render (and can be opened/deleted) on the data-pipeline canvas - frontend pipeline graph: add amqp to NativeTriggerKind, the add-trigger menu, node presentation, event-trigger set, annotation keywords, and the editor/service registrations - git-sync: add the amqp_trigger include pattern (+ test) so an AMQP git-sync deployment stages only its .amqp_trigger.* file, not an unrelated same-path object - preprocessor starters: add the AMQP event to the generated TS/Python/PHP trigger event types (kind/payload/exchange/routing_key/queue_name/redelivered/ delivery_tag) Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(triggers): finish AMQP pipeline/parser wiring, prefetch validation, source lists - fix a stray edit that corrupted the pre-existing MqttTriggerEditor import ($lib/... path) in PipelineTriggerEditors.svelte - reject prefetch_count = 0 server-side in validate_config (RabbitMQ treats 0 as unlimited) and defensively skip basic_qos(0) in build_consumer (covers the capture path that bypasses CRUD validation) - recognize `// on amqp` in the canonical parser (TriggerSpec::Amqp) and add amqp to the CLI non-autorun/event-trigger sets so a pipeline cascade never runs an AMQP-only node as a manual root without an event - add amqp to the preprocessor intro lists and both pipeline AI instructions Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(triggers): reject zero AMQP prefetch in all paths and finish guidance lists - extract a shared validate_amqp_options used by both CRUD validate_config and build_consumer, so capture configs (which bypass CRUD validation) also reject prefetch 0 instead of silently connecting with an unlimited buffer (+ unit tests for 0/1/65535/None) - add AMQP to the main script-writing preprocessor-sources prompt and the CLI triggers-skill guidance list Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * docs(triggers): de-duplicate AMQP prefetch comment and fix GET response text - keep the zero-prefetch rationale only on the shared validate_amqp_options doc; drop the redundant call-site comments - correct the getAmqpTrigger OpenAPI 200 description ("deleted" -> "retrieved") Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * chore: update ee-repo-ref to aaa6cb89b05b76139252c64f057e53b94d12ac60 This commit updates the EE repository reference after PR #680 was merged in windmill-ee-private. Previous ee-repo-ref: 5da5fd65aca9594b2611837a52e4677b544b0380 New ee-repo-ref: aaa6cb89b05b76139252c64f057e53b94d12ac60 Automated by sync-ee-ref workflow. * chore(migrations): consolidate the four AMQP migrations into one The table and the three enum ADD VALUE statements (trigger_kind, job_trigger_kind, draft_kind) are one atomic feature. ALTER TYPE ... ADD VALUE runs inside the migration transaction on PG >= 14 (Windmill's minimum) since the amqp_trigger table doesn't reference those enum types, so they can share a single migration instead of four. Verified applying cleanly in a single transaction on a fresh DB. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Co-authored-by: windmill-internal-app[bot] <windmill-internal-app[bot]@users.noreply.github.com>	2026-07-21 15:10:29 +00:00
Ruben Fiszel	7d2c5ceb0f	fix(flows): make updateFlow body path optional so AI can update flows (#10176) ... * fix(mcp): default a body field to its same-named path param so updateFlow works Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * docs: trim mcp path-param fallback helper comment Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * refactor(mcp): keep path params un-mangled so update tools take plain `path` Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * fix(flows): default update_flow body path from URL via EditFlow Harmonizes updateFlow with the EditVariable/EditResource/EditApp convention: the flow to update is identified by the URL, so the body path is optional and only needed to rename. Fixes the 422 at the API layer for every client (MCP, the in-app AI chat, raw HTTP), not just the MCP tool schema. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * refactor(mcp): drop redundant body-path fallback now that the server defaults it Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * docs: fix stale generator comment after removing mcp body-path fallback Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * fix(flows): mark updateFlow body path optional in the openapi contract Adds an `EditFlow` schema (path optional) for the update route so the public contract matches the server; createFlow keeps `OpenFlowWPath` (path required). Also trims two test comments to record constraints rather than history. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>	2026-07-17 19:22:31 +02:00
hugocasa	7ebfad382a	feat(ai-agent): give tools a real description instead of the tool name (#10083) ... * feat(ai-agent): use a real tool description instead of the tool name Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(ai-agent): render tool-name error full width and hoist it above the description Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(ai-agent): make tool description field hug its content so a single line is vertically centered Add an optional minHeight param to the autosize action (default unchanged at 30px) and pass minHeight 0 for the tool description so an empty/one-line field no longer reserves the 30px floor and leaves dead space below the text. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * chore(ai-agent): regenerate OpenFlow-derived prompts, CLI guidance, and copilot zod schema for tool description Fixes the check-freshness CI failure (system_prompts + skills.gen.ts) and makes the flow copilot's openFlow.json / openFlowZod.gen.ts aware of the new AgentTool.description field so AI-authored tools can set it. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>	2026-07-14 22:33:16 +02:00
Ruben Fiszel	710a13a59d	fix(apps): cover script/flow component outputs in deployed-app S3 provenance gate (#10070) ... * fix(apps): cover script/flow component outputs in deployed-app S3 provenance gate Deployed apps read S3 files on-behalf of the app author for logged-in viewers (#10048). A confused-deputy guard confines those reads to files the app "produced", but the recent-production check only matched inline `appscript`/ `preview` jobs nested under the app path. Files produced by the deployed script/flow components an app is wired to run (e.g. a SQL query persisted to S3) were therefore denied "File restricted" for every viewer, admins included. Expand the provenance check to also match completed `script`/`flow`/`flowscript`/ `flownode` jobs whose `runnable_path` is one of the app's declared triggerables, and accept the author identity via `permissioned_as = on_behalf_of` (not only `created_by = caller`) so files produced on-behalf of the author are covered. Reads outside the app's declared triggerables stay denied. Adds a regression test seeding a script-kind produced file that reproduces the "File restricted" denial before the fix and passes after. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(apps): key S3 provenance on on-behalf identity + cover flow steps (review) Addresses the CI review on the S3 provenance gate: - P1 (confused deputy): the recent-production check keyed on `created_by = caller`, so a viewer who can run a declared script/flow directly (outside the app, with un-pinned inputs) could craft a result naming an author-only key and read it back through the app as the author. Key provenance instead on the producing job's `permissioned_as` matching the on-behalf identity the download reads as (the author in author-mode); a viewer's direct run has `permissioned_as = viewer` and no longer clears the gate. Drops `created_by` from both the appscript/preview and script/flow branches, closing the same latent hole in the pre-existing inline-script branch. - P2 (dead flow-step branch): `flowscript`/`flownode` jobs have `runnable_path = <flow_path>/<step_id>`, which exact `= ANY(...)` never matched. Split script vs flow triggerable paths; flow kinds now match the flow's own job (bare path) and its step jobs via a `<flow_path>/%` prefix, bounded to declared flows. - P2 (test realism): the regression test now uses the production component-prefixed triggerable key format (`<id>:script/...`), exercises a flow-step-produced key, and asserts a viewer's own direct run of a declared script stays denied (the P1 case). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(apps): tie deployed-app S3 provenance to an app-origination marker (review) Second CI-review round flagged that `permissioned_as` still does not prove a job was app-launched: a runnable configured with its own `on_behalf_of` makes a direct `/jobs/run` resolve `permissioned_as` to that identity (the app author), so a viewer with run access could execute a declared runnable directly, craft an S3 result, and read it back through the app. The flow-path `LIKE fp || '/%'` match also let `_`/`%` in a declared path admit unrelated flows. Introduce a real app-origination marker instead of inferring provenance: - Add `JobTriggerKind::App`; `execute_component` stamps every app-launched job with `trigger_kind = 'app'` + `trigger = <app path>`. A direct `/jobs/run` cannot set this, so it is the authoritative signal that a file was produced *by the app*. - The provenance gate's recent-production check collapses to `trigger_kind = 'app' AND trigger = <this app path>` (+ the 3h window and result containment). This drops the forgeable `created_by`/`permissioned_as`/ `runnable_path`/kind logic entirely and removes the `LIKE` wildcard issue. - Provenance is scoped to THIS app's path, so another app's jobs (even same author) do not authorize this app's reads. Regression test rewritten to the marker model: an app-produced key clears for viewer and admin; a direct run whose `permissioned_as` resolves to the author stays denied (the forgery); another app's output stays denied. Adds `app` to the OpenAPI JobTriggerKind enum. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * test(apps): assert execute_component stamps trigger_kind='app' at runtime Adds an end-to-end test that runs a real script component through the app runtime (`apps_u/execute_component`) and asserts the enqueued job carries the app-origination marker `trigger_kind = 'app'` + `trigger = <app path>` (not the runnable path). The provenance-gate tests seed the marker directly; this proves the runtime actually produces the exact marker the gate depends on. execute_component commits the job row and returns its id, so the assertion reads the row directly — no worker needed to run the job. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(triggers): reject trigger_kind=app for suspended-job reassignment (review) `JobTriggerKind::App` (added for the app-origination S3 marker) became a valid value for the resume/cancel suspended-trigger routes, whose handler derives the table name `<kind>_trigger`. There is no `app_trigger` table, so both endpoints would fail with a missing-relation database error (500). Reject `App` in `get_suspended_trigger` alongside webhook/schedule so it returns a clean 400. Adds a regression test asserting the reassignment route returns 400 (not 500) for trigger_kind=app. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(apps): don't stamp app-origination marker on preview runs (review) The app-origination marker (trigger_kind='app') was stamped unconditionally, including preview mode. A preview lets a `jobs:run` caller supply arbitrary `raw_code` against ANY app path without that app's deployed policy (raw_code with no path/id skips all app authorization), so a preview returning `{"s3":"<author-only-key>"}` would forge the exact marker the S3 provenance gate trusts and read the victim app author's file. Gate the marker on `!is_preview`: only deployed, policy-checked executions are app-provenanced. Preview/editor S3 display does not rely on this marker (the editor routes reads through the force_viewer allowlist), so nothing legitimate regresses. Adds a regression test asserting a preview run's job is not stamped trigger_kind='app'. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(apps): editor-authorize preview marker + per-viewer S3 provenance isolation (review) Closes the codex P1 (preview forgery) without breaking editor preview downloads, and adds cross-viewer isolation to the provenance gate. - Preview marker now requires app write: `execute_component` stamps the app-origination marker on a preview only when the caller can EDIT that app (`require_is_writer`), instead of never stamping previews. An app editor already wields the app's author identity (they can deploy a component that reads the same file), so marking their own preview is no escalation and keeps preview-produced S3 results downloadable in the editor; a `jobs:run`-only caller who cannot edit the app still cannot forge the marker. Deployed runs are unchanged (always marked). - Per-viewer isolation: the provenance gate now also requires `j.created_by = <this caller>`. The security boundary stays the un-forgeable `trigger_kind='app'` marker; `created_by` is an additional filter ANDed under it, so it only narrows — a viewer can only download keys their OWN app runs produced, not another viewer's result. Restores the per-caller scoping #10048 had, now safe on top of the marker. Tests: preview marked iff caller can edit the app; cross-viewer isolation (another viewer's app-marked key denied, no admin bypass); direct-run and other-app keys still denied; deployed run still stamped. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(apps): require apps:write scope (not just writer ACL) to mark preview provenance (review) require_is_writer checks the user's underlying ACL but ignores token scopes, so a writer's token deliberately scoped to apps:run/apps:read/jobs:run but WITHOUT apps:write could still mark a preview and forge provenance — even though that token cannot deploy the app (update_app requires apps:write), breaking the "any marked caller can deploy equivalent code" rationale. Require BOTH apps:write:<path> scope (check_scopes) AND the writer ACL (require_is_writer) before stamping a preview's app-origination marker. Deployed runs unchanged. Adds a scope-restricted-writer token to the test (apps:run/read + jobs:run, no apps:write) and asserts its preview stays unmarked; retains the full-editor positive case and the non-editor negative case. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(apps): never app-provenance preview runs; read editor S3 as the caller (review) Simplifies the preview handling: a preview executes as the *caller* (Viewer mode), never as the author, so its results must be read back as the caller — never author-mode — and must never carry the app-origination marker. This removes the whole `require_is_writer` / `apps:write` / `can_preserve_on_behalf_of` reasoning (which was also unsound: a writer's token or session may not be able to deploy a component running as the app's on-behalf identity, so marking their preview could still escalate). - Backend: mark the app-origination marker for deployed runs only (`!is_preview`). - Frontend: `getS3File` (AppImage/AppPdf/AppDownload) now routes editor/preview reads through the viewer-scoped `job_helpers/download_s3_file` endpoint (reads as the caller), matching what DisplayResult/ParqetCsvTableRenderer already do; only a deployed app view uses the provenance-gated `apps_u` endpoint. This is the path that previously relied on marking previews, so nothing regresses. Test: a preview is never app-provenanced (owner's own preview and a non-editor's both stay unmarked). Cross-viewer isolation, deployed marking, and the reassignment guard are unchanged. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(apps): app components run on-behalf of the app, not the referenced runnable (review) Root-causes codex's on-behalf-preview finding: `execute_component` was overriding the app's resolved on-behalf identity with the referenced script/flow's OWN `on_behalf_of` (its `on_behalf_of_email`). That is wrong in the app context — the app's execution mode should govern: - A Viewer-mode app could execute a component AS the referenced runnable's on_behalf identity (privilege confusion / escalation), instead of as the viewer. - A preview would run as that identity rather than as the caller, so its S3 output could not be read back as the caller — the download-identity mismatch codex flagged. Always use the app-resolved identity (author in author-mode, caller in viewer/preview); a referenced runnable's own `on_behalf_of` no longer leaks into app execution. Direct `/jobs/run` still honors a runnable's `on_behalf_of` (unchanged). With this, previews always run as the caller, so reading editor/preview S3 as the caller (viewer-scoped `job_helpers`) is unconditionally correct. - Test: the deployed-component e2e now seeds the script with a distinct on_behalf and asserts the component job's `permissioned_as` is the app identity, not the script's. - Also reword the getS3File `configuration` param comment to describe current state only (AGENTS.md comment rule). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * chore(apps): surface 'app' trigger kind in Runs UI; condense provenance comments (review) Addresses codex review nits: - Add `app` to `jobTriggerKinds`, `triggerIconMap` (LayoutDashboard), and `triggerDisplayNamesMap` so app-component jobs (which now carry `trigger_kind = 'app'`) are filterable in Runs and render their trigger info. - Condense the app-origination marker, on-behalf-identity, and provenance-gate comments to state each invariant once in <=4 lines at its relevant site (AGENTS.md comment rule). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>	2026-07-14 10:29:48 +02:00
Ruben Fiszel	5d7fb6deca	feat(pipelines): asset freshness — fresh/stale badge (CE) + watchdog (EE) (#9909) ... * feat(pipelines): passive asset freshness tracking on the graph Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * chore(pipelines): drop dead freshness-enforcement stub, document query ordering Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * feat(pipelines): freshness watchdog (EE) — auto re-run stale producers Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(pipelines): watchdog review fixes — archived workspaces, badge kind parity, scan index Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(pipelines): CI review — no singlestepflow in freshness, +N parity, completion-time fallback Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(pipelines): CI review — history completedAt, freshness/asset trigger UI metadata Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * chore: update ee-repo-ref to 6f5fe0f7f56696fbef5a8349da38496c32e71666 This commit updates the EE repository reference after PR #643 was merged in windmill-ee-private. Previous ee-repo-ref: 1f13380354bf591ae25a2c20d36917534bcc5459 New ee-repo-ref: 6f5fe0f7f56696fbef5a8349da38496c32e71666 Automated by sync-ee-ref workflow. --------- Co-authored-by: Claude Fable 5 <noreply@anthropic.com> Co-authored-by: windmill-internal-app[bot] <windmill-internal-app[bot]@users.noreply.github.com>	2026-07-04 06:23:44 +02:00
Ruben Fiszel	3cda447621	fix(flows): reject corrupt step paths at deploy + atomic cache writes (#9751) (#9813) ... * fix(flows): reject corrupt step paths at deploy + atomic cache writes (#9751) A flow step could execute an unrelated (and in the reported case, destructive) script at runtime even though every stored definition looked correct. A forensic dump traced it to two issues: - Deploy accepted absolute/local step paths. `wmill sync push` from a feature- branch checkout under /tmp baked an absolute path (`/tmp/.../ops/scripts/clean_device/...`) into a step's `value.path`. Persisted verbatim, it mis-resolved to an unrelated script at runtime. - The on-disk cache write was neither truncating nor atomic. `FsBackedCache::put` used `write+create`, so a shorter overwrite left stale trailing bytes and concurrent writers could interleave into a torn file — a corrupt cached blob that a worker then scheduled from. Fixes: - Reject non-workspace flow step paths (must be u/, f/, g/ or hub/) in `validate_flow_value` (covers create_flow + update_flow, recursively through loops/branches/AI-agent tools) and early in the CLI `pushFlow`. - Make `FsBackedCache::put` write a unique temp file (truncate + fsync) then atomically rename it over the target, cleaning up on error. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * fix(flows): validate failure/preprocessor module paths + sub-flow paths in CLI Address PR review (cubic + claude): - Backend `validate_flow_value` is the authoritative guard but only walked `modules`; extend it to also validate `failure_module` and `preprocessor_module` (which can themselves be sub-flows/loops/branches), so an absolute path there can't be persisted. - CLI preflight only collected `type: "script"` paths; now collects sub-flow (`type: "flow"`) step paths too (recursively, incl. failure/preprocessor), so the comment's claim matches the behavior. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * fix(cli): include AI-agent tool step paths in flow path preflight Address Codex review: collectStepPaths skipped aiagent tools, so a bad path in a tool fell through to the API error instead of the local fail-fast. The backend already validates these (traverse_modules walks AIAgent tools); this aligns the CLI early-error with it. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * test(flows): make failure/preprocessor path test key explicit The test used `slot:` as a json! key. json! does interpolate an ident key to its variable's value (json!({slot:1}) with slot="failure_module" => {"failure_module":1}), so the test was correct and exercised the validation — but the behavior is subtle, so build the key explicitly via serde_json::Map to remove ambiguity (review nit). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * fix(cache): use a UUID temp name for atomic put (shared-volume safe) Address Codex (P1): pid+counter temp names collide across container PID namespaces on a shared cache volume (same pid, PUT_SEQ resets to 0 per process), so two workers could truncate/clobber the same temp file before rename. Use a random UUID suffix (matching worker.rs's atomic-write helpers) — globally unique, so the cross-process temp-file hazard is closed. Also trims the comment to the AGENTS.md <=4-line limit (Pi nit). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>	2026-06-26 18:06:27 +02:00
Ruben Fiszel	12f92e3ab7	[ee] feat(backend): native script retry without one-step-flow wrapping (#9688) ... * feat(backend): native script retry without one-step-flow wrapping Schedules and data pipelines that retry a single script previously wrapped it in a one-step flow (JobKind::SingleStepFlow), creating extra job rows, a v2_job_status row, and UI projection complexity. This adds native retry on a plain JobKind::Script job. - RetrySettings: flatten Retry into a deduped retry_settings table, carried via the existing runnable_settings_handle (lazy, off the hot path). - push() materializes a bare-script-with-retry SingleStepFlow into a native Script job (gated on min-version + no handlers/retry_if). - add_completed_job re-pushes the next attempt on failure with backoff, tracking the attempt counter in v2_job_queue.extras and the chain via parent_job; schedule completion handlers fire only on the terminal attempt. - frontend: ScriptRetryChain shows the attempt chain on the run page. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(backend): native retry_if eval + per-occurrence schedule handlers Extends native script retry to the two cases that previously stayed on the one-step-flow path: - retry_if: evaluated natively on the failure path via a feature-gated windmill-jseval dep (quickjs) over the failure result + flow_input; push materializes such policies natively only when quickjs is available. - on_failure_times / on_recovery: apply_schedule_handlers now resolves each past scheduled occurrence's terminal status across its native-retry chain (root OR any parent_job=root child succeeded) and excludes the current occurrence, so the counting is per-occurrence rather than per-attempt. All scheduled-script retries now go native (schedule.rs gate removed). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * refactor(backend): always materialize retry_if natively; unsupported without quickjs retry_if is evaluated by the worker (which always has quickjs), not the pusher, so gating materialization on the pusher's feature was wrong. The flow path was never a real fallback either — the flow runtime needs quickjs to evaluate retry_if too. retry_if now always goes native; on a worker without quickjs it is unsupported and fails closed (no retry). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(backend): un-park asset-cascade (pipeline) retry Native retry resolves the blocker that parked pipeline retry: a retried subscriber is now a Script job (not a one-step flow / flow step), so it stays eligible for asset dispatch and can trigger its own downstream on recovery. - scripts.rs: persist // retry <count> [<delay>] to script_trigger on asset edges (was dropped with a TODO warning). - asset_dispatch.rs: is_eligible_kind keys off flow_step_id, not parent_job, so native-retry attempts dispatch on success while flow steps stay excluded. - tests: retry-bearing subscriber now dispatches as a native Script carrying the policy in runnable_settings_handle; native-retry attempt is eligible. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(backend): cap native retry interval, lazy result serialization, idempotent retry push Hardening from a self-review of the native retry path: - Cap the backoff at MAX_RETRY_INTERVAL to match the flow-runtime path (evaluate_retry); the exponential formula could otherwise schedule up to ~18h vs the flow path's 6h. - Serialize the failure result lazily (only when a retry_if policy needs it), so the common failure no longer pays the serialization on the failure path. - Push each retry with a deterministic id per (root, attempt). If a worker dies between enqueueing the retry and finalizing the current attempt, the reaper re-handles the attempt and lands here again — push rejects the duplicate id, so the retry is enqueued exactly once (no double-retry). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(backend): defer schedule handlers idempotently on retry-push replay (review P1) Address local-review findings: - P1: retry_pending was derived from the retry push *result*, so on a worker crash + reaper replay the duplicate-id push returned Err → retry_pending flipped to false → apply_schedule_handlers fired for the non-terminal attempt (and the terminal attempt later fired them again). Pre-check whether the deterministic retry id already exists and report it as pending without re-pushing, so the handler-deferral invariant is crash-idempotent too. - P2: refresh the stale 'wrap the script in a one-step flow' comment in the asset-cascade retry push — it now materializes a native Script. - Add RetrySettings <-> Retry round-trip unit tests (clamping edges). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * test(backend): native retry chain + per-occurrence status sqlx tests Close the two integration-test gaps flagged in local review: - chains_attempts_and_is_idempotent: drives maybe_enqueue_native_script_retry through attempt0 -> retry1 -> retry2 -> exhausted (counter, backoff, max-attempts) and asserts crash-replay idempotency (the P1 fix: a replayed completion reports pending without double-enqueueing). - per_occurrence_status_counts_recovered_as_success: pins the exact per-occurrence terminal-status query from jobs_ee::apply_schedule_handlers — a retried-but- recovered occurrence counts as success, retries (parent_job set) are excluded from occurrence counting, and the current occurrence is excluded. - canceled_job_does_not_retry: cancellation wins over a pending retry. Runtime sqlx API (no .sqlx cache entry needed). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(frontend): exclude schedule handlers from the retry-attempt chain The retry chain listed all script children of the root by parent_job, but schedule completion handlers (on_failure/on_recovery/on_success) are also script children — when the occurrence has no retries, the handler's parent is the root itself, so a successful, never-retried job rendered a bogus 'Retries (1)' badge pointing at the handler. Filter children to re-runs of the same script (matching script_hash); real retries keep the root's hash, handlers run a different script. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(frontend): surface schedule handlers on the run page Extend the run-page chain component with schedule completion handlers: - A 'Handlers' row on a scheduled job links to the on_failure/on_recovery/ on_success runs that fired for that occurrence (found as children of the terminal attempt, identified by their synthetic created_by). - A handler's own run page now shows a 'Failure/Recovery/Success handler' label with a link back to the run it handled and its schedule. on_recovery and on_success share created_by, disambiguated by the recovery-only error_started_at arg. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(backend): restore folder_default_permissioned_as sqlx caches dropped by prepare An earlier `cargo sqlx prepare` on this branch ran before #8801's folder_default_permissioned_as test merged in, so it pruned the 3 query caches that test needs; cargo_test then failed under SQLX_OFFLINE. Restore them from main. * fix(backend): only cascade assets from native retry attempts, not handlers (review P1) is_eligible_kind keyed dispatch on flow_step_id alone, so every parented Script child became asset-eligible — including schedule/error/recovery handlers (Script jobs with parent_job set and no flow_step_id). A handler that declares assets would then trigger a cascade the old parent_job IS NULL guard prevented. Gate parented jobs on being a genuine retry attempt: a re-run of the SAME runnable as its chain parent (handlers run a different script). Runtime query, no sqlx cache. * fix(backend): cache the private-gated retry_setting asset-dispatch test query The same prepare-without-private that dropped the folder_default caches also pruned the cache for the retry_setting_dispatches_subscriber_as_native_script test query (asset_trigger_dispatch.rs:721). Regenerated with --features private. * fix(backend): exclude handler children from per-occurrence recovery (review) A scheduled occurrence's on_failure/on_success handler runs as a successful child (parent_job = occurrence), and the per-occurrence success EXISTS counted ANY successful child — so a failed occurrence whose error handler succeeded was marked 'recovered', breaking on_recovery (test_script/flow_schedule_handlers in the merge) and on_failure_times counting. EE query now scopes the EXISTS to same-runnable children (only native retry attempts); regenerate sqlx cache + bump ee-repo-ref. native_retry_test gains a handler-child regression case. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * test(backend): scheduled-script retry is a native Script, not SingleStepFlow test_push_script_with_retry / test_try_schedule_with_retry (from main) asserted the old SingleStepFlow wrapping for scheduled-script retry; this PR makes it a native Script. Update both to assert kind='script' and that the retry policy is carried via runnable_settings_handle. * fix(backend): preserve dedicated_worker on native retry + saturate count casts (cubic) Address cubic CI review: - P1: the SingleStepFlow->native Script materialization dropped dedicated_worker, so a dedicated-worker scheduled script lost its dedicated pool on retry. Resolve it from the script row in push so the materialized Script keeps the dedicated tag. - P2: saturate the u32->i32 retry-attempt narrowings (RetrySettings::from) and the u32->i16 // retry count narrowing (scripts.rs) instead of wrapping. * fix(backend): use a retry-specific signal, not runnable equality (codex review) Address Codex CI review: - P1: is_native_retry_attempt treated any same-runnable parented Script child as a retry. WAC v2 inline children have that exact shape, so an inline child of an asset producer would cascade. Use a retry-specific signal instead: the job carries a retry_settings policy (always re-inserted by maybe_enqueue) and has no flow_innermost_root_job. Apply the same flow_innermost guard to the EE per-occurrence EXISTS (WAC inline children must not count as a recovery). - P1: the deterministic retry-id pre-check raced with push; a concurrent duplicate now resolves as 'retry pending' (re-check on the duplicate-id error) instead of flipping retry_pending to false and firing handlers early. - Tests: native_retry + asset_trigger_dispatch gain WAC-inline-child cases. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * refactor(backend): explicit native_retry_attempt marker, drop heuristics Replace the per-site "is this a retry?" inference (parent_job + runnable match + flow_innermost / retry_settings) with one explicit marker: a sparse native_retry_attempt(job_id, attempt) table, written in maybe_enqueue. The marker also carries the attempt counter (previously in v2_job_queue.extras), so it's the single source of truth. - asset_dispatch: is_native_retry_attempt is now one indexed EXISTS on the marker. - EE per-occurrence query: joins the marker instead of guessing by runnable/flow_innermost. - maybe_enqueue: reads/writes the marker (persistent) instead of queue extras. - Lifecycle: swept with the job in retention (log_cleanup), no FK to keep bulk delete cheap. - Eliminates handler / WAC-inline-child misclassification by construction. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(backend): sweep native_retry_attempt markers in the periodic retention path too (codex) The marker has no FK and relies on retention cleanup; log_cleanup.rs swept it but the periodic monitor.rs path deleted v2_job rows without it, orphaning markers. Add the same WHERE job_id = ANY(...) sweep there. * fix(backend): widen native_retry_attempt.attempt to integer (cubic) The smallint column was cast to/from u32 and could wrap a retry chain longer than i16::MAX into premature exhaustion. Use integer, matching the retry policy's i32 attempt count, so no narrowing occurs on the maybe_enqueue read/write path. * feat(frontend): mark retries via is_retry on listJobs; drop SAVEPOINT - Expose an is_retry flag on jobs (UnifiedJob/CompletedJob/QueuedJob + openapi), computed from the native_retry_attempt marker. The run-page chain now filters retry attempts by is_retry instead of the script_hash heuristic, so WAC v2 inline children (same script, parent_job) no longer render as retries (codex). - Revert the marker-cleanup SAVEPOINT (an unused pattern in this codebase): keep the plain catch-and-continue matching the other side-table deletes; the table is created by a startup migration so it always exists when cleanup runs. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(backend): mark is_retry sqlx(default) so non-list job queries can omit it The single-job GET query maps directly to CompletedJob/QueuedJob via FromRow but does not select is_retry, which errored with "no column found". Only the list endpoint populates the marker; #[sqlx(default)] lets every other query omit the column and default to None. * feat(backend): select is_retry in single-job GET too for consistency The list endpoint already exposes the marker; populate it on the single-job GET (both completed and queued variants) as well so a run loaded directly reflects its retry status. #[sqlx(default)] stays as a safety net for any other query. * feat(backend): reap orphaned native_retry_attempt markers via periodic sweep The marker has no FK to v2_job (to keep the hot bulk retention delete cheap), so direct job deletions (workspace/job delete, schedule clearing) would leave marker rows orphaned. Rather than add explicit cleanup to every v2_job delete site (which must then be remembered for every future path), reap orphans in the periodic delete_expired_items pass: DELETE FROM native_retry_attempt WHERE NOT EXISTS (the job). The table is sparse so the anti-join drives off it and probes v2_job by PK — cheap. Retention still sweeps markers inline (keeps the table small so this stays cheap); a transient orphan is harmless (nothing reads is_retry for a gone job). * fix(frontend): include flow handlers in retry chain handler row (codex) Schedule on_failure/on_recovery/on_success handlers can be flow paths (flow/...), whose handler job is a flow, not a script. The chain fetched children with jobKinds:'script', hiding flow handlers. Drop the kind filter — retry attempts are still selected by is_retry and handlers by created_by, so both kinds surface. * fix(backend): carry concurrency/debouncing settings into native retries maybe_enqueue re-pushed the next attempt with ConcurrencySettings/DebouncingSettings ::default(), dropping the script/pipeline concurrency settings the failed job carried in its runnable_settings_handle. A retry of a concurrency-limited script then inserted no concurrency_key and ran unbounded. Resolve both from the same handle (cached) and pass them in the payload, which push forwards to the materialized retry. Adds a regression test asserting the retry's handle resolves to the concurrency settings. * fix(backend): carry concurrency/debounce into scheduled-retry root + document retry-helper auth (codex) P1a (schedule.rs): the scheduled-retry materialization fetched the script's concurrency/debounce settings but passed ConcurrencySettings/DebouncingSettings ::default() into the SingleStepFlow payload, so the root attempt's handle held only the retry policy and the whole chain ran unbounded. Pass the fetched settings. Regression test asserts the root handle resolves to retry + concurrency. P1b (jobs.rs): document maybe_enqueue_native_script_retry's authorization contract — it is pub only for the integration test; the sole production caller is the worker completion path passing a DB-derived, already-authorized MiniCompletedJob. * docs(backend): attach native-retry auth contract to the function itself (codex) The doc block was merged with eval_retry_if's doc and bound to that function, leaving maybe_enqueue_native_script_retry undocumented. Split them: eval_retry_if keeps its own doc; the native-retry + authorization contract now sits directly above maybe_enqueue_native_script_retry. * docs(backend): regenerate served openapi-deref with is_retry + fix stale comments (codex) - Regenerate openapi-deref.{yaml,json} (served from lib.rs): they were stale since 1.734.0 and lacked is_retry on QueuedJob/CompletedJob, so clients reading the served spec couldn't see the field. Now current at 1.739.0. - schedule.rs: a retry_if gate is evaluated at failure time and fails closed without quickjs (no retry); it does not fall back to a flow path. - windmill-types jobs.rs: is_retry is selected by both the list and single-job GET endpoints (not list-only). * docs(backend): fix remaining stale retry_if/quickjs comments (codex) The retry_if block and the push materialization comments claimed push keeps retry_if on a flow path / the worker always has quickjs. The code always materializes native retry and the no-quickjs eval_retry_if path fails closed — correct the comments to that constraint. * docs(backend): fix stale quickjs-fallback + schedule-handler-restriction comments (codex) - Cargo.toml quickjs feature: without quickjs a retry_if gate cannot be evaluated and the job does not retry (no one-step-flow fallback). - jobs.rs handler-defer comment: apply_schedule_handlers resolves per-occurrence failure/recovery status across the retry chain, so the old 'restricted to schedules whose handlers don't need per-occurrence counting' claim is dropped. --------- Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>	2026-06-25 13:22:26 +00:00
Ruben Fiszel	7155a0bb96	feat: Data Pipelines alpha (#9193) ... * feat: add workspace asset graph view Workspace-wide canvas of assets and their producer/consumer scripts, reachable from the assets page. Left-to-right layered layout via d3-dag sugiyama, rendered with @xyflow/svelte (same stack as the flow editor). GET /w/:ws/assets/graph returns deduped nodes + edges. Follow-ups: filters (kind/folder/search), node detail drawer, inline script edit from a clicked node. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * all * all * all * update * all * all * all * feat(pipeline): output-kind picker and per-(lang, output) templates Add a third stage to PipelineInsertMenu that asks what kind of asset the new script will produce (datatable / ducklake / s3 parquet / s3 object / none). The picked kind drives a real wmill SDK skeleton — typed datatable inserts, ducklake CREATE+INSERT, s3 parquet COPY, etc. — with the upstream asset auto-wired as the input source when added from an asset node. Reorder languages to bun → duckdb → python → sql so data-shaped languages surface first. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * all * chore(main): release 1.693.4 (#8994) * chore(main): release 1.693.4 * Apply automatic changes --------- Co-authored-by: rubenfiszel <275584+rubenfiszel@users.noreply.github.com> * feat: ansible delegate_to_git_repo install_requirements, dynamic fields, --limit (#8997) * feat: ansible delegate_to_git_repo install_requirements, dynamic fields, --limit Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * fix: include .yaml variants in collections/roles requirements lookup Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com> * fix(cli): only preserve case for raw-app runnableIds, not app/flow summaries (#9000) * fix(cli): only preserve case for raw-app runnableIds, not app/flow summaries PR #8940 stopped lowercasing in sanitizeForFilesystem to fix #8939, where a raw-app runnableId like CamelCaseTSRunnable produced a CamelCase YAML metadata file but a lowercased code file, making them desync and register as duplicate runnables on push. That fix overshot. sanitizeForFilesystem is also reached by newPathAssigner, which serves normal apps and flows where the input is the script's human summary ("Get Users Data") rather than an identifier. There the on-disk filename is the only artifact — there's no companion YAML to keep in sync — so lowercasing was the right behavior. Removing it changed both the on-disk filename and the !inline reference in app.yaml / flow.yaml from get_users_data.inline_script.ts to Get_Users_Data.inline_script.ts on the next pull, surfacing as unwanted case churn for users updating to 1.693.x. Add a preserveCase option to sanitizeForFilesystem (default false → lowercase). newRawAppPathAssigner opts in; newPathAssigner stays on the default. Update unit tests accordingly and add an end-to-end raw-app round-trip in raw_app_sync.test.ts that pushes a CamelCase backend runnable, pulls it back, and asserts both YAML and code file preserve case with no lowercase orphan. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * test(cli): use readdir for exact-case orphan check on Windows The CamelCase round-trip test used fileExists("camelcasetsrunnable.ts") to assert no lowercase orphan was produced, which false-positives on Windows since the filesystem is case-insensitive and resolves the lookup to the existing CamelCaseTSRunnable.ts. Switch to readdir + toContain so the exact on-disk casing is compared identically on Linux and Windows. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * feat(cli): wmill-lock.yaml auto-fill + --rehash-only + path-prefix dedup (#8978) * fix(cli): canonical lockfile hashes + lock upgrade migration to v3 Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * fix(cli): use __app_hash subpath in rehash missing-entry check Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * fix(cli): run sync pull lockfile auto-fill regardless of changes Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * chore: regenerate system prompts for new lock and rehash-only commands Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * fix(cli): address review feedback on lock upgrade Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * fix(cli): drop v3 marker; always run fallback; fail-fast on unknown lockfile version Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * fix(cli): drop yaml-round-trip legacy hash variant; recover via --rehash-only Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * fix(cli): include legacy hash in script push staleness warning check Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * revert(cli): drop canonical hash formula; keep raw-bytes hashing Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * perf(cli): reuse change-tracker map for sync pull lockfile auto-fill Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * fix(cli): address review feedback on rehash-only Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * test(cli): pin lockfile hash + yaml format and cover regression cases Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * test(cli): byte-stable snapshot tests for flow.yaml format Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * test(cli): add app and script-metadata yaml snapshot fixtures Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * fix(cli): address claude review on rehash-only Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * refactor(cli): factorize script-path to remote-path derivation Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * fix(cli): address claude + cubic review (dry-run mutation, rehash short-circuit) Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * refactor(cli): make rehash a subcommand and factorize fs walks Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * fix(cli): normalize line endings in yaml snapshot tests for windows ci Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * fix(cli): address review feedback on rehash + auto-fill - Flat-layout scripts now clearGlobalLock before rehash write so legacy ./-prefixed duplicates get cleaned up (matches flow/app behavior). - Add MalformedLockfileError; sync pull auto-fill re-throws it alongside UnknownLockVersionError instead of silently warning + continuing. - Document the legacy step-removal false-negative in isFlowDirectlyStale / isAppDirectlyStale and the categorizeLocalFiles ignore-filter invariant. Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com> * fix: use otel.status_message for OTLP Status.message on failed jobs (#8995) tracing-opentelemetry only recognizes otel.status_code and otel.status_message as fields that map to the OTLP Status proto. The previously-used otel.status_description fell through to the generic attribute recorder, leaving Status.message unset and preventing OTLP consumers from filtering spans on error status. Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * fix: route email trigger path through standard info channel (#8996) * docs(skill): document email triggers and S3 attachments Add an "Email triggers" section to the triggers skill covering the local-part config, the parsed_email/raw_email/email_extra_args payload, the URL-style extras convention, where to find trigger_path (only with a preprocessor, at event.trigger_path), and — most importantly — that binary attachments are uploaded to the workspace S3 bucket and surface as `{ s3: "windmill_emails/<job_id>/attachments/<filename>" }`. Scripts must use wmill.loadS3File / wmill.load_s3_file to read them. Also pulls EmailTrigger into the schema mappings so a real `email_trigger.schema.yaml` is generated, and adds Email/Azure to the trigger kinds list in the CLI agent guidance. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * chore: update ee-repo-ref for email trigger path fix Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * chore: update ee-repo-ref to 26184ab7a4aadfc529dcedf038aa08d36c7ad381 This commit updates the EE repository reference after PR #553 was merged in windmill-ee-private. Previous ee-repo-ref: 318a46897a605dc9be3817901f35ba5a99a0a525 New ee-repo-ref: 26184ab7a4aadfc529dcedf038aa08d36c7ad381 Automated by sync-ee-ref workflow. --------- Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com> Co-authored-by: windmill-internal-app[bot] <windmill-internal-app[bot]@users.noreply.github.com> * update git sync version to 1.693.5 * fix: pair PG arg type with actual Rust binding to keep query_typed_raw safe (#8999) * fix: pair PG arg type with actual Rust binding to keep query_typed_raw safe Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * fix(pg): wrap encoder errors with arg context, add fallback test Followups on #8999 review: - Wrap rust-postgres "error serializing parameter N" failures with the arg name, JSON value kind, and asserted Postgres type plus a hint about an explicit cast — so users see actionable context instead of an opaque WrongType. - Drift-prevention meta-test: assert otyp_to_pg_type and convert_val agree on the Type for every recognised arg_t when the JSON value matches its natural Rust kind. Catches future drift if either side changes. - Integration test for the prepare + query_raw fallback path: confirms unrecognised arg_t (custom enum) is routed through prepare and the server-resolved type appears in the failure surface — flips into a test failure if a regression accidentally routes unrecognised types through query_typed_raw. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * fix(pg): add otyp_inferred flag + regex-based placeholder renumbering Two follow-ups from the review of #8999: 1. **Issue #1 (Number/Bool + explicit text decl in WHERE)** Add `Arg::otyp_inferred: bool` to the parser. The PG SQL parser sets it `true` only at the "no info → fall back to text" site (bare `$N`, no inline cast, no `-- $N (TYPE)` decl). All other arg sources keep it `false`. In `convert_val` this flag distinguishes: - explicit text-like target (`-- $1 (text)` or `$1::text`) — coerce `Bool`/`Number` → `Box<String>` so `WHERE text_col = $1` works (`text = text` operator). Pre-#8988 behaviour, restored. - parser-default text (bare `$N`) — bind the value's natural Rust type so the regression case (`Value::Bool` against a real `bool` column via `CAST AS bool`) keeps working. `Arg` is in `windmill-parser`; the new field has `#[serde(default)]` so persisted signatures stay backward-compatible. 2. **Issue #4 ($5/$50 substring rewrite collision)** Replace the per-index `String::replace` chain (which turned `$50` into `$10` when oidx=5 was processed first) with a single regex pass. `\d+` is greedy, so `$5` and `$50` match as distinct units; indices outside the mapping are left intact. 3. Tests: - parser: `test_parse_pgsql_otyp_inferred_flag` covers bare/inline- cast/decl/mixed shapes. - executor unit: `convert_val_bool_against_every_arg_t` and `convert_val_*_number_*` split each text-like target into explicit vs inferred expectations. - executor unit: `renumber_sparse_placeholders_no_collision`. - integration: `test_postgresql_arg_type_combinations` adds 4 cases covering decl(text)+Number/Bool in WHERE, bare $1+Bool, and sparse positional args ($5/$50). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * fix(pg+sdk): enum support, extended String arms, position-aware $N rewrite, SDK quality Backend: 1. **`AnyTextValue` ToSql/FromSql wrapper**: vanilla `tokio_postgres`'s `ToSql for String` / `FromSql for String` reject `Kind::Enum` and `Kind::Domain` even though the wire format is plain UTF-8. The wrapper accepts those kinds in both directions. End result: explicit `$1::my_enum` / `CAST($1 AS my_enum)` casts now round-trip without the ugly `CAST($1::text AS my_enum)` workaround, AND `SELECT enum_col` results come back as JSON strings instead of erroring at the FromSql layer. 2. **#10 — Value::String → numeric/real/double/oid/bool**. Without these arms, a string-encoded value (`"3.14"`, `"true"`) for a non-text / non-temporal arg_t fell through to `Box<String> + TEXT`, which then failed at the server (no implicit cast text→numeric in expression context). Now strings are parsed into the matching native type with clear error messages on parse failure. 3. **Position-aware `$N` rewrite**: replaces the regex-based renumbering (which fixed the `$5/$50` substring collision but still walked through string literals and comments, mangling `'price: $5'` etc.) with a walk over `parse_pg_statement_arg_positions` — the same string/comment/dollar-quote-aware tokenizer used for index discovery. Adds `parse_pg_statement_arg_positions` to the parser's public API. SDK: 4. **BigInt support**: `JSON.stringify(BigInt)` throws. The SDK now stringifies bigints before serialisation; the executor accepts numeric strings into BIGINT arg slots via the existing `Value::String → INT8` parsing arm. SDK-side `inferSqlType` is split so `BigInt` always resolves to `BIGINT` (was reaching `Number.isInteger(BigInt)` which returns false → wrong default). 5. **Homogeneous array auto-tag**: `${[1,2,3]}` against an `int[]` column now emits `$1::BIGINT[]` instead of `$1::JSON`. Detection covers primitive types only (number / bigint / string / boolean); mixed or nested arrays still fall back to JSON. Mixed int/float widens to `DOUBLE PRECISION[]`. 6. **`.query()` positional bug**: previously the `.query()` method abused the template-tag builder, which appended `$N::TYPE` after the user's literal SQL string instead of binding by position (`SELECT $1, $2` became `SELECT $1, $2$1::BIGINT`). Now `.query()` builds the executor-shaped content directly: a `-- $N argN (TYPE)` declaration block followed by the user's SQL verbatim. Tests: - Parser: `test_parse_pg_statement_arg_positions_skips_strings_and_comments` asserts string literals, comments, and dollar-quoted blocks don't produce positions (so renumbering doesn't mangle them). - Executor unit: `renumber_sparse_placeholders_no_collision_no_string_mangling` uses the new position-aware path and includes string-literal + comment + `$$…$$` cases. Existing convert_val tests grow to cover new String→numeric/real/double/oid/bool arms. - Integration: `test_postgresql_arg_type_combinations` adds 13 cases (enum round-trip both directions, string→numeric/real/double/bool/oid, string-literal `$N` non-mangling). The prepare-fallback test now asserts SUCCESS (not failure) for enum encoding via AnyTextValue. - SDK: new `typescript-client/tests/sqlUtils.test.ts` (42 tests) exhaustively covering inferSqlType primitives + arrays, parseTypeAnnotation, datatable() template tag (with all the new shapes — BigInt, homogeneous arrays, RawSql, schema preamble), datatable().query() positional, and ducklake() shape. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * fix(pg): replace DISCARD ALL with curated reset (preserves typeinfo cache) Found while exhaustively probing custom-type DX: every cached-connection reuse was running `DISCARD ALL`, whose included `DEALLOCATE ALL` deallocates *all* prepared statements server-side — including the typeinfo statements that tokio_postgres caches per-Client to resolve custom enum / domain Oids. tokio_postgres still held `Statement` objects whose names the server had forgotten, so the next custom-type query failed with intermittent "prepared statement \"sN\" does not exist" errors. The failure was easy to reproduce: any sequence that forced typeinfo lookup for two different custom-type kinds on the same cached connection (e.g. enum followed by domain) would hit it. Replace `DISCARD ALL` with a curated reset that explicitly targets the state we actually care about, *without* touching prepared statements: RESET ALL — GUC parameters (search_path, application _name, statement_timeout, …) RESET SESSION AUTHORIZATION — undoes both `SET SESSION AUTHORIZATION` and `SET ROLE` (RESET ALL does NOT — these aren't GUC parameters, so without this an elevated role from a previous job would silently leak) UNLISTEN * — drops LISTEN registrations CLOSE ALL — closes open cursors Trade-off: temp tables, advisory locks (session-scoped), and user-created PREPARE statements may persist across cached-connection reuse — rare in datatable / PG-script workloads. tokio_postgres's typeinfo cache survives intact, so custom enum / domain queries are fast on subsequent reuse. Tests: - `test_postgresql_custom_types_on_cached_connection` — runs 10× alternating enum + domain queries on a cached connection. Pre-fix this failed with `prepared statement "sN" does not exist` after the first reuse; post-fix passes. - `test_postgresql_set_role_does_not_leak_across_cached_connection` — switches `SET ROLE` and `SET SESSION AUTHORIZATION` to a non-postgres role, then runs a follow-up job and asserts current_user/session_user are restored. Specifically catches the case where someone might switch back to `RESET ALL` alone (which doesn't cover SET ROLE / SESSION AUTHORIZATION) and silently introduce a permission-leak vector. - All existing session-isolation tests (`test_postgresql_cached_connection_resets_session`, `test_postgresql_single_worker_session_isolation`, `test_postgresql_100_jobs_cached`) continue to pass. Found via end-to-end probing of datatable / PG-script DX, not previously covered: the existing isolation tests only did `SET ROLE postgres`, the connecting user, so the leak was invisible. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * fix(pg): address PR #8999 review (cubic + claude) cubic (P1, real bug): - `convert_vec_val` for `timetz` array asserted `Type::TIMETZ_ARRAY`, but chrono `NaiveTime` only encodes for TIME (same caveat as the scalar arm). Switch to `Type::TIME_ARRAY`; rely on PG's implicit `time→timetz` assignment cast at the column site. Add an explicit unit test. claude (#1, silent failure → explicit error): - `Bool` + explicit `(char)` / `(character)` decl previously silently bound BOOL, hoping the server would cast at the use site — but PG has no implicit `bool→char` and the resulting error ("operator does not exist: bool = char") was opaque. Now error at bind time with an actionable hint to use `bool` decl or pass the value as a "t"/"f" string. claude (#2, asymmetry doc): - Object/Array still coerce to text on `matches!(typ, Typ::Str(_))` (covers both explicit AND inferred-default text), unlike Bool/Number which key on `explicit_text_target`. The asymmetry is intentional (no implicit `jsonb → text` cast in expression context vs PG having implicit `bool/int → text` casts) — added a body comment so future maintainers don't try to "align" them. claude (#3, perf): - `parse_pg_statement_arg_indices` and `parse_pg_statement_arg_positions` walked the SQL tokenizer twice. Fold into a single pass that derives the index set from the position list. claude (#4, fmt drift): - `cargo fmt` over the parser crates I touched with perl scripts in the earlier commit (windmill-parser-{sql,bash,ts,go,php,java,csharp,nu,py, rust,graphql,yaml,r}). Net cosmetic. claude (#5, parseTypeAnnotation): - One-line caveat in the SDK's `parseTypeAnnotation` that the returned string is presence-only (e.g. `${x}::DOUBLE PRECISION` returns `"DOUBLE"`, `CAST(${x} AS int)` returns `"int)"` — neither matches a real PG type, but the only consumer just checks `!== undefined`). While here — discovered + fixed independently while exhaustively probing DX: - **Replace `DISCARD ALL` with curated reset** (`RESET ALL; RESET SESSION AUTHORIZATION; UNLISTEN *; CLOSE ALL;`). DISCARD's `DEALLOCATE ALL` killed tokio_postgres' typeinfo cache, producing intermittent `prepared statement "sN" does not exist` errors on custom-type queries after cached-conn reuse. New regression tests: `test_postgresql_custom_types_on_cached_connection` and `test_postgresql_set_role_does_not_leak_across_cached_connection` (the latter catches the case where someone might switch back to `RESET ALL` alone and silently introduce a permission-leak vector — RESET ALL doesn't cover SET ROLE / SET SESSION AUTHORIZATION). - **ISO-8601 timestamp results** (`pg_cell_to_json_value`). Pre-fix `TIMESTAMP` was rendered with a space separator ("2024-01-15 10:30:00") and `TIMESTAMPTZ` with " UTC" suffix ("2024-01-15 10:30:00 UTC") — neither parseable by `date-fns parseISO`, JavaScript `new Date()` is lenient enough to handle them but several frontend `App*Input.svelte` components use parseISO and fail silently. Switched to ISO-8601 with `T` separator and `+00:00` offset; arg-parsing path still accepts the legacy " UTC" suffix for back-compat. Test coverage: - 17/17 unit (`pg_executor::tests`) - 9/9 integration (`backend/tests/worker.rs`, `test_postgresql_*`) - 27/27 parser (`windmill-parser-sql`) - 42/42 SDK (`typescript-client/tests/sqlUtils.test.ts`) Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * fix(pg): bounded one-shot warning on numeric precision loss + ISO-8601 + NaN handling Found while probing PG-script DX with millions of numeric cells: 1. **Numeric precision-loss warning**: `numeric` results are still serialised as JSON Number (back-compat — switching to JSON String would silently break user code doing arithmetic on results), but we now detect `Decimal -> f64 -> Decimal` round-trip failure and emit a single job-log warning recommending a `::text` cast in the SQL. Bounded by `NUMERIC_PRECISION_CHECK_BUDGET = 256` cells per query (one atomic load + one fetch_sub on the hot path; first lossy value short-circuits to a single load thereafter). Worst-case overhead on a 1M-cell numeric-heavy query: ~25µs of checks + 5ns × N atomic loads (vs. ~100ms unbounded). 2. **ISO-8601 timestamps**: `pg_cell_to_json_value` previously returned `"2024-01-15 10:30:00"` (TIMESTAMP) and `"2024-01-15 10:30:00 UTC"` (TIMESTAMPTZ) — neither parseable by date-fns `parseISO`, which is what the apps `App*Input.svelte` components use, so timestamp values silently failed to round-trip into date pickers. Switch to ISO-8601 (`T` separator + `+00:00` offset) on the result side; arg-parser continues to accept the legacy `" UTC"`-suffixed format for back-compat. 3. **Float NaN / Infinity results**: `Number::from_f64` returns None for NaN / ±Inf, which `pg_cell_to_json_value` was raising as "invalid json-float" — failing the *entire* query if any cell held one of these special values. Now serialise them as JSON strings ("NaN", "Infinity", "-Infinity") and let the rest of the row come through. Arg-side: `s.parse::<f64>()` already accepts the same strings. Tests: - `decimal_fits_f64_losslessly_predicate` — covers fits / doesn't-fit cases for the precision-loss predicate. - `precision_check_budget_caps_per_query_overhead` — locks in the budget cap and the loss-flag short-circuit. - All 9 PG integration tests + 17 unit tests pass. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * fix(pg): add pg_advisory_unlock_all to reset; warn on missing args; honor decl defaults While probing PG-script DX further found three more frictions: 1. **Advisory lock leak** (cubic P2): switching from `DISCARD ALL` to `RESET ALL; RESET SESSION AUTHORIZATION; UNLISTEN *; CLOSE ALL;` meant session-scoped advisory locks (`pg_advisory_lock`) leaked across cached-connection reuse. Add `SELECT pg_advisory_unlock_all()` to the chain — `DISCARD ALL` covered this implicitly via `DISCARD PLANS / DEALLOCATE / pg_advisory_unlock_all` and we lost it in the switch. 2. **Missing-arg silent NULL**: an arg declared in the SQL (e.g. `-- $1 amount (numeric)`) but not provided in the args object was bound as NULL with no error / warning. Misspelling the key in the args object silently produced a row of NULLs — a notorious DX debugging trap. Now: collect the names of declared-but-missing args during dispatch and emit a single one-shot warning to the job logs at end-of-query naming each one. Bound NULL is preserved for back-compat. 3. **Declaration defaults ignored**: `-- $1 a (int) = 5` carries `arg.default = Some(Number(5))`, but the dispatch fell straight to NULL when the arg was missing. Now: respect the default — user-supplied value > declaration default > NULL. Also fixes the warning logic above (only warn for args that *don't* have a default). Tests: existing 19 unit + 9 integration pass. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * fix(pg): multi-word PG types with [] suffix lost the array-ness; array arms accept stringified values Two more frictions found while probing SDK end-to-end against a real datatable resource: 1. **Multi-word array types lose the [] suffix in the parser**. `transform_types_with_spaces` recognises aliases for "double precision", "character varying", "timestamp with time zone", etc. but its return type was `&'a str` — only the bare alias, never with a trailing `[]`. The `RE_CODE_PGSQL` regex's `\w+` captures stop at the first space, so the regex's own `(?:\[\])?` array-suffix branch sees only `"double"` (not `"double precision[]"`); the `[]` was silently lost. Result: `$1::double precision[]` (which the SDK now emits for homogeneous float arrays via the new auto-tag) routed through `Value::Array → Type::JSONB` and the server failed with "cannot cast type jsonb to double precision[]". Fix: switch `transform_types_with_spaces` to return `Cow<'a, str>` and re-check the trailing bytes after a multi-word match. If they start with `[]`, return `format!("{alias}[]")` — Owned. Single-word types and the no-match path keep returning Borrowed slices, so no allocation in the hot path. 2. **Array arms in `convert_vec_val` rejected stringified values for numeric / int* / bool / oid / real / double**. The scalar `convert_val` already parses strings into the matching native type for these arg_ts, but the array variant only accepted JSON-native counterparts. Sending `["1.5", "2.5", "3.5"]` against `$1::numeric[]` (e.g. via `unnest` for bulk loading, or `JSON.stringify(BigInt[])` round-trip) failed with "Mixed types in array". Now the array arms mirror the scalar ones — `as_<native>().or_else(|| as_str().and_then(parse))` — so both shapes round-trip cleanly. Tests: 19 unit + 9 integration pass; existing parser tests cover the multi-word array forms (the regex-cap behaviour didn't break for single-word types, and Cow plumbing is transparent to all callers). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * fix(parsers): add otyp_inferred field to Arg literals in tests + 3 missed src files CI failures: the perl-driven sweep that added `otyp_inferred: false` to every `Arg { ... }` literal when I introduced the field in the parser schema covered `src/lib.rs` files but missed: - parsers/windmill-parser-bash/src/lib.rs (mass-edited but a later format pass un-applied a few sites) - parsers/windmill-parser-go/src/lib.rs (same) - parsers/windmill-parser-graphql/src/lib.rs (same) - parsers/windmill-parser-nu/tests/tests.rs (test file — not swept the first time) - parsers/windmill-parser-ts/tests/tests.rs (test file — same) Also tightened the regex to handle `oidx: None` without the trailing comma (some test files had the field as the last initialiser line). `cargo build --features <CI feature combo> --workspace --all-targets` is clean. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * fix(sdk): Date → TIMESTAMPTZ; NaN / ±Infinity → string Two more frictions found while running the actual SDK end-to-end against a live datatable resource: 1. **JS `Date`** fell into the typeof "object" branch and was tagged `::JSON`. It worked accidentally for `${date}::timestamptz` via PG's `json → text → timestamptz` implicit cast chain, but `${date}` against a `timestamptz` column without a user-supplied cast bound the value as a JSON string and the comparison `timestamptz = json` failed. Now: `inferSqlType` recognises `Date` and tags `::TIMESTAMPTZ`; `serializeArgValue` emits `Date.toISOString()` so the executor's `Value::String → TIMESTAMPTZ` arm parses it cleanly. 2. **JS `NaN` / `±Infinity`** silently became NULL. `JSON.stringify(NaN)` returns `"null"` per the JS spec, so the value reached the executor as JSON null — the SDK's `::DOUBLE PRECISION` tag then bound a NULL double. Fix: detect non-finite numbers in `serializeArgValue` and stringify them as `"NaN" / "Infinity" / "-Infinity"`. The executor's `Value::String → FLOAT8` arm (`f64::from_str`) accepts these literals directly, and the result-side already renders the values as JSON strings (matching round-trip). SDK unit tests grow from 42 → 44 passing. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * test(pg): integration coverage for multi-word arrays + stringified array elements Locks in the two array fixes from the previous commit (`fix(pg): multi-word PG types with [] suffix lost the array-ness`) with end-to-end cases in `test_postgresql_arg_type_combinations`: - `double precision[]`, `character varying[]`, `timestamp without time zone[]` — verifies the parser keeps the `[]` suffix after multi-word alias resolution. - `numeric[]` / `int[]` / `bool[]` from stringified primitives — verifies the array arms of `convert_vec_val` apply the same string-coercion the scalar arms do. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * style: fix indentation drift on otyp_inferred lines cargo fmt cleanup of leftover indentation where the perl-driven sweep that introduced the otyp_inferred field landed at the wrong column. No behaviour change. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com> * feat: support assigning a worker tag to app inline scripts (#9002) * feat: support assigning a worker tag to app/raw-app inline scripts Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * fix: omit empty tag field from inline script raw_code payload Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * style: shrink tag popover width --------- Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com> * feat(pipeline): 2-col picker, draft path edit, save-all + leave guard Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * all * all * update * fix(cli): forward HEADERS env var on every backend fetch call (#9075) Several `fetch()` callers in the CLI bypassed `OpenAPI.HEADERS` and skipped the `HEADERS` env var, causing requests to fail behind auth gateways like Cloudflare Access (same shape as #6421): - `pushScript()` `/scripts/create` and `/scripts/create_snapshot` — regressed in #8936 when the call switched from `wmill.createScript()` (SDK) to a raw `fetch` for the `skip_if_noop` query param. - Script preview `/jobs/run/preview_bundle`. - App dev `/jobs_u/getupdate_sse` SSE stream. - `wmill docs` `/api/inkeep`. All four now spread `getHeaders()` and call `detectAuthGatewayChallenge()` so a Cloudflare/SSO challenge surfaces a clear error instead of an opaque JSON parse failure. Adds `test/headers_env_var.test.ts`: spins up an auth-gateway proxy that 403s requests missing `CF-Access-Client-Id` / `CF-Access-Client-Secret` and otherwise reverse-proxies to the test backend, then runs `wmill sync push` of a fresh script through the proxy. Negative case (no `HEADERS` env) verifies the proxy actually gates; positive case asserts every request including `/scripts/create` reaches the backend with the headers attached. Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * feat(cli): add --parallel flag to generate-metadata (#9074) * feat(cli): add --parallel flag to generate-metadata * fix(cli): validate --parallel input and harden flush ordering * perf(flows): skip flow_env DB+transform work when no resolution is needed (#9078) * fix(cli-tests): stabilize flow lock-gen race + Windows path (#9080) * fix(cli-tests): stabilize flow lock-gen race + Windows path Three CLI test failures on the latest main, all flaky on CI: 1. `Mixed Case Paths: pull and push flow with capitalized folder` and `Integration: Mixed scripts and flows with nonDottedPaths are idempotent`: flow create/update queues an async FlowDependencies job that fills inline-script lockfiles and rewrites flow.value. The tests pulled/pushed before the worker finished, so dry-run idempotency saw phantom `*.inline_script.lock` adds and `flow.yaml` edits. Added a `waitForFlowDependencyJob` helper that polls `/flows/get` for the latest `dependency_job` and `/jobs_u/completed/get` until it lands, and called it after each API/CLI flow write in both tests. 2. `HEADERS env var is forwarded on every CLI fetch` (Windows-only, added in #9075): the new test built the CLI entrypoint via `new URL("..", import.meta.url).pathname`, which yields `/C:/...` on Windows and `Bun.spawn` rejected before reaching the proxy, leaving `rejectedRequests.length` at 0. Switched to `fileURLToPath` + `node:path.join` to match `cargo_backend.ts`. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * fix(cli-tests): use /flows/deployment_status to actually wait for dep job CI reviewers (Claude, Codex) flagged the prior `waitForFlowDependencyJob` as a no-op: it read `flow.dependency_job` from `/api/w/{ws}/flows/get`, but `Flow` / `FlowWithStarred` (backend/windmill-types/src/flows.rs:20-60) do not include that field. The helper exited on the first iteration without polling. Switch to `/api/w/{ws}/flows/deployment_status/p/{path}`, which returns `{ lock_error_logs, job_id }`. `job_id` is the FlowDependencies UUID written into `deployment_metadata` in the same tx as the dep-job push (backend/windmill-api-flows/src/flows.rs:660-672 and :1275-1292), so by the time the create/update API call returns, the response carries the latest dep-job UUID. Then poll `/jobs_u/completed/get/{job_id}` as before. Local runtime for `mixed_case_paths.test.ts` jumps from ~9s to ~32s, confirming the helper now actually waits instead of returning immediately. The 404 short-circuit in `sync_pull_push.test.ts` still works — `get_deployment_status` returns 404 when the flow is absent. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * perf(flows): cache resolved flow_env per flow execution (#9079) * perf(flows): cache resolved flow_env per flow execution * perf(flows): tighten flow_env cache cap to 1024 and clarify memory note * perf(flows): don't cache transient flow_env resolution failures * chore(main): release 1.698.0 (#9076) * chore(main): release 1.698.0 * Apply automatic changes --------- Co-authored-by: rubenfiszel <275584+rubenfiszel@users.noreply.github.com> * fix: reject root-rooted paths in ansible playbook validator on windows (#9081) * fix(native-triggers): serialize Google channel renewal across replicas (#9060) * fix(native-triggers): serialize Google channel renewal across replicas `sync_all_triggers` runs every 5 minutes on every windmill-app replica with no leader election. Multiple replicas were each rotating the webhook token, creating a new Google watch channel, and racing the trigger UPDATE — leaving the loser's new token (in `token`) and channel (in Google) orphaned. Cloud was accumulating ~5 leaked tokens/week without the silent best-effort `delete_token_by_hash` ever logging a warning. Wrap each per-trigger renewal in a transaction and acquire the row with `SELECT … FOR UPDATE SKIP LOCKED`. Contending replicas skip the row instead of duplicating the work. The lock spans `rotate_webhook_token` → Google API call → `update_native_trigger_service_config` and is only released on commit. Re-checks `should_renew_channel` after acquiring the lock so a replica that committed seconds earlier doesn't trigger a duplicate renewal. The pattern matches existing batch-cleanup paths in `monitor.rs` (job-retention sweep) and other `FOR UPDATE SKIP LOCKED` call sites. Also logs at `debug!` when `delete_token_by_hash` finds no matching row, so future investigations can distinguish "deleted" from "not found" without changing the `Ok(false)` contract. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * fixup! fix(native-triggers): serialize Google channel renewal across replicas * fixup! fix(native-triggers): serialize Google channel renewal across replicas fixup! fix(native-triggers): serialize Google channel renewal across replicas Address claude review: - #5: per-skip log info -> debug (expected outcome under SKIP LOCKED) - #2: warn moved out of delete_token_by_hash to the call site that knows the expected state (try_renew_channel_locked); other callers are race-prone and shouldn't warn - #3: NULL service_config now warns (anomalous case) - #4: post-Google-API DB-update + commit failures log distinctly so the channel-orphan case is grep-able Plus: add 14d expiry to Google webhook tokens via ServiceName::webhook_token_expiration, mint fresh ephemeral-webhook-{service}-{rd5} labels at create + rotate so the existing 'ephemeral-' filter excludes them from user-token email/critical-alert paths (no filter changes in 3 places). Orphans now self-clean via the existing expiry sweep in monitor.rs. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * fixup! fix(native-triggers): serialize Google channel renewal across replicas fixup! fix(native-triggers): serialize Google channel renewal across replicas Address second-round review: - Claude #1 (P2): username_override_from_label now strips the 'ephemeral-' prefix for ephemeral-webhook-* labels, so created_by stays webhook-{service}-{rd5} instead of changing to label-ephemeral-webhook-... (preserves audit/job-list filter compatibility) - Codex (P2): updated renew_channel doc — labels are no longer copied; rotate mints fresh ephemeral-webhook-google-{rd5} with 14d expiration - Claude #3 (optional): test_rotate_webhook_token now asserts the rotated Google token has an ephemeral-webhook-google-* label and a populated expiration Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * fixup! fix(native-triggers): serialize Google channel renewal across replicas fixup! fix(native-triggers): serialize Google channel renewal across replicas Reconsider the previous fixup: stripping the 'ephemeral-' prefix made created_by no longer match token.label exactly, defeating the linking purpose. Just allowlist 'ephemeral-webhook-' alongside the other recognized webhook/email/ws prefixes — created_by becomes ephemeral-webhook-google-XXXXX, matching token.label exactly. The 'ephemeral-' substring also informs operators that this is a system-managed auto-expiring token vs a user-managed webhook trigger. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * fix(cli): bump svelte version in `wmill app new` template (#9084) * fix(cli): bump svelte version in `wmill app new` template The svelte5 template pinned `svelte` to `5.45.2`, but the Svelte compiler bundled in `wmill app dev` emits `$.delegated('click', ...)` calls. The `delegated` export was added later, so 5.45.2 doesn't have it — esbuild warns `Import "delegated" will always be undefined`, replaces the call with `void 0`, and the page crashes at first event-handler bind (white screen). Bump to `^5.55.5` so the compiler and runtime stay in sync. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * fix(frontend): bump svelte version in raw_apps UI template Mirror the CLI fix: the UI's `Add raw app` flow scaffolds a package.json with `svelte: "5.45.2"`. That works today only because the bundled rolldown worker also pins 5.45.2 — when the worker is upgraded past 5.51.1, the compiler will emit `$.delegated()` and the runtime won't have it, producing the same white-page crash that hit the CLI. 5.55.5 still exports `event` (used by the current bundled compiler), so this is forward-compatible: it works with the 5.45.2 compiler now and won't break when the worker is upgraded. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * perf(flows): gate flow_env resolve on expr text and share cache with handle_flow (#9085) * feat: parse windmill_failure field to tag run as failure (#9073) * feat: parse windmill_failure field in job result to tag run as failure * feat: preserve top-level fields when windmill_failure tags a run as failure * fix: address review findings on windmill_manual_failure * refactor: rename windmill_manual_failure to wm_failure and add wm_* aliases * fix: prefer injected ManualFailure error over sibling name/message in OTel * fix: hide _ENTRYPOINT_OVERRIDE jobs from script/flow history panel (#9088) * fix(flows): populate error handler input args from failure picker (#9087) * fix(flows): populate error handler input args from failure picker * style(flows): fix indentation in failure-step branch * fix(python): verify wheel RECORD on cache pull/install, finalize piptar (#9090) The Python per-package dependency cache could persist an incomplete wheel extraction with `.valid.windmill` set, then propagate that broken artifact to every worker through the object store. Customer hit this on argon2-cffi==25.1.0 (missing argon2/_utils.py), and previously on botocore/httpx (truncated tars). Symptom is a runtime ImportError that looks like a missing dependency declaration rather than a Windmill bug. Three changes that together stop the propagation: 1. After `pull_from_tar`, parse the wheel's `<dist-info>/RECORD` and confirm every listed path exists on disk before writing `.valid.windmill`. On failure, wipe the directory and fall through to a fresh local install — the next install also self-heals the broken object-store entry by pushing a fresh tar. 2. After `uv pip install` succeeds, run the same RECORD check before queuing the piptar upload or writing `.valid.windmill`. A bad install never becomes the source of a broken tar in the object store. 3. Finalize the tar (`drop(tar.into_inner()?)`) before reading its bytes for upload, so we never push an unfinalized archive (no end-of-archive marker) to the object store. Verified with a 60-package end-to-end integration test (first-fill → clear-local-cache → re-pull-from-objectstore → corrupt-objectstore-tar → detect-and-self-heal). All 27 packages on the live test pulled cleanly, and the deliberately corrupted argon2-cffi tar was caught with the exact expected log line ("wheel RECORD lists files missing on disk: argon2/_utils.py") and replaced with a fresh tar. Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * chore(main): release 1.699.0 (#9082) * chore(main): release 1.699.0 * Apply automatic changes --------- Co-authored-by: rubenfiszel <275584+rubenfiszel@users.noreply.github.com> * feat(cli): auto-infer args for `wmill app push` (#9091) Run `wmill app push` from inside an app folder (e.g. `f/foo/my_app.app/`) with no args. The local path defaults to CWD, and the remote path is derived from CWD relative to `wmill.yaml`, with `.app`/`.raw_app`/ `__app`/`__raw_app` suffixes stripped. Either, both, or neither positional argument can be passed. Also resolves `file_path` against the user's original CWD before `resolveWorkspace` may chdir to the wmill.yaml root, so a relative `file_path` argument is interpreted from where the user invoked the command (previously it could resolve against the wrong directory). Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * all * fix(pipeline): live-update graph for annotations and body assets * fix(pipeline): persist draft body edits across node switches * fix(pipeline): persist live writes per draft to keep output node fresh after switch * feat(pipeline): animate graph edges only while a runnable is executing * feat(pipeline): add run button on script nodes + recomputing hint on preview * feat(pipeline): compact preview layout, two-way Test/Run sync * fix(pipeline): test button cross-browser placement (no overflow trick) * style(log-viewer): replace took/mem-peak labels with timer/cpu icons * style(log-viewer): hyphenate Auto-scroll label and prevent wrapping * style(log-viewer): lowercase auto-scroll label, force vertical scrollbar * style(log-viewer): force horizontal scrollbar instead of vertical * fix(log-viewer): scope overflow-x to top bar so pre doesn't drive panel width * fix(pipeline): overlay live body-asset writes for persisted scripts too * fix(pipeline): persist inferred body assets at save so edges survive page reload * fix(pipeline): snapshot live draft writes at persist time so they survive reload * fix(pipeline): keep inferred body writes on the canvas across selection changes * fix(pipeline): untrack inferredWrites cache mutation to break effect loop * fix(pipeline): refetch asset graph after persisted-script save * feat(pipeline): optional AI prompt when creating a pipeline script * all * all * test: cover asset-trigger dispatch end-to-end through worker * feat(pipeline): split-button Test with optional downstream cascade * feat(pipeline): cascade option on graph Run + match button heights * style(pipeline): match caret bg/text to Test button's accent-secondary * feat(pipeline): split Run pill on graph node exposes cascade option * feat: live run activity + status badges in pipeline asset graph - folder-scoped queue poll lights up the downstream asset-trigger cascade (not just the launched script); zero requests at rest, catch-up for fast hops, auto-disarm when idle - per-runnable node badge: last-run status + session run count - animate unsaved/live-parsed edges (was unconditionally suppressed) - background-pane click no longer clears selection - run-bridge guarded so node selection/save no longer triggers a test Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * feat: live activity log, optimistic badges, node-avoiding graph edges - collapsible folder activity log (PipelineEventLog): live job feed, polls only while open/active, slow idle cadence, capped + pruned - composable: observe mode + events list + run-count anchored to graph-open time (pre-existing history excluded) - optimistic node badge: launched script shows running instantly via the zero-latency activeRunnable hint, keeps the polled run count - activity pane height capped (min(18rem,40vh)) then scrolls - route asset-graph edges through sugiyama-computed waypoints so they go around nodes instead of under them; bezier fallback for adjacent-layer / draft-overlay edges Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * feat: prefetch all folder script assets so graph is stable on load On pipeline load, eagerly infer body assets for every persisted folder script and seed the existing inferredWritesByPath overlay, instead of only filling it when a node is selected. Scripts whose persisted asset rows are missing (e.g. object-form writeS3File) now have their edges from first paint, so clicking a node no longer re-layouts the graph. One-shot per (workspace, base-graph) load, untracked map reads, generation-cancelled, pool-capped fetches. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * perf: guard no-op poll re-layout; dedupe write-asset extraction - skip reactive ids/states/events reassignment when unchanged, so an idle poll tick no longer re-runs the full sugiyama layout every 3-6s - bound countedJobIds (rebuilt from eventsById in lockstep with prune) - extract shared extractWrites() helper, replacing 4 copy-pasted write-asset filter/map blocks in the pipeline page - compute activeRunnable node-id once, reuse for the active-edge set and the optimistic badge (flattened ternary); trim narrating docs Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * feat: live read-lineage overlay for inferred body assets Renaming e.g. duckdb read_parquet('s3://...') / loadS3File now updates the asset->reader edge live instead of only after Save re-derives the persisted asset rows. - extractReads() (+ shared refsByAccess) mirroring extractWrites - inferredReadsByPath sticky cache, filled by handleAssetsChange and the load prefetch alongside writes - replace the write-only overlay loop with one overlayLineage(map, access) helper invoked for both 'w' and 'r' (net DRY) Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * fix: detect S3 assets passed as SDK object arg in ts parser Mirrors merged PR #9181 so feat/asset-graph-view is self-contained (local origin/main is stale and lacks it). Object/{ s3, storage } form of writeS3File/loadS3File is now detected, not only the bare s3:// string literal. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * chore: regenerate wasm Cargo.lock + frontend package-lock Lockfile churn from local wasm-pack (asset target) + npm operations during the asset-graph work. No source/dependency-intent change. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * fix: revert to bezier graph edges; add parsing-assets hint The sugiyama-waypoint routing looked worse than the original; revert AssetGraphEdge/assetGraphLayout to the pre-routing bezier logic (same as the flow editor's BaseEdge) and drop the now-unused route plumbing from the canvas. Add a small 'Parsing assets…' hint shown while the load-time prefetch sweep is still inferring folder scripts. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * refactor: extract pure resolveGraph merge + unit tests Move the ~230-line graphWithDraft precedence/merge (base < session- inferred < draft-seeded < open-script-live, +read/write/annotation overlays, +dedup) out of the 1648-line route into a pure, testable resolveGraph() module; the route's graphWithDraft is now a thin $derived. Behaviour extracted verbatim. 10 unit tests cover the precedence matrix. Phase 1 of the state/render split. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * style: graph controls top-right, lift minimap, hide Save when unchanged Controls -> top-right horizontal, no lock toggle; MiniMap !mb-10 so it clears the activity bar; hide the per-script Save button when the script is already at its latest save point (drafts still show Create). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * fix: scope runtime-asset prune by id to spare static lineage rows prune_runtime_assets deleted by (workspace_id, path, kind) tuple, so trimming surplus usage_kind='job' rows for an s3 path also wiped the static usage_kind='script'/'flow' producer rows for the same path — silently breaking the asset-trigger cascade (fetch_producer_writes found no writes; downstream never dispatched; required band-aid re-syncs). Delete the surplus job rows by id instead; the inner query is already scoped to usage_kind='job'. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * fix: don't re-pulse already-running jobs after they finish The catch-up pulse re-added a completed job to the active set if its start was within the (lagging) lookback window — even one we'd already animated the whole time it ran — keeping its edges lit ~a poll interval past completion (~5s after a 3.5s test). Track job ids seen in-flight and skip the pulse for them; it still fires for hops whose whole lifetime fell between two polls. Bound the set in lockstep with eventsById; cleared on dispose. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * fix: don't catch-up-pulse the runnable launched from the graph If the poll never sampled a launched run's in-flight window, the catch-up pulse re-flashed its edges one tick after it correctly stopped (the page already animated it zero-latency via activeRunnable). arm(launchedId) records the launched runnable id; catch-up skips it. Cascade hops (other ids) still pulse. launchedIds cleared on stop. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * style: nudge graph controls left to clear panel toggle Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * feat: partition value resolver + asset-cascade propagation windmill-common/partition: pure resolver — time kinds (tz/format/start anchor) + dynamic $.a.b JSONPath; 9 unit tests. asset_dispatch: read the producer's resolved partition and thread it into every cascaded subscriber's args + trigger.partition, so a chain resolves once at the top. No migration (cascade needs no spec lookup). Stage 1+3 of pipeline partition runtime; run-start resolution is Stage 2. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * feat: show args form in compact pipeline preview when script has inputs AssetGraphDetailsPane keeps the compact (hideArgs) preview but, via a new previewPanel.argsAboveLogs flag, renders a compact SchemaForm between the floating Test button and the logs/result panel when the script declares inputs (e.g. a partitioned script needing a `partition` arg). The preview pane also grows ~18pts so the args form doesn't shrink logs/result. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * feat: parser join-mode (`// trigger all`) + script_trigger.join_all Stage A: JoinMode{Any(default),All} + `// trigger any|all` directive in parse_pipeline_annotations; TriggerSpec::is_partition_bearing() (path contains {partition}); join_mode threaded through all 4 asset-parser crates (ts/py/sql/yaml). Stage B: reversible migration adds script_trigger.join_all; insert_script_trigger writes it; deploy path sets it from the parsed annotation. No reader yet (AND-join dispatch is the next stage) so runtime behaviour is unchanged. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * feat: resolve pipeline partition at job execution time Stage C: in handle_code_execution_job, once the script content is loaded, parse the // partitioned annotation (free here) and resolve the concrete partition once — schedule fire-time (scheduled_for anchor, not wall-clock) for time kinds, triggering payload for dynamic. The value is injected into the in-memory args the body sees (via a shadowed job clone) and persisted back to v2_job.args so dispatch_asset_triggers propagates the same value down the cascade. Already-set (explicit/backfill/cascade) partitions are never re-resolved (run identity immutable); unresolvable partitioned runs fail with a clear error. Integration test exercises the full worker loop + cascade propagation. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * feat: AND-join barrier for partitioned pipeline subscribers Stage D: a // trigger all subscriber no longer fires on any input. New join_pending_inputs slot table keyed (workspace, subscriber, partition); fetch_subscribers now returns join_all and the dispatch loop records each partition-bearing input arrival, pushing the subscriber once only when every partition-bearing input it declares is present for that partition. Per-partition slots, cleared on fire (re-accumulate, no double-fire), skew-immune (unlike debounce). Case-3 guard: an unpartitioned producer or a reference (non-{partition}) input never fires a partitioned join. Integration test covers wait/fire/isolation/no-double-fire. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * feat: opt-in // debounce for asset-cascade subscribers (parser + schema) Stage E1+E2. Parser: script-level // debounce <dur> + per-// on debounce=<dur> override (edge wins, else script default, else none = fan-out, unchanged); TriggerSpec::Asset carries the per-edge override; split_trailing_kv_opts separates the ref from trailing key=val opts. Schema/deploy: reversible migration adds script_trigger.debounce_s; parse_duration_secs (bare int or <n>s|m|h|d, fail-safe on garbage) resolves the effective per-edge window at deploy and writes it per row. No reader yet (dispatch wiring is E3) so runtime is unchanged. New unit tests for the parser directive and duration parsing. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * feat: apply opt-in debounce to asset-cascade subscriber dispatch Stage E3. fetch_subscribers now also returns debounce_s; push_subscriber builds real DebouncingSettings (delay + a (subscriber, partition) key, so distinct partitions never collapse and latest-in-window falls out) instead of ::default() when the edge opted in. Default stays no-debounce (fan-out — the prior deliberate behaviour, now overridable rather than reversed). Wiring test asserts the dispatched job carries the configured window/key and an undebounced edge carries none. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * fix: atomic AND-join gate + preserve resolved partition; drop scratch artifacts Addresses local-review findings before PR: - P1: record_and_check_join_slot was a non-atomic check-then-act on a pooled connection; concurrent completion of a subscriber's last two partition-bearing inputs on different workers could double-dispatch. Now one transaction guarded by a tx-scoped advisory lock keyed on (workspace, subscriber, partition) so the gate fires exactly once. - P2: the preprocessed-args overwrite in result_processor replaced args wholesale, dropping a partition resolved by resolve_partition_for_job; the UPDATE now preserves an existing persisted partition key. - P2: gate resolve_partition_for_job on a cheap code.contains check so non-pipeline script jobs skip the annotation scan on the hot path. - P2: remove 40 scratch screenshot PNGs, a flicker-debug script and a local scheduler lock accidentally committed; gitignore the lock. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * test: AND-join fires once under concurrent upstream completion Regression for the check-then-act race fixed by the advisory-locked transactional gate: releases N producer dispatches simultaneously via a barrier and asserts the AND subscriber is pushed exactly once and the slot is cleared. The invariant holds for the correct gate regardless of interleaving; a non-atomic regression fails it. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * test: fuller partitioned join + multi-hop pipeline coverage Exercises a complex pipeline combining options end to end: two partitioned producers fanning into a // trigger all join, then a multi-hop downstream chain. Asserts the resolved partition propagates unchanged at every hop, chain depth increments per hop, the AND barrier fires exactly once, and a second partition opens an independent slot with no cross-partition bleed across the whole graph. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * refactor: simplify pipeline code per review (dedup, single-parse, constant) - ParseAssetsOutput::new() collapses the 6-line annotation copy-paste across the 4 asset-parser crates to one call site. - asset_dispatch: parse the cascade trigger object once and pass it to the depth/partition readers instead of deserializing it twice; add a TRIGGER_ARG constant for the previously stringly-typed key (3 sites). - scripts deploy: drop a redundant debounce_default clone. No behavior change; 29 parser + 6 dispatch integration tests green. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * feat: reap abandoned AND-join slots after a TTL (default 60d, per-slot) join_pending_inputs slots are normally cleared when the join fires; partial slots whose inputs never all arrive (upstream removed/renamed, one-off dynamic partition key, permanent skew) would otherwise leak. windmill_queue::asset_dispatch::reap_stale_join_slots, called from the monitor's delete_expired_items loop, deletes a (workspace, subscriber, partition) slot only when its MOST RECENT row is older than JOIN_SLOT_TTL_SECS (60d) — per-slot, never per-row, so a legitimately slow join is not corrupted mid-accumulation. Conservative default; per-join configurable TTL via the annotation is a planned follow-up. Test covers stale-reaped / fresh-kept / mixed-slot-kept. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * update * feat: path-less native trigger markers + missing-trigger placeholder * feat: pipeline // tag and // retry annotations + dispatch_event log * fix: derive test-pane min from split-axis dimension (height in bottom layout) * feat: show last run logs/result when a script node is selected * fix: backfill asset rows from script.assets for pre-feature scripts * feat: job-id link + dispatch popover above script log/result * style: drop 'dispatched' label, keep just the check icon * fix: drop tag picker from pipeline script editor (set via // tag annotation) * Nicer UI * refactor: move google ai proxy handling to windmill-ai (#9260) * refactor: add ai proxy execution mode * refactor: move google ai proxy handling * refactor: share google ai request building * fix: early return should consider failure_module result (#9241) * fix(flows): flag noLogs jobs and lazily resolve them in log panel (#9099) * fix(flows): flag noLogs jobs and lazily resolve them in log panel * fix appending to flag * fix: preserve WM_LOGS_SKIPPED sentinel on SSE/replay completion pickMoreCompleteLogs resolved both sentinel and undefined to '', so the SSE completion event (whose job field is fetched .without_logs()) would clobber the sentinel placed by flagSkippedLogs. The module log panel then saw '' instead of the sentinel, defeating the lazy-resolve path. Also wire onLogsResolved on the OutputPickerInner inline LogViewer so a lazy resolve writes back to flowStateStore.previewLogs, matching ModulePreviewResultViewer and avoiding repeated fetches on remount. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * chore(main): release 1.705.0 (#9229) * chore(main): release 1.705.0 * Apply automatic changes --------- Co-authored-by: rubenfiszel <275584+rubenfiszel@users.noreply.github.com> * chore: add playwright mcp for frontend verification (#9269) * feat: CLI datatable serve / psql (#9267) * feat(cli): add datatable list and run commands * feat(cli): render datatable query results as a table * feat(cli): serve datatables as a postgres-wire endpoint * feat(cli): add 'datatable psql' to launch psql against the proxy * feat(cli): route datatable serve by client-supplied database name * override database list + password option * fix: support extended queries in datatable serve * fix: correct cloud size threshold log and parse CLI descriptions with parens/trailing comma * refactor: extract raw_output envelope encoding into pg_raw_output module --------- Co-authored-by: claude[bot] <41898282+claude[bot]@users.noreply.github.com> * oom_adj nit * feat: add UV_PYTHON_INSTALL_MIRROR env and instance setting (#9271) * feat: add UV_PYTHON_INSTALL_MIRROR env and instance setting Allows operators to point `uv python install` at a private mirror of the python-build-standalone releases. Configurable via the `UV_PYTHON_INSTALL_MIRROR` env var or the `uv_python_install_mirror` instance setting, with the env var as the boot fallback and the instance setting taking precedence at reload. Fixes WIN-1966 Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * fix: hoist uv_python_install_mirror binding above sandboxing branch The non-sandboxed uv pip install branch referenced a binding that was only declared inside the sandboxed branch. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * fix: neutral placeholder for uv_python_install_mirror The previous placeholder was the default public URL the setting is meant to redirect away from. A neutral example mirror URL is clearer. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * fix(indexer): tell admins when ingress routes search to wrong pod (#9274) * [ee] fix(indexer): tell admins when ingress routes search to wrong pod When the IndexReader is absent on the pod handling a search request but another pod is actively holding the indexer lock, the EE handler now returns a tailored error pointing at the ingress/load-balancer configuration instead of the generic "indexer not running" message. The indexer status endpoint reads the DB lock so it reports "running" from any pod, but search endpoints need the in-memory IndexReader that only exists on the lock holder. In multi-replica deployments this looks like the indexer is healthy but every search 404s. Companion: windmill-labs/windmill-ee-private#TBD Fixes WIN-1968. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * chore: update ee-repo-ref to eb18d7b4c0e37fea3f6e1e2cc44e0fddd74ff817 This commit updates the EE repository reference after PR #586 was merged in windmill-ee-private. Previous ee-repo-ref: 7dd43d1850813071cc18ba49ba090583e7321f4b New ee-repo-ref: eb18d7b4c0e37fea3f6e1e2cc44e0fddd74ff817 Automated by sync-ee-ref workflow. --------- Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com> Co-authored-by: windmill-internal-app[bot] <windmill-internal-app[bot]@users.noreply.github.com> * feat(cli): add `wmill init prompts` and custom override slot (#9266) * feat(cli): add `wmill init prompts` and custom override slot Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * refactor(cli): replace init prompts with refresh prompts + AGENTS.md/AGENTS.cli.md split Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * feat(cli): dedupe claude skills via @-includes and add prompts freshness check Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * refactor(cli): drop migration-choice flags from `refresh prompts` Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * docs(cli): add 'Running and previewing local changes' section to AGENTS.cli.md Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * fix(cli): write full skill content to .claude/, drop @-include wrapper Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * feat(cli): reconcile CLAUDE.md the same way as AGENTS.md Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * fix(cli): address PR review nits — argv parsing, lazy import, comment detection, error propagation Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * feat: add yolo mode for ai chat tools (#9258) * feat: add yolo mode for ai chat tools * nit * fix: align chat footer controls * feat: add ai chat autonomy modes * feat: add autonomy mode dropdown * fix: highlight yolo autonomy icon * fix: auto accept flow edits * fix: hide unsupported autonomy modes * fix: handle auto-accept flow editor races * fix(debugger): add non-root user support to Dockerfile (#9277) Mirrors the main Windmill Dockerfile pattern: creates a windmill user (UID/GID 1000) and makes cache/work directories world-writable so the image runs cleanly under Kubernetes securityContext.runAsNonRoot or runAsUser: 1000 without permission errors on Bun, pip, or windmill cache writes. Fixes WIN-1969 Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * fix(ai): enforce RLS and scope check on user-supplied X-Resource-Path (#9276) * fix(ai): enforce RLS and scope check on user-supplied X-Resource-Path The AI proxy handler accepts an X-Resource-Path header to override the configured workspace AI provider. When supplied, the handler loaded the resource value from the resource table using the root DB pool with no resources:read scope check, so any authenticated workspace user could point X-Resource-Path at a restricted AI resource (e.g. one in a folder they cannot read) and the proxy would use that resource's provider credentials for the outbound AI request. For user-supplied resource paths, now require resources:read:{path} scope and fetch the resource through user_db.begin(&authed) so RLS enforces the same folder/group boundary as the resource API. The RLS- scoped $var: resolution stays in place as defense in depth. The admin-configured workspace/instance ai_config path is unchanged. Fixes WIN-1971 Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * test(ai): regression test for X-Resource-Path RLS enforcement Cover all four cases: - non-admin pointing X-Resource-Path at a restricted resource is rejected - non-admin pointing it at a resource they own still works - admin can point it at any resource - workspace-configured proxy flow (no X-Resource-Path) is unchanged Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * feat: add userdraft listing primitives (#9268) * feat: add userdraft listing primitives * fix: cancel stale userdraft discard writes * docs: remove global ai userdraft plan * feat(nsjail): optional disk-backed /tmp via instance setting (#9272) * feat(nsjail): optional disk-backed /tmp via instance setting * test(nsjail): unit-test tmp mount resolver and narrow visibility * refactor(nsjail): switch tmp backing to select + conditional UI * ui(nsjail): make tmpfs the visible default in /tmp backing select * fix(nsjail): refuse preexisting jail_tmp to block symlink escape * fix(nsjail): allow jail_tmp reuse on sequential nsjail calls Codex flagged that python/ruby/rust executors invoke nsjail twice per job_dir (install then run). The previous resolver treated any preexisting jail_tmp as hostile and silently fell back to tmpfs on the second call, so disk-backed mode never reached the main script run for those langs. Use symlink_metadata().is_dir() to distinguish a real directory left by an earlier call in the same job_dir (safe to reuse) from a symlink or other entity (still refused, as the codebase-tar escape requires). Also loosen the frontend visibility predicate: only hide nsjail settings when job_isolation is explicitly 'none' or 'unshare', so deployments that enable nsjail via DISABLE_NSJAIL=false with no DB setting can still see the controls. * chore(main): release 1.706.0 (#9270) * chore(main): release 1.706.0 * Apply automatic changes --------- Co-authored-by: rubenfiszel <275584+rubenfiszel@users.noreply.github.com> * fix(nsjail): gate unix-symlink test behind cfg(unix) for Windows build (#9280) The disk_backed_refuses_preexisting_symlink_at_jail_tmp test calls std::os::unix::fs::symlink directly, which doesn't exist on Windows targets. Without a cfg gate, `cargo check --tests` fails on Windows with E0433. Other symlink call sites in this crate (php_executor, bun_executor, rust_executor, etc.) already follow this pattern. Fixes WIN-1972 Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * Reduce slim image vulnerability surface (#9279) * Reduce slim image vulnerability surface * chore(docker): drop apt-get upgrade -y from slim images apt-get upgrade hurts build reproducibility (same Dockerfile + same commit at different times produces divergent images) and trips hadolint DL3005. The freshness it buys is dominated by simply rebuilding against the periodically-refreshed debian:bookworm-slim base image. The --no-install-recommends and apt-list cleanup wins are kept. --------- Co-authored-by: Ruben Fiszel <ruben@windmill.dev> * fix(git-sync): bump to hub/28234 with stateless gpg.program wrapper (WIN-1974) (#9282) * fix(git-sync): revert LATEST_GIT_SYNC_SCRIPT_PATH to hub/28230 to restore GPG-signed deploys (WIN-1974) hub/28231 (PR #9230) is the "thin" script that hands the actual `git commit` to the CLI's hidden `sync git-deploy`. The hub script still does the GPG setup (import key into a fresh GNUPGHOME, dummy `gpg -bsau` to warm the agent passphrase cache, then `git config user.signingkey` + `commit.gpgsign` locally), but the commit no longer runs in the same `git_push` flow — it runs minutes later inside the CLI after workspace API resolution, zip pull, file extraction, and lockfile autofill. By the time the spawned `git commit` asks gpg-agent for the cached passphrase, the cache state is no longer reliable (or the spawned `gpg` ends up talking to a fresh agent), so signing fails non-interactively with `gpg failed to sign the data`. hub/28230 is hub/28217's in-script logic rebuilt with windmill-cli@1.703.3: the GPG setup and the in-script `sh_run("git commit ...")` happen back-to-back in `git_push`, so the cache is always fresh. It preserves wm_deploy / fork branch behavior, the EE deployment-callback `main()` signature is unchanged, and the only min-version check in EE (`is_script_meets_min_version(28103)`) is comfortably below 28230 — so this revert is safe. Forward fix (separate PR): publish a new thin script that, alongside the existing GPG setup, writes a `gpg.program` wrapper using `--pinentry-mode loopback --passphrase-file` so signing is independent of the agent's cache state. Re-bump past 28231 then. Fixes WIN-1974 Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * chore(git-sync): check in source-of-truth for the next hub script (gpg.program wrapper) This is the script that will be published to hub.windmill.dev once verified on a customer GPG-signed deploy. It replaces hub/28231's agent-cache pre-warm (`gpg -bsau` with --passphrase) with a stateless gpg.program wrapper + chmod-600 passphrase file. Every git-invoked gpg call goes through the wrapper, which always uses --pinentry-mode loopback (and --passphrase-file when a passphrase exists). Signing no longer depends on gpg-agent having a cached passphrase by the time the CLI's `git commit` runs — which closes WIN-1974. Not wired in yet: LATEST_GIT_SYNC_SCRIPT_PATH stays on hub/28230 until this script is uploaded and the new hub id is known. This file is checked in so the diff is reviewable, future bumps have a source of truth, and a CLI regression test can `cat` it for fixture parity. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * fix(frontend): skip format/pattern validation for $var/$res/$jsonvar references in ArgInput A resource field with a `pattern` constraint (e.g. the gpg_key.private_key field, whose pattern enforces a `-----BEGIN PGP PRIVATE KEY BLOCK-----` prefix) rejects values like `$var:u/me/gpg-private-key` with an "invalid format" error in the resource editor — even though `$var:`/`$res:`/`$jsonvar:` are placeholders the backend resolves at runtime, not the actual string that needs to match the regex. Bail out of all format/pattern checks (email, ipv4, ipv6, uuid, custom pattern) when the value is one of these references. Required/numeric bounds/array checks still apply since they're shape-level, not regex. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * feat(git-sync): bump LATEST_GIT_SYNC_SCRIPT_PATH to hub/28234 (gpg.program-wrapper fix) hub/28234 is the forward fix for WIN-1974: replaces hub/28231's agent-cache pre-warm (which became stale by the time the CLI's `git commit` ran) with a stateless `gpg.program` wrapper that uses `--pinentry-mode loopback` (and `--passphrase-file` when a passphrase exists) on every gpg invocation. Bundled CLI is windmill-cli@1.705.0. Verified via reproducer at /tmp/git-sync-diff/test-gpg-fix.sh: deliberately killing gpg-agent between GPG setup and `git commit` reproduces the customer's `gpg failed to sign the data` error verbatim under the old flow, and the wrapper signs through it. Holds for passphrase-protected keys, split-subkey [C]+[S] layouts, and unprotected keys. Drops the local source-of-truth copy (`hub-scripts/`) — hub is canonical now that 28234 is published. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * chore(git-sync): drop verbose comment above LATEST_GIT_SYNC_SCRIPT_PATH The git history (this PR) carries the why; the constant name + value carry the what. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * refactor(cli): wmill sync git-deploy stops committing; caller owns commit+push (#9284) Single contract for the deployment-callback path: the CLI does branch checkout + pull, the caller (hub script in production, test in test) does git add + commit + push. This restores the WIN-1974 invariant — GPG setup and `git commit` run back-to-back in the same process, so the agent's pre-warmed passphrase cache is still warm at sign time — without needing a `--skip-commit` flag for the hub case and a default "also-commit" for everything else. Same behavior in every call site. Changes: - sync.ts: drop the gitSyncDeployPush call from pull()'s deploy path (both the onlyCreateBranch fast-return and the post-pull commit). `gitSyncDeployPush` stays exported for any caller that wants the same commit/push semantics — just not invoked by the CLI subcommand. - gitsync_promotion.test.ts: e2e test now does its own git add + commit + push after `wmill sync git-deploy`, mirroring what the hub script does in production. Same regression coverage (wm_deploy branch created in Case A, main untouched; main updated in Case B, no new wm_deploy). CLI typecheck unchanged (two pre-existing TarAsZip errors at lines 2578/3307, present before this PR). All 743 unit tests still pass. The accompanying hub script (option-C — CLI for branch+pull, script for commit+push) lives at /tmp/git-sync-diff/sync-script-to-git-repo-windmill.option-C.ts. Once published, a follow-up bumps LATEST_GIT_SYNC_SCRIPT_PATH to its id. Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * bump git sync to 28236 * fix: fork compare visibility for non-admins and stale-token superadmins (#9283) * fix: use fork-scoped authed for fork visibility in compare_workspaces * test: add EE end-to-end repro for fork rename visibility * chore: restore concurrency_locks sqlx cache lost in cleanup * test: add regression for stale-superadmin-token fork visibility bug * chore: update sqlx cache for new test queries * chore(main): release 1.706.1 (#9281) * chore(main): release 1.706.1 * Apply automatic changes --------- Co-authored-by: rubenfiszel <275584+rubenfiszel@users.noreply.github.com> * feat: add wmill job rerun subcommand (#9275) * feat: add wmill job rerun subcommand * feat: add wmill job restart subcommand for flow restart-at-step * chore(system_prompts): point plugin skills sync at plugins/windmill/ (#9287) * chore(system_prompts): point plugin skills sync at plugins/windmill/ The plugin checkout's plugin folder is being renamed from `plugins/windmill-code-plugin/` to `plugins/windmill/` to shorten the slash-command namespace and align with the matching Cursor plugin layout. Paired with windmill-labs/windmill-claude-plugin#8. That PR must merge first so the next sync run finds the new folder. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * docs(system_prompts): update plugin-dir example to plugins/windmill Co-authored-by: centdix <centdix@users.noreply.github.com> --------- Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com> Co-authored-by: claude[bot] <41898282+claude[bot]@users.noreply.github.com> Co-authored-by: centdix <centdix@users.noreply.github.com> * fix(cli): wmill sync pull updates wmill-lock.yaml for raw apps (#9289) * fix: flow recording teardown crash + rename package to @windmill-labs/components (#9288) * fix: guard against null recording during FlowRecordingReplay teardown Navigating away from a flow recording inside a workspace file-tree view threw `TypeError: Cannot read properties of null (reading 'flow')` from FlowGraphViewer once during the teardown tick. Svelte 5 compiles child component props as live getters that close over `$$props.recording.flow`. When `recording` flips to null on the parent's navigation, an outer `{#if !recording?.flow}` doesn't stop those getters from firing one more time as derived effects re-evaluate before the unmount lands — so the getter dereferences null and throws. Fix at the two layers where the deref actually happens: - FlowRecordingReplay: use `recording?.flow` at the binding sites (FlowViewer + graph-snippet FlowGraphViewer) so the compiler emits an optional-chained getter, and guard the snippet branch with `{:else if recording?.flow}` so it doesn't mount when there's nothing to show. - FlowGraphViewer: finish the optional chaining the rest of the file already used everywhere else (`flow?.value?.skip_expr`, `flow?.value?.cache_ttl`, `flow?.schema`). When the upstream binding returns undefined during teardown, the graph degrades to an empty frame instead of crashing. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * chore: rename package to @windmill-labs/components - frontend/package.json: rename `windmill-components` → `@windmill-labs/components` - frontend/publish.sh: drop the in-place sed rename dance; the checked-in name now matches what's published, so `npm run package && npm publish` is enough - frontend/package-lock.json, system_prompts/auto-generated/prompts.d.ts: regenerated by `npm run package` under the new name Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * default script name * save logic * Keyboard nav * finish keynav * nits * CI fix * nit stop propagation * Merge branch 'main' into feat/asset-graph-view * commit * update * fix: cropped save button on small screens * progress * managed scheduled removed * all * progress * feat: add data upload pipeline trigger with auto S3 picker Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix: avoid pane editor remount flicker when deploying a pipeline draft Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix: show only the edited script's I/O in the asset graph, not the saved version's Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat: derive script asset rows server-side at deploy Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * test: shared fixture corpus keeps annotation parsers in parity Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * feat: dev-run draft pipeline chains, live badges, deploy drift warning Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * refactor: ungate cascade producers, squash pipeline migrations Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * chore: drop committed cli-sync fixtures and stray screenshots Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * feat: show skip-asset-dispatch flag as badge instead of args row Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * feat: pipeline view mode default with activity feed, drafts overlay chip Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix: treat DROP TABLE as table-level write in sql asset parser Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * feat: wmill datatable create + actionable sql extension error Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * chore: ephemeral data-pipelines demo sync repo zip for handoff Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * feat: wmill pipeline list/show renders the asset DAG in the terminal Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * nits * nits * nits * nits * fix: defer draft persist-back past the batch so discard sticks first click Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * feat: band-reserving tidy-tree asset graph layout with join breakpoints Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix: route skip-layer and long graph edges around occupied columns Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix: seed s3 template outputs with canonical leading-slash paths Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * all * feat: bundle data-pipeline drafts into the DB-backed user draft system Pipeline drafts were browser-only (localStorage `pipeline-<folder>`), so they didn't sync across devices, weren't server-visible, and never showed in the drafts list. Store them instead as one per-user `draft` row of a new `data_pipeline` kind, keyed at the folder (`f/<folder>/data_pipeline`), holding the same `{ drafts, activeDraftPath }` bundle. Stage 1 — backend kind: add `data_pipeline` to DRAFT_KIND (migration) and `UserDraftItemKind` (deployed_table=None, private). The list/update handlers and folder-path access check already cover a backing-table-less kind. Stage 2 — sync: add `GET /drafts/get_own/{kind}/{path}` so an editor with no deployed-overlay GET can load its own draft. The pipeline page now hydrates from the DB on mount (one-time localStorage import for in-flight drafts) and persists via UserDraftDbSyncer (debounce + optimistic-concurrency), keeping a localStorage crash mirror. Stage 3 — surface: the drafts review page renders the bundle as a "pipeline" row that opens `/pipeline/<folder>` (open-only; excluded from bulk deploy). Verified end-to-end in-browser: DB-seeded draft hydrates to "Edit (1)", edits persist back, and the row shows with Open pipeline / Discard. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat: pipeline Activity panel grouping, run↔graph highlight, deploy-conflict handling Activity panel (view mode): - Group cascade runs by the connected component of the asset-dispatch graph (new GET /jobs/asset_dispatch_edges over the dispatch_event table, incl. join_pending inputs), headed by the earliest originating run + its trigger, with a "+N" chip for joins fed by multiple triggers. - Success/failure count histogram with drag-to-filter brushing, an always-on time axis + per-bar tooltips, a Reset, and Last hour/24h/48h/7/30/90d ranges. - Node run-count/status badges now derive from the same merged historic+live events the panel shows (previously session-only). Run ↔ graph highlight: - Hovering a run row (or a group header → the whole cascade) rings the node(s), animates their incident edges, and borders the adjacent assets in the edge hue (blue write / gray read); expanding a run pins a soft-blue ring. - Switching edit→view re-surfaces the Activity feed. Deploy: - Live-content autosave for the open pipeline draft + an autosave indicator. - Re-saving a script now chains off the hash just created instead of a stale parent_hash (fixes the "lineage must be linear" error on a second save), and a genuine concurrent deploy opens a keep-mine / view-latest conflict modal. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix: pipeline editor badge requires asset-parse, not just main-function parse A pipeline script's asset lineage is load-bearing — a deploy that can't parse assets silently records no edges. The editor "parsable" dot only reflected inferArgs (the main function), so a body the asset parser rejects (e.g. a trailing `/////` in DuckDB) still showed green and deployed with empty lineage. ScriptEditor gains `requireValidAssets` (set by the pipeline pane); when on, the EditorBar badge is green only if BOTH the main function and inferAssets parse, with the tooltip distinguishing "Main function not parsable" / "Assets not parsable" / "Parsable". Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat: route asset-graph edges around nodes that sit in their path Edges could draw straight through an unrelated node (a join fan-out or long cross-component edge), making it ambiguous whether that node shared the input. AssetGraphEdge only saw its own endpoints, so it could only detour the near-vertical same-column skip case. The canvas now (once per layout, O(edges × nodes) — no per-frame cost) samples each edge's straight run against every non-incident node center and, on a crossing, passes a clear gutter lane to the edge via `data.detourX`; AssetGraphEdge routes the rounded-orthogonal detour through it. Verified: 0 edge↔node box crossings on the orders pipeline. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix: deploy pipeline drafts with freshly-inferred assets, not a stale snapshot "Save all" spread `...draft.script` into createScript, which carries a `assets` snapshot that isn't refreshed when the body is edited. So a renamed/removed output (e.g. an old `CREATE TABLE exciting_en32z9` later changed to `exciting_880909`) was re-deployed as a phantom write edge and lingered as an orphan asset on the graph — shown with no producer, and shifting position on click as the graph re-derived. saveDraft now re-runs inferAssets on the current body and passes the result as `assets`, overriding the snapshot — mirroring the per-pane save. The backend clears+reinserts from the sent set, so a re-deploy drops the stale rows. Verified: deploying with the fresh asset set removes the orphan from the graph. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix: collect upstream reads from CTAS and CREATE VIEW in SQL asset parser `CREATE TABLE x AS SELECT … FROM y` (and `CREATE VIEW`) recorded only the write to x — the source read of y was silently dropped. Table-level reads are gathered in the `Statement::Query` arm via handle_table_with_joins; the generic table-factor visitor only picks up read-functions and string literals, not plain `FROM <table>` references. The AS-query of a CTAS isn't a `Statement::Query`, so its FROM tables were never walked. On the pipeline canvas this meant a `datatable://…` upstream consumed by a CTAS step showed no read node/edge — the step looked like it produced its output from nothing. Factor the Query arm's read collection into handle_query_reads and call it from the CreateTable (when it has an AS-query) and CreateView arms, balancing the cte_name_stack push in post_visit_statement. Updated the drop_then_create test (which had pinned the old drop-the-read behavior) and added CTAS + CREATE VIEW read coverage. Verified against the rebuilt asset wasm: the live editor now infers the read. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * update * updates * refactor: dedup asset-graph code, squash migrations, drop artifacts Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * perf: gate asset dispatch on a cached per-workspace producer set Cache the producer-path→writes map per workspace and invalidate it from the asset-clear paths via the notify_event polling system, so a top-level script/preview completion that isn't an asset producer costs an in-memory lookup instead of a per-completion query. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix: remove dead unquote fn that failed backend check under -D warnings Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix: green the frontend check (pin published wasm-asset, fix type errors) Pin windmill-parser-wasm-asset to the published 1.728.1 (was a file: link to a gitignored, CI-unbuilt pkg-asset). Exclude test files from svelte-check (the parity test reads a backend fixture via node:fs, which the browser app tsconfig has no @types/node for; vitest still runs them). Fix pre-existing branch type errors: drop the unsupported 2nd getScriptByPath arg, cast script.schema to Schema for inferArgs, coerce has_preprocessor to a definite boolean, and wrap the cancelJob handler so it isn't possibly-undefined. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat: move pipeline partition resolution to ee-private (free-CE) Partition resolution becomes a private module (partition_ee in windmill-ee-private, hidden from the public repo) with an OSS no-op fallback (partition_oss); call sites resolve via the aliased windmill_common::partition. Not enterprise-gated — free to run in CE. Bumps ee-repo-ref to the ee branch carrying partition_ee. Verified building in default, private, and private,enterprise (offline). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat: move asset-cascade join/debounce/retry to ee-private (free-CE) Join barrier, debounce, and retry become the private windmill_queue::cascade module (cascade_ee in windmill-ee-private); OSS gets cascade_oss no-op fallbacks (plain OR fan-out). Core cascade stays public. Bumps ee-repo-ref. Verified default/private/private,enterprise. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat: skeleton enterprise pipeline freshness + backfill (TODO, ee-private) Gated windmill_common::pipeline_advanced (private; pipeline_advanced_ee) with OSS fallback; entry points return a clear not-implemented error. Deploy surfaces a TODO when a script declares // freshness. Bumps ee-repo-ref. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix: repair asset_trigger_dispatch test after cascade carve-out + cache its queries Stage-2 moved reap_stale_join_slots to windmill_queue::cascade; update the integration test's import. Also commit the test's sqlx query cache (was never prepared with --tests, so SQLX_OFFLINE cargo test failed pre-existing). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * test: invalidate producer-cache in asset dispatch tests (mirror deploy) The tests seed asset rows directly and run no notify poller, so the per-workspace producer cache went stale across tests → 0 dispatched. Clear it at the seed point, as a deploy would via notify_event. All 8 asset_trigger_dispatch tests pass. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * chore: update ee-repo-ref to ba677ea142011462ad4dfe77e8375a6dd274cdef This commit updates the EE repository reference after PR #619 was merged in windmill-ee-private. Previous ee-repo-ref: 925c350cff55d3ea738d9e2e4098d9ce4bdda418 New ee-repo-ref: ba677ea142011462ad4dfe77e8375a6dd274cdef Automated by sync-ee-ref workflow. * test: disable producer cache in asset dispatch tests (isolated-DB safe) The .remove(WS) approach still raced: #[sqlx::test] gives each test its own DB but they share one workspace id, so the WS-keyed process-global cache clobbered across DBs under concurrent threads. Add an ASSET_PRODUCER_CACHE_DISABLED test hook and set it in the tests so every dispatch reads its own DB. 8/8 pass at --test-threads=10. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat: replace asset-cascade depth cap with cycle detection The hardcoded MAX_CHAIN_DEPTH=5 truncated legitimate deep pipelines (silently — the check returned before event logging). Replace it with per-edge cycle detection: carry the producer lineage in trigger.chain and skip only a subscriber already in the chain, recording a visible cycle_detected dispatch_event. Acyclic pipelines of any depth now cascade fully; a high MAX_CHAIN_LEN backstop guards against runaway. Tests + UI label updated; 8/8 pass at --test-threads=10. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * docs: update dispatch_event reason examples (depth_cap → cycle_detected) Comment-only; the migration is idempotent and already in the potentially_stale self-heal list, so the checksum change re-applies cleanly. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix: park cascade retry (P1 dead-end) + clear stale script_triggers on rename Two deploy-path fixes: - Retry is parked: a retried subscriber is wrapped in a SingleStepFlow, whose run is a flow step and ineligible for asset dispatch, so it would silently dead-end the cascade (P1). Stop persisting retry to script_trigger and warn at deploy; TODO(pipeline-retry) to re-enable once dispatch handles flow-wrapped producers. (Dispatch plumbing kept + still tested via direct seeding.) - Rename leaves stale script_trigger rows: clear was keyed on ns.path only, so old-path '// on' edges lingered and could trigger a script later recreated at that path. Also clear the old path on rename (assets already handled via the parent-hash clear). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com> Co-authored-by: rubenfiszel <275584+rubenfiszel@users.noreply.github.com> Co-authored-by: hugocasa <hugo@casademont.ch> Co-authored-by: windmill-internal-app[bot] <windmill-internal-app[bot]@users.noreply.github.com> Co-authored-by: Arnaud <31803803+Araden14@users.noreply.github.com> Co-authored-by: Diego Imbert <diego@windmill.dev> Co-authored-by: centdix <40307056+centdix@users.noreply.github.com> Co-authored-by: Diego Imbert <70353967+diegoimbert@users.noreply.github.com> Co-authored-by: claude[bot] <41898282+claude[bot]@users.noreply.github.com> Co-authored-by: Aldrin Jenson <aldrinjenson@gmail.com> Co-authored-by: centdix <centdix@users.noreply.github.com>	2026-06-18 18:09:02 +02:00
Ruben Fiszel	e1e2a24b6a	fix(flows): stop serializing default retry/stop_after_if fields (#9583) ... A flow module with a constant-only retry is stored by the frontend as `{ constant: {...} }`, but round-tripping through the `Retry` struct (e.g. the dependency/lock job, which re-serialises the flow value) materialised a full default `exponential` block (`seconds: 0`, `random_factor: null`) and a `null` `error_message`, because those fields are non-`Option` / `Option` without `skip_serializing_if`. The defaults then got baked into stored data, surfaced on `wmill pull`, and were rejected by the linter. Skip serialising `Retry.constant`/`Retry.exponential` when they equal their default, and `StopAfterIf.error_message` when it is `None`. Deserialisation is unchanged (`#[serde(default)]` refills the in-memory structs), so the worker retry logic and the frontend (which already optional-chains these fields) are unaffected. Verified end-to-end against a running backend: a flow created with an explicit default exponential block + `error_message: null` comes back clean after its lock job re-serialises it. Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>	2026-06-15 19:19:08 +02:00
Diego Imbert	1fc355709c	feat: Db-backed user drafts (#9351) ... * Db draft removal * refactor: drop unsaved-changes confirmation modal from editors * fix: remove nodraft from flow row edit link * fix: remove nodraft from app and raw app edit buttons * fix: remove nodraft from all edit links * fix: merge backend defaults into legacy autosaves to avoid spurious restore toast on raw apps * feat: add username column to draft table for user-scoped drafts * feat: add sync_drafts and list_users_with_draft_on_path endpoints * feat: add UserDraftDbSyncer service for bi-directional draft sync * feat: wire UserDraft.save through DbSyncer + conflict modal * refactor: gate useLocalStorageValue nested-update effect behind opt-in flag * refactor: move sync force flag from request-level to per-entry * feat: sync all userdraft kinds, switch draft owner to email FK, add id PK, scope draft list to readable paths * refactor: route draft permission check through authed.folders + RLS, drop client-supplied email * feat: support draft deletion via sync (value: null) with same conflict semantics * feat: surface other users' drafts in editors with diff+fork action * refactor: unify draft schema migrations and type kinds via DRAFT_KIND enum * perf: add (workspace_id, email, created_at) partial index for sync hot path * chore: update ee-repo-ref to a30079e75dc5b7d7413aa8ee20e40e80bfea9cbd This commit updates the EE repository reference after PR #597 was merged in windmill-ee-private. Previous ee-repo-ref: 55c19293232be379a3044eb78f677b545882ffd6 New ee-repo-ref: a30079e75dc5b7d7413aa8ee20e40e80bfea9cbd Automated by sync-ee-ref workflow. * fix(userdraft): trigger sync on deep mutations via readFieldsRecursively * Rollback UserDraft * remove queuing logic * pushDrafts * refactor: remove draft sync layer and conflict modal * feat: add save_draft, list_drafts, get_draft routes * feat: add get_draft overlay to getScriptByPath * feat: extend get_draft overlay to flow, app, resource, variable, schedule, triggers * feat: support null value in save_draft for deletes * readLastSyncMap * feat: redirect /add pages to /edit/draft_uuid with new_draft flag * fix: inline get_draft query field instead of flattening * fix: drop dangling nobackenddraft assignment in flows edit * feat: include user drafts in list endpoints with is_draft flag * fix: prefix draft paths with u/{user} and seed editor state on new_draft * fix: route draft-only deletes through UserDraftDbSyncer on home page * feat: delete user drafts when their underlying item is deleted * fix: empty path seed on new_draft so friendly auto-name fires * feat: re-add Draft and Draft only badges on home page rows * fix: synthesize value wrapper on draft-only raw_app response * fix: tolerate missing latest-version on draft-only flow reload * fix: skip first observable change in DB sync effect to match LS persist * fix: remove URL-hash sync from script editor (already marked TEMP) * refactor: drop localStorage layer from UserDraft * refactor: drop vestigial LS-era code from UserDraft * feat: migrate localStorage drafts to DB on layout mount * fix: migrate session runtime + script view to per-user draft API * feat: add 'Reset to deployed' action on draft-loaded toast * feat: hide 'Reset to deployed' action when no deployed version exists * createCoalescingKeyedRunner * example ts doc * createDebouncerByKey * refactor: drop await on draft-delete in reset flows, refetch deployed directly * fix: bridge saved-draft shape to wire shape in apps/resources/variables loaders * feat: route UserDraftDbSyncer.save through debouncer + coalescing runner * feat: add immediate-save bypass that cancels pending debouncer + runner tasks * fix: seed UserDraft cell from spec defaultValue on acquire * fix: redirect /add routes at load phase to eliminate white flash * fix: drop +page.js files in /add routes that conflicted with +page.ts * refactor: send draft as separate .draft field instead of deep-merging onto deployed * feat: surface draft path in home list when user typed one different from URL * feat: add UserDraft.stopSync/restartSync, wire on script + low-code app /add init * fix: thread URL path into ScriptBuilder.stopSync (was using empty initialPath) * fix: also stopSync in route's new_draft branch + queue pre-acquire suspensions * feat: add AutosaveIndicator backed by reactive UserDraftDbSyncer.getState * refactor: drop draft-loaded toast in non-route editors, banner now compares draft vs deployed * fix: gate per-user draft-only rows in listings on include_draft_only flag * feat: flush pending draft saves via keepalive fetch on tab hide / pagehide * autosave indicator nits * fix: route create-vs-update on /add deploys; seed policy.execution_mode; sync script template * chore: add [draft-sync] console logs to trace script bootstrap autosave * fix: seed auto-generated path in script new-draft route to suppress Path widget's autosave-triggering mutation * fix: defer script restartSync until script.path lands (Path widget gated on $userStore + $workspaceStore) * fix: poll script.path via tick() until Path widget settles before restartSync * chore: log inferArgs underlying error on deploy to diagnose 'Could not parse code' toast * fix: wait for script.path to stabilize across two ticks before restartSync * revert: drop unsuccessful path-stabilization heuristics + leftover [draft-sync] logs * fix: seed new-draft script schema as emptySchema() so inferArgs doesn't trip on undefined properties * fix: heal legacy drafts with schema={} (no .properties) on deploy * autosave indicator * refactor(editors): drop UnsavedConfirmationModal mount + Show diff button * feat(drafts): collaboration banner, cross-tab conflict detection, raw app template picker - Other-users-drafts banner (Modal2): the deployed-overlay response now carries `other_drafts_users` (workspace usernames only, never emails); each row offers View JSON + Fork. Drops the standalone `listUsersWithDraftOnPath` endpoint; `getDraftForUser` now takes a workspace `username` query param (resolved to email server-side). - Cross-tab/browser save conflict detection: the syncer attaches `last_sync` to every save (defaults to non-force); on a `conflict` response it parks a snapshot in a reactive map. Each route mounts a `DraftSyncConflictModal` and seeds the per-tab `last_sync` via `recordRemoteSync(query, draft_saved_at)` on every `get_draft` load. Keepalive flush also respects optimistic concurrency. - Raw app template picker re-added after the /add ⇒ /edit refactor: framework (React 19 / 18 / Svelte 5), data table + schema config, and optional AI prompt — extracted into `RawAppTemplatePicker.svelte` and driven by `new_draft=true` on the edit route. * fix(drafts): suppress autosave during /add template seeding on script + raw app editors - ScriptBuilder: delay `restartSync` 500ms past `initContent` + stores- ready so the Path widget's `$workspaceStore && $userStore`-gated `initPath → reset → onMetaChange → bind:path` cascade lands inside the suspension window. Two `tick()` waits weren't enough — the bind:path mutation fired ~100ms after the prior `restartSync` and posted as a "user edit". - apps_raw route: suspend autosave on `new_draft=true` and resume only after the framework picker closes (via `onStart` or X dismissal), with a two-tick settle so the picker's seeded `files/runnables/data/policy` mirror to `draftHandle.draft` observably advances `lastSerialized` before sync re-arms. * fix(drafts): land /add redirects on the real workspace username, not "me" The `/add` → `/edit/u/{username}/draft_{uuid}` redirects ran during SvelteKit's load phase, BEFORE the (logged) layout's async `getUserExt` populated `userStore`. `get(userStore)?.username` returned undefined and fell back to the `'me'` placeholder on every fresh nav, producing `u/me/draft_{uuid}` paths instead of the user's real namespace — broke ownership checks against `authed.username` and silently scoped autosaves under the wrong path. Layout now persists `username` to localStorage on every successful `getUserExt`, and `getUsernameForNamespace` (new shared helper, used by all four `/add/+page.ts` files) reads the live store first, falls back to the cached value, and only then to `'me'` for true first-ever loads. * fix(drafts): key low-code app autosave on the URL path, not the empty string `AppEditor` keyed its `UserDraft.use` handle on `newApp ? '' : path` — a legacy leftover from when `/apps/add` was its own URL (no path). With the `/add` ⇒ `/edit/u/{user}/draft_{uuid}` redirect, `newApp=true` made autosaves land on the `('app', '')` row instead of the URL path: - The `apps/list?include_draft_only=true` query joins drafts onto `app.path`, surfacing drafts at the URL path. The empty-path row didn't match the user's URL so the draft never appeared in the home list. - Refreshing `/apps/edit/u/{user}/draft_{uuid}` re-fetches at the URL path with `?get_draft=true`, finds nothing, and 404s. Drop the ternary so the handle always uses `path` — the same as scripts/flows/raw_apps. The route's `?new_draft=true` branch already seeds the empty-template baseline, so there's no longer a "the draft sits under '' until first save" race to worry about. * fix(raw_app): propagate template picker X / Esc dismissal so autosave resumes The picker mounted `<Modal kind="X" open ...>` (one-way prop, not `bind:open`). When the user dismissed via X / Esc / click-outside, the inner Modal flipped its own local `open` to false (hiding the UI) but never wrote back to the picker's `open` $bindable. The route's `templatePicker → false` watcher — the one that calls `restartSync` two ticks after the picker closes — never fired, so autosave stayed suspended and the user's edits after dismissal were silently dropped. Switch the inner Modal to `bind:open` so the dismissal bubbles all the way up to the route's state. "Start without AI" already worked because its `onStart` handler explicitly sets the picker's `open = false`. * nit unused * fix(drafts): make the home-page View/Edit JSON action work on draft-only apps The "View/Edit JSON" entry on the home page called `AppService.getAppByPath` without `get_draft=true`, so for draft-only items at `u/{user}/draft_{uuid}` the backend 404'd with "App not found at path …". Pass `get_draft=true` and render the synthesized stand-in's editable shape: - App drafts come back as `{summary, value, path, policy, ...}` — `value` is the App definition the editor was working on; show that. - Raw-app drafts come back as the flattened `{files, runnables, data, summary, policy, ...}` with no nested `value`; show the whole shape. On save, draft-only items can't go through `updateApp` (no deployed row). Route the edit through `UserDraftDbSyncer.save` (with `immediate: true` so `await` resolves after the POST lands) and relabel the button "Save draft" + Save icon. Deployed items keep the existing "Deploy" flow unchanged. * fix(drafts): render the right shape in View/Edit JSON for draft-only items The previous fix landed `fapp.value` into the editor, but the deployed-overlay flattens the bare editable shape into `inner`/the top-level response — drafts have no nested `.value`. So: - App drafts (`{grid, breakpoints, hiddenInlineScripts, …}`) rendered as empty (`fapp.value` was undefined). - Raw-app drafts 404'd outright: `get_draft=true` with no `rawApp` flag can't tell which draft kind to look up, defaults to `app`, doesn't find one. Thread the row's `raw_app` flag from AppRow → `appExport.open(path, rawApp)` → `getAppByPath({..., rawApp})` so raw-app drafts resolve to the right `UserDraftItemKind`. Read `fapp.draft` (the bare editable shape from `fetch_draft_only`) into the JSON editor for draft-only items — clean payload, no `is_draft` / `no_deployed` / overlay noise. Save the same bare shape back through the syncer so the regular editor reads it unchanged on the next mount. * fix(drafts): skip public-secret-URL fetch in the Deploy drawer for draft-only apps Opening the Deploy drawer on a `/edit/u/{user}/draft_{uuid}` app fired `AppService.getPublicSecretOfApp` immediately because the gating effect only checked `appPath != ''` + `savedApp`. The `/secret_of/{path}` route plain-SELECTs `app.id`, so a draft-only path 404'd with "App not found at name …" and the public-URL ClipboardPanel spun forever waiting on `secretUrl`. Thread the existing `newApp` signal (already on `AppEditorHeader` / `RawAppEditorHeader`) into `AppEditorHeaderDeploy`, gate the fetch behind `!newApp`, and render the existing "Deploy this app once to get the public secret URL" placeholder instead of the spinner for draft-only items. * fix(drafts): disable Diff button on draft-only items across the 4 editors Diff has no baseline to compare against on draft-only items — the button used to be gated by the pre-PR `/add` route's own state, but the `/add → /edit` redirect landed everything under the regular `/edit` page where the gate was missing. - ScriptBuilder: gate the topbar Diff on `savedScript.no_deployed`; seed `no_deployed: true` on the route's `new_draft` empty NewScript so the gate fires before the first deploy. - FlowBuilder: gate the topbar Diff on `newFlow` (route already sets it from `backendFlow.no_deployed` and the new-draft branch). - AppEditorHeader: gate both the "Diff" dropdown action and the Deploy-drawer's "Diff" button on `newApp`. - RawAppEditorHeader: gate the topbar Diff + the Deploy-drawer's "Diff" button on `newApp`. Each gate also rewrites the tooltip ("Deploy this … once to compare against the deployed version") so the hover state explains why. * fix(drafts): disable the "No login required" toggle on draft-only apps Flipping the toggle called `setPublishState`, which POSTs the new `policy` through `AppService.updateApp` — that handler's `UPDATE app ... RETURNING path` finds nothing on a draft-only path and `not_found_if_none` 404s with "App not found at name …" (apps.rs:1975). Gate the Toggle on `!newApp` too so the user has to deploy once before configuring the publish state. * refactor(drafts): drop dead draft_path field from list responses The draft-only listing branches in scripts/flows/apps computed a `draft_path` from the draft JSON (when the user-typed path differed from the URL's autogenerated `u/{user}/draft_{uuid}`), and `{Script,Flow,App} Row.svelte` preferred it over `path` for the row title. In practice that path is never written: the app, raw-app and flow editors all warn "Deploy the X to make the path change effective" — the rename only lands on deploy, never in the draft. So the field is always None and the home rows always show the autogenerated slot anyway. Drop the field from the three `Listable*` structs, the three draft-only push sites, the three OpenAPI response schemas, and the three frontend row components. Client regenerated. * fix(drafts): seed a friendly name on /flows/add The flow route passed `initialPath={page.params.path ?? ''}` to FlowBuilder, so on the `/flows/add → /flows/edit/u/{user}/draft_{uuid}` redirect the Path widget's `initPath` saw a non-empty `initialPath` and skipped the `reset()` branch that auto-generates the friendly `<random_adj>_flow` name. The other three editors all clear `initialPath` in their `new_draft` branch for exactly this reason. Track `initialPath` as route-owned state (defaults to the URL path) and clear it to '' inside the `new_draft` branch, then bind it through to FlowBuilder so any post-deploy update from the editor still propagates. * feat(drafts): render friendly user-typed path on home list for all 4 kinds Reinstate `draft_path` on `Listable{Script,Flow,App}` so the home rows prefer the user-typed name over the autogenerated `u/{user}/draft_{uuid}` URL slot, with two source rules — one per how each editor wires the Path widget: - Scripts already work: `ScriptBuilder` binds the Path widget directly to `script.path`, so the typed path round-trips through the draft JSON's own `path` field. Backend extracts `v["path"]` when it differs from `row.path`. - Flows / apps / raw apps don't write the typed path into the autosaved value (`Flow.path` is one-way-bound to `$pathStore`; the bare `App` / raw-app value has no `path` field at all). Introduce an explicit `draft_path` field on the draft JSON, written by the editor ONLY when the typed path differs from the deployed/seeded `savedX.path`: - FlowBuilder: $effect on `$pathStore` mutates `flow.draft_path`. - AppEditorHeader: $effect on `newEditedPath` mutates `$app.draft_path`. - RawAppEditorHeader: $effect surfaces `pendingDraftPath` up via the bind chain (RawAppEditor → route); the route's draftHandle.draft spread includes `draft_path` when set. Backend extracts `v["draft_path"]` and `None` when unchanged or after deploy (deploy clears the whole draft, so the field naturally disappears post-deploy without bookkeeping). Flow route's `new_draft` branch now stops sync around the Path widget cascade, with a 700ms scheduled `restartSync` (mirrors the existing scripts/apps/raw_apps stoppers) — the new draft_path mutation lands inside that window so `/flows/add` no longer fires an autosave before the user's first edit. openapi/sqlx regenerated. * fix(drafts): preserve the user-typed draft_path on reload of draft-only items The flow / app / raw-app editors all dropped the saved `draft_path` back to the URL's `u/{user}/draft_{uuid}` slot the moment the user reloaded a draft-only edit page: the route sourced the Path widget's initial path from `page.params.path` instead of the previously-saved `draft_path`, and the first user edit then mirrored that URL path back into the autosaved draft — silently overwriting the friendly name in both the row and the editor. - Flow route: after computing `effectiveFlow`, override `flowInitialPath` with `effectiveFlow.draft_path` when set. - App route: pass `newPath={(app.value as any)?.draft_path ?? app.path}` through to `AppEditor`; AppEditorHeader's `newEditedPath` default now prefers a non-empty `newPath` over the random `<adj>_app` seed (the `newApp && !newPath` branch keeps the `/apps/add` friendly auto-name). - Raw-app route: surface `savedRawAppDraft.draft_path` onto `backendApp` so the `extractRawApp` path seeds `newPath` with the friendly name. Reload + a subsequent edit now leaves `draft_path` intact for all three kinds; verified end-to-end via the `/drafts/get_draft/...` endpoint. * fix(ui): default Modal2 target to 'body' so omitting the prop doesn't throw Modal2 defaulted `target = ''` and forwarded it to `Portal`, which calls `document.querySelector(target)` — an empty selector throws "Failed to execute 'querySelector' on 'Document': The provided selector is empty" and the modal silently fails to mount. That's why `OtherUsersDraftsModal` (and `DraftSyncConflictModal`) never appeared on editors where another user had a draft — both omit the `target` prop. Other Modal2 callers (StorageSettings, CriticalAlert, CustomInstanceDbWizardModal, …) pass an explicit `target="#content"` and were unaffected. Match Portal's own default of `'body'` so omitting the prop is now a no-op rather than a runtime throw. * fix(drafts): Reset to deployed no longer resurrects the draft The toast's "Reset to deployed" callback POSTed `value: null` to the syncer, then handed control to the route's `onResetToDeployed` (which wipes the in-memory handle and reloads the deployed payload via `getDraft: false`). Both writes flowed through the reactive sync effect: the wipe scheduled a delete, the reload scheduled a re-save of the deployed value as the new draft. Coalescing collapsed them and the draft came back — making the "discard" action effectively a no-op. Wrap the whole callback in `UserDraft.stopSync` / `restartSync`. The explicit `value: null` POST still goes through (it's a direct `UserDraftDbSyncer.save` that doesn't depend on the reactive effect), the route's wipe-then-reload mutations advance `lastSerialized` silently under suspension, and the next user edit (after two ticks past the deployed-seed write) is the first real save again. * ui nit * feat(drafts): autosave-indicator popover with Reset-to-deployed action Click the cloud icon → popover with "All changes are saved as a draft on the server. The draft is per-user — your teammates' editors keep their own." When the editor isn't on a draft-only path AND the user has a draft (UserDraft.has returns true), a "Reset to deployed" button mirrors the load-time toast action — stops sync, POSTs `value: null`, runs the route's reload-without-draft callback, restarts sync past two ticks so the deployed-seed write doesn't resurrect the draft. Threaded `onResetToDeployed` from each route down to its builder (ScriptBuilder / FlowBuilder / AppEditorHeader / RawAppEditorHeader) and into the indicator. `draftOnly` is wired from `savedScript.no_deployed` / `newFlow` / `newApp` so the action hides where there's nothing to fall back to. The indicator's trigger now has a hover affordance + matches Portal's default target ('body') via Modal2's earlier fix. * fix(drafts): wait for the fork POST to land before navigating OtherUsersDraftsModal's Fork action called UserDraft.save, which routes through the autosave debouncer (1500ms). The subsequent goto fired within the same tick, so the destination editor's get_draft=true read ran before the POST landed and 404'd — refreshing worked because by then the debounced save had fired. Call UserDraftDbSyncer.save with immediate: true and await it. The syncer cancels any queued debouncer task for the key and resolves the promise only after the POST completes, so the route load can find the forked draft on the first try. * fix(drafts): conflict detection — keep last_sync map tab-local instead of in localStorage Two tabs editing the same draft both load with last_sync = T0. Tab-1 saves; the server accepts, returns T1, and the syncer wrote T1 into localStorage. Tab-2 then tries to save: it reads the SHARED localStorage map, sees T1 instead of its own baseline T0, sends last_sync = T1, and the backend's WHERE clause (`created_at <= last_sync`) is true → tab-2 clobbers tab-1's edit without ever seeing a conflict. Move the map to tab-local memory (`new Map<string, …>`). Reload of the tab now starts with an empty map; that's fine because the editor's load path calls `recordRemoteSync(query, draft_saved_at)` right after `get_draft=true` returns, reseeding from the authoritative server timestamp before any user edit could fire a save. * fix(drafts): OtherUsersDraftsModal — close on Fork, don't leak clicks through nested JSON Two bugs in the per-editor "another user has a draft" banner: - Fork landed the immediate save but didn't close the banner before navigating. Svelte hadn't torn down the previous route's components by the time goto returned, so the banner lingered on top of the destination editor. Comment the explicit isOpen=false on the happy path so it's clear it MUST run before goto. - Clicking anywhere on the screen while the View JSON drilldown was open closed the underlying banner too. Modal2's clickOutside action fired on every Modal2 instance — both the JSON modal and the underlying banner — because both attach their own listener at the document level. Add `closeOnOutsideClick` opt-out on Modal2 and pass `closeOnOutsideClick={!jsonOpen}` to the outer modal so clicks outside the JSON drilldown only close the drilldown. Drive-by: Modal2's keydown handler now ignores Escape when its own isOpen is false (was a no-op closer that would still preventDefault on every key press, swallowing key events for any siblings). * fix(drafts): conflict modal wording — drafts are user-scoped, not teammate-scoped * fix(drafts): defer reset-to-deployed restart until first user interaction Two-tick `restartSync` was too aggressive: editor remounts emit a tail of cascading writes (Monaco setValue acks, schema re-infer, UI Builder iframe handshakes, schedule-config recomputes, …) that land well after two ticks and would clobber the just-deleted draft with an upsert of the deployed value — making "Reset to deployed" a no-op in practice, the user kept seeing the draft come back. Centralise the suspension lifecycle in a new `runResetToDeployed` helper. It stopSyncs around the reset, POSTs the explicit delete, runs the route's wipe-and-reload, and then arms a one-shot listener on document keydown / input / pointerdown that restartSyncs on the user's next real interaction. A 5-second fallback re-arms sync if the user walks away without touching the editor, so suspensions don't leak. Use it from both the load-time toast (`notifyDraftLoaded`) and the autosave-indicator popover so the two stay in sync — fixes both entry points. * indicator ui nits * fix(drafts): split tab-switch and unload flushes — kill self-conflict on visibility change The single keepalive flush bound to both `visibilitychange → hidden` and `pagehide` self-conflicted on tab switch: visibilitychange fires on every tab/app switch with the page still alive, the keepalive POST advanced the server's `created_at` to a fresh `now()`, the client discarded the response (no listener), the local `lastSync` stayed at the old value, and the next foreground autosave sent that stale timestamp → server saw `created_at > last_sync` → conflict modal for the user's own background-tab write. A still-pending debouncer task made it worse: it fired a second runner POST after the keepalive with the same stale `last_sync`, the second self-conflicted too. Split into two paths: - `visibilitychange → hidden` → `flushOnVisibilityHidden`: route through the normal runner pipeline. The page is alive, so the response can land and `setLastSync` keeps the baseline current. Call `debouncer.cancel(key)` first so a queued keystroke can't double-fire with the same stale `last_sync`. - `pagehide` → `flushOnPageHide`: keep the `keepalive: true` raw fetch for the genuinely-going-away case (the JS context is torn down, the response is necessarily discarded). Same `debouncer.cancel(key)` guard. On the next mount, the route's `recordRemoteSync(query, draft_saved_at)` reseeds `lastSync` from authoritative server state before any user edit can fire a save. * fix(drafts): drop the visibilitychange flush — debouncer keeps running on hidden tabs Tab switching just hides the page; the JS context survives and the debouncer's `setTimeout` keeps counting down. When it fires, the runner POSTs normally and the server's response updates `lastSync`. There's nothing left for a visibilitychange-driven flush to do that the ordinary pipeline doesn't already handle, and adding one only creates extra POSTs to reason about. `pagehide` remains the single trigger for the keepalive flush — that's the case where the JS context is actually being torn down and the runner's pending fetch would otherwise be killed mid-flight. * nit * refactor(drafts): drop LS-era pipeline; backend is canonical on load The PR's iteration left behind a meta/staleness pipeline carried over from the localStorage era — per-rev tracking, a LocalDraftStaleModal, a 'Restored from local storage' toast, and a localDraft-vs-backend comparison branch in every editor loader. With drafts now living in the DB and the optimistic-concurrency lastSync check handling divergence, that whole stack is dead weight. Worse, the comparison branch caused 'Load from server' in the conflict modal to do nothing: the loader preferred the in-memory cell over the backend, so the user-clicked 'load from server' just re-displayed the local edits AND fired two confusing toasts (Restored from local storage + Loaded your saved draft). The rip: * userDraft.svelte.ts: drop UserDraftMeta, StoredDraft.meta, checkStaleness, UserDraftStalenessCause, normalizeForCompare, localDraftDiffers, saveMeta, getMeta, setDraftAndMeta, setMeta, handle.meta/setDraftAndMeta/setMeta, force option. Handle is now just { draft }. * userDraftToast.ts: drop notifyRestoredFromLocal + RestoreFromLocalActions. Update copy. * LocalDraftStaleModal.svelte: deleted. * AppEditor.svelte: drop initialRevs prop and the firstMirror wipe-then-restore dance (it existed only to consume the meta-mismatch skip slot). * All 4 editor routes: backend is canonical on load — the in-memory cell is overwritten with the deployed+draft overlay, the syncer's seed guard swallows the first write so we don't POST it back. * VariableEditor / ResourceEditor: drop the staleness pipeline + rev bookkeeping; backend wins on open. * useTriggerDraftSync.svelte.ts: inline the JSON-normalize + deepEqual utility as a private cfgDiffers helper (kept for the form-vs-deployed dirty check, which is a genuine semantic compare, not LS legacy). * copilot core.ts / userDraftAdapter.ts: drop meta argument from saveAppDraft, loadAppDraftValue, write*Draft. Test assertions on getMeta dropped. Net: -22 typecheck errors, fewer moving parts, conflict modal works. EOF ) * refactor(drafts): remove dead endpoints + UserDraftDbSyncer.getLastSync The list_drafts and get_draft (own) routes were added during PR iteration and never wired up to any frontend caller — the editor overlay path uses the per-kind get-by-path getDraft query parameter, and the home page lists drafts via the per-kind list endpoints, not via /drafts. Drop both routes (+ sqlx caches + OpenAPI entries). UserDraftDbSyncer.getLastSync was a peep-hole for callers that never materialised — the per-tab lastSync map is only ever read by postSave internally, where the bookkeeping already lives inline. * refactor(drafts): extract DraftEditorModals trailer block The four editor routes (scripts/flows/apps/apps_raw) mounted an identical pair of trailer modals — DraftSyncConflictModal + OtherUsersDraftsModal — wrapped in the same guard chain and {#key path} remount. Lift the markup into one component; routes thread their itemKind, path, editPathFor, and loader callback. Pure markup extraction, no state ownership change. Drops the unused userStore import where the trailer was the only consumer. * refactor(drafts): UserDraft.useReactive — kill array-of-one boilerplate The script + flow routes both wanted a handle that re-keys when the URL path changes. UserDraft.use() can't do that (its opts getter is untracked), so each route hand-rolled the same useMany-array-of-one + proxy idiom: const handles = useMany(() => [{ kind, path: reactive }]) const handle = { get draft() { return handles[0]?.draft }, ... } Add UserDraft.useReactive(getSpec) that internally wraps useMany with a single spec and returns the stable proxy. Callers collapse to one line. * refactor(drafts): unify bootstrap suspension via armRestartOnFirstInteraction The flow and raw-app routes each rolled their own end-of-bootstrap resume: a 700ms setTimeout for flows and a templatePicker watcher with double-tick gating for raw-apps. Both are timing-fragile (the comments admit it) and drift from each other. armRestartOnFirstInteraction already existed in userDraftToast.ts for reset-to-deployed: keydown/input/pointerdown listeners (capture phase) that fire restartSync on the first real user touch, with a 5s belt-and-braces fallback. Export it and use it everywhere we'd previously have picked a magic number. For raw-apps this is a tiny behavioural change: the user's template choice now POSTs immediately (the pointerdown that picks the template also resumes sync, so the picker's onStart write rides the wake-up). Previously the choice only persisted on the user's NEXT edit. That's strictly better — navigating away preserves the choice now. * refactor(drafts): type App.draft_path; drop the as-any cast The audit asked for the three editors to converge on one draft_path injection pattern. For App and Flow, the in-builder $effect-mutates- the-store idiom is wedged into a shape that doesn't natively own the field — App's editor type genuinely has no draft_path so the writer had to cast through `as any`, and consumers downstream did the same. The minimum viable fix: declare draft_path on the local App type (it's already a field on the autosaved JSON). Lifting the writes upward into a route-side merger would mean restructuring the AppEditor mirror $effect and the FlowBuilder pathStore plumbing — larger change for the same shape, deferred to a follow-up. Flow already has the typed cast localised at one site. Will get the OpenAPI-level draft_path field as part of task 47 (drop as-any casts on backend overlay reads). * refactor(drafts): extract makeDraftAddLoad helper Four identical /add/+page.ts files differing only by the edit-route prefix. Lift the redirect into a factory, slim each entry point to two lines. * refactor(drafts): type UserDraftOverlay.other_drafts_users in the OpenAPI The backend response carried other_drafts_users on every get-by-path that supports the draft overlay, but the OpenAPI schema didn't declare the field. Each route had to cast the typed response to `any` to read it (and the sibling draft_saved_at), which obscured the real shape from the type system and rotted the discoverability of the draft surface. Add it to UserDraftOverlay. Frontend casts collapse to plain property reads in the three editor routes. * feat(drafts): list & open draft-only items for variables, resources, schedules, triggers For scripts/flows/apps the list and get-by-path endpoints already surface per-user drafts that have no deployed counterpart — that's what gates the home page from 404'ing on an AI-agent-created draft. Extend the same support to the other UserDraftItemKinds: Backend (list endpoints): - Add include_draft_only to ListVariableQuery, ListResourceQuery, ListScheduleQuery, StandardTriggerQuery (the latter covers the 11 trigger kinds via the generic TriggerCrud). - Append per-user draft rows whose path has no deployed row. Same gate as scripts/flows/apps: non-operators, page 0, no narrowing filters. Synthesis is per-kind: ListableVariable/Resource get field-for-field synthesis; ScheduleLight reads NewSchedule shape; Trigger<T> uses a best-effort JSON merge + serde_json::from_value (rows skipped on deserialize failure rather than failing the list). - Add draft_only: Option<bool> with sqlx(default) to each row type so it serializes as the column is opt-in. Backend (get-by-path endpoints): - get_variable, get_resource, get_schedule, get_trigger<T> fall back to fetch_draft_only when the deployed row is missing and the caller passed get_draft=true. Mirrors scripts/flows/apps. OpenAPI: - Shared IncludeDraftOnly parameter under components/parameters, wired into the 11 trigger list endpoints + listRawApps. Inline declarations on listVariable / listResource / listSchedules / listAzureTriggers. - draft_only field on ListableVariable, ListableResource, Schedule, TriggerExtraProperty. Frontend: - variables, resources, schedules, and the 10 trigger list pages (routes + 9 *_triggers) pass includeDraftOnly: true on the initial fetch and render <DraftBadge draft_only> on synthesized rows. Trigger pages got a sed/perl bulk update — pattern is the same across kinds. * fix(drafts): swap crypto.randomUUID() for the project's randomUUID helper crypto.randomUUID() is gated on a secure origin (HTTPS or localhost). Self-hosted Windmill instances often run on a bare HTTP origin or a LAN IP where the WebCrypto API is unavailable, so the /add redirect would throw before issuing the 307. Use the existing RFC4122 v4 helper in FlowChatManager that the rest of the codebase already imports for this exact reason. * fix(editor): leading-edge fire + max-wait cap on Monaco debounce The Editor debounced `onDidChangeModelContent` purely on the trailing edge — every keystroke rescheduled a 500ms timer, and uninterrupted typing held the bindable `code` prop stale until a pause. Stacked behind our 1.5s autosave debouncer that meant our clock didn't even start ticking until 500ms after the user paused, and the `code` binding never updated mid-burst for downstream consumers (lint, live preview, change listeners). Switch to leading + trailing + max-wait: * First keystroke of a burst fires `updateCode` synchronously, then stamps a wall-clock chain start. * Each subsequent keystroke (re)arms a trailing timer at `min(now + changeTimeout, chainStart + maxChangeTimeout)` — the cap is what makes continuous typing materialize at least once per maxChangeTimeout window instead of indefinitely. * When the trailing fires it resets the chain so the next keystroke after a pause is a fresh leading fire. New prop `maxChangeTimeout` (default 1000ms) sits next to the existing `changeTimeout` (default 500ms). Dispose path clears the chain stamp alongside the timer. * feat(drafts): wire Ctrl/Cmd+S to flush the pending autosave immediately Each builder already had a Ctrl/Cmd+S keybinding routed through a saveDraft() no-op left over from the LS-era — the comment said "persistence happens via the page-level UserDraft autosave" but the shortcut was the user's only way to actually force a save without waiting for the 1.5s debounce. Restore the intent. * UserDraftDbSyncer.flush({ workspace, itemKind, path }) — new method that re-submits whatever's queued in pendingSaveOpts with immediate: true. No-op when nothing's pending. * Editor.svelte.flushPendingChanges() — exposes a synchronous updateCode() with chain reset, so callers can drain Monaco's own trailing debounce before asking the syncer to flush. Without this step a Ctrl+S within ~500ms of typing would POST the pre-burst content. * ScriptBuilder.saveDraft() — editor?.flushPendingChanges() → await tick() → UserDraftDbSyncer.flush(). Toast on result. * FlowBuilder.saveDraft() — no direct Monaco ref (flows have many per-module editors); just flushes the syncer. Editor.svelte's new 1s max-wait cap means at most the last <1s of typing in a module Monaco won't be in this POST; it follows in the next autosave round. * RawAppEditor.handleKeydown — adds a 's' case that flushes before the focus guard, so the shortcut fires regardless of where focus is in the editor pane. * fix(drafts): low-code apps — drop spurious autosave on /edit + remount on Load from server Two bugs in low-code app editor (raw apps use a separate code path): 1. Every /edit visit looked like an autosave because loadApp() called UserDraft.discard('app', path, undefined). The comment claimed "this load doesn't POST" but discard always POSTs value: null server-side — that surfaced as a DELETE-my-draft on every page load AND a flash in the AutosaveIndicator. The discard was originally intended to wipe the in-memory cell so AppEditor remounts "fresh". But the path-change $effect upstream already sets app = undefined before each loadApp, which unmounts AppEditor and releases the handle's entry — so a remount via app = backendApp naturally starts with an empty handle. Drop the discard. 2. The conflict modal's "Load from server" called loadApp() but didn't remount AppEditor. Since AppEditor's stateApp is captured once at mount and doesn't react to prop changes, the editor kept showing the conflicting local edits even after a successful reload. Wrap the onLoadFromServer to await loadApp() then bump redraw to force a fresh mount. * feat(drafts): home-page Draft badge — show user-initial circles, drop the '+' The home-page Draft badge previously showed '+Draft' as a flat label. Add per-user awareness: up to 3 user-initial circles render to the left of the label, ordered alphabetically; with 4+ users we collapse to the first 2 + a '+N' overflow circle so rows stay compact. Backend: * New `DraftUserRef { username: Option<String> }` in windmill-types::user_drafts, re-exported from windmill-common so the list endpoints in scripts/flows/apps crates share one import path (windmill-types/windmill-common can't be reordered without a cycle). * ListableScript / ListableFlow / ListableApp gain a `draft_users: Option<sqlx::types::Json<Vec<DraftUserRef>>>` field. The list SQL adds a per-row subquery `SELECT json_agg(...) FROM draft d LEFT JOIN usr u ...` that aggregates the workspace users with a per-user draft at this path. NULL (no drafts) decodes to None; LEFT JOIN against `usr` lets orphaned drafts (user removed from workspace) still surface with username = None. * Synthesized draft-only rows set draft_users to a single-element vector with the authed user (those rows come from `email = $2`). OpenAPI: `draft_users` added to listScripts / listFlows / ListableApp response shapes as an array of `{ username }` with nullable username. Frontend DraftBadge: * Accepts `draft_users: { username?: string | null }[]`. Renders up to MAX_CIRCLES (3) initial circles; at 4+ users renders first 2 + a gray '+N' overflow circle. * Initials: 'john.doe'/'john_doe' → 'JD', 'alice' → 'AL', the legacy NULL-email row → '?'. * Color picked deterministically from a 6-entry palette so the same user gets the same circle color across rows. * Label is now just 'Draft' (dropped the '+'). 'Draft only' is unchanged. * Tooltip lists every user in full. ScriptRow / FlowRow / AppRow thread `draft_users` through their prop types and pass it to DraftBadge. * fix(drafts): suppress 'You have unsaved changes' banner when deployed baseline is null A brand-new variable/resource/trigger (no deployed row yet) has `getDeployed() == null`, but the caller's `show` prop is computed off `current != deployed` which is trivially true while the user types. Result: the banner appeared with 'Show diff' (no-op — the drawer early-returns on null deployed) and a 'Discard' that's semantically backwards (there's nothing to revert to). Gate `show` internally on `getDeployed() != null`. The check sits in the banner rather than each caller because every caller would otherwise need the same boilerplate guard. * fix(drafts): hide LocalDraftBanner when deployed and current match the DiffDrawer's compare Earlier I gated the banner on `getDeployed() != null`, but the user still saw it fire on entries where 'Show diff' opens to 'No changes detected'. That means `show` (the caller's coarse dirty check) flagged a difference the DiffDrawer treats as a no-op — typically toggle defaults (`false ↔ undefined`), removed empty arrays, or key-ordering noise that `cleanValueProperties + orderedYamlStringify` collapses. Replicate the drawer's comparison inside the banner: stringify both sides through the same pipeline and only render when the keys differ. A single `diffKey()` helper keeps the logic local; the catch-and-empty fallback survives a non-serializable side rather than throwing. * ui(drafts): nest user-initial circles inside the Draft badge Previously the circles sat alongside the Badge in a parent flex container; the result read as two separate UI elements. The Badge component already exposes its children as a snippet rendered inside its own flex row, so moving the circles into it makes them feel like part of the same chip. Knock-on tweaks: shrunk the circles from h-4/w-4 to h-3.5/w-3.5 so the badge stays compact, and tinted each circle's ring with the badge's indigo palette (instead of plain white) so the overlap reads as a deliberate stack rather than dots floating on top of the chip. * feat(drafts): drop the authed user's circle, mark own drafts with a '*' suffix Three tweaks to the home-page Draft badge: 1. Filter the authed user out of `draft_users` before rendering circles. The row already signals 'this user has a draft' via the asterisk (below), so a circle for them would be redundant noise. New `currentUsername` prop on DraftBadge — pass `$userStore?.username` from each row. The tooltip still lists every user (with `(you)` next to the authed one) so the full picture is one hover away. 2. The badge already showed whenever `is_draft || draft_users.length > 0` (per-user OR any-user). Spelled the rationale out in a comment — no logic change. 3. Append '*' to the displayed summary when `is_draft` is true. Falls back to `draft_path`/`path` when summary is empty so the marker never decorates an empty string. Threaded the same expression into ScriptRow / FlowRow / AppRow. Slice/overflow math now keys on the post-filter `otherUsers` list, so dropping the authed user doesn't silently shrink the visible count (e.g. 3 users incl. self → 2 circles, not 1 circle + a '+1' bubble). * feat(drafts): clone per-user drafts when forking a workspace `clone_workspace_data` clones every other workspace-scoped table on fork creation (resources, variables, scripts, flows, apps, raw apps, triggers, schedules) but quietly dropped the `draft` table. With per-user drafts that meant any open editor in the parent lost its pending edits the moment a fork was created — surprising and inconsistent with how forks treat the deployed surface. New `clone_drafts` mirrors the existing clone helpers: a single INSERT...SELECT into the target workspace, preserving `path`, `typ`, `value`, `created_at`, and `email`. The `email` FK targets `password.email` which is instance-scoped so it carries across workspaces without remap. `created_at` is preserved on purpose so the per-tab `last_sync` baseline lines up with the parent's timeline — otherwise the fork's next autosave would race a stale `last_sync` and trip the conflict modal on every cloned draft. Plain INSERT (not UPSERT) is safe because the fork target is empty at create time; no conflict against the partial unique indexes (`draft_pkey_with_user` / `draft_pkey_legacy`). The synthetic BIGSERIAL `id` PK is regenerated by the default so it stays out of the column list. * ui(drafts): pin the authed user to the first circle instead of hiding them Previously the authed user was filtered out of the circle row entirely on the theory that the row's '*' suffix already signalled 'this user has a draft'. New requirement: they should always lead the circle row when they have a draft so the visual half of the signal lines up across rows (consistent leading-slot identity, easy scan). Switch from a filter to a sort: `orderedUsers` finds the authed user in `draft_users` and splices them to index 0; everyone else keeps the backend's alphabetical order behind. Slice/overflow math now keys on `orderedUsers`, which guarantees the authed user never falls into the '+N' bubble — they're at position 0 and the slice keeps the head. The popover's '(you)' annotation moves to the circle's title attr too, so hovering the leading circle confirms the identity. * feat(drafts): drop draft_only column from script/flow/app Drafts now live in the `draft` table exclusively — `draft_only` stubs in script/flow/app are redundant. Migration `INSERT INTO draft ... ON CONFLICT (workspace_id, path, typ) WHERE email IS NULL DO NOTHING` so real per-user drafts already at the same path are preserved; only rare stubs that lost their draft get a synthesised workspace-level row. Stubs are then deleted (FKs cascade to *_version) and the column is dropped. List endpoints keep a synthesised `draft_only: true` on rows sourced from the draft table itself (sqlx default on the struct field). Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * ui(drafts): surface draft state in AutosaveIndicator instead of toast+auto-modal The "Loaded your saved draft" toast and the auto-opening OtherUsersDraftsModal both surprised users on every editor mount. Move both signals into the AutosaveIndicator label: "Loaded from draft" or "Others are working on this {kind}" (priority) sits where Saving/Saved do, with a one-shot light-green flash behind the indicator that fades to transparent. Saving/Saved still win when they fire. The popover gains a "See others' drafts" button that flips the modal open on demand; the modal itself is now externally controlled via a bindable \`isOpen\` threaded through DraftEditorModals. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * ui(drafts): per-user View JSON / Fork actions in DraftBadge popover Hover popover used to be a plain text list of usernames. Now each row gets a colored circle icon + name + "(you)" for the authed user, and every OTHER user's row carries View JSON / Fork buttons mirroring the OtherUsersDraftsModal. For draft-only entries owned solely by the authed user, the popover ends with "Only you can see this {kind}" so the row's privacy is obvious. ScriptRow / FlowRow / AppRow thread workspace + itemKind + path + editPathFor through; AppRow switches between app / raw_app on app.raw_app. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * nit * fix(drafts): clone only the forker's per-user drafts on workspace fork clone_drafts copied every user's drafts, but only the forker gets added to the fork's usr table. Drafts owned by absent users LEFT-JOIN to NULL in the home page's draft_users aggregate, surfacing as multiple legacy-style rows at one path and crashing the popover with each_key_duplicate. Filter the clone to email = forker OR email IS NULL, and key the popover's #each by index defensively so future legacy collisions can't crash the page either. Also re-adds `draft_only: None` to NewScript/CreateFlowBody literals in tests — the auto-generated windmill-api-client still carries the field and the previous commit dropped them too aggressively. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * fix(drafts): always populate other_drafts_users in maybe_overlay_draft Reset-to-deployed reloads the deployed payload with get_draft=false, which made the backend return other_drafts_users=[]. The route then reassigned otherDraftsUsers to the empty list, dropping the count to 0 and hiding "See others' drafts" in the AutosaveIndicator popover — but the other users' drafts hadn't actually gone anywhere. Fetch the list independently of get_draft so the popover stays accurate across reset reloads. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * feat(drafts): alert user when their draft is older than the latest deploy Open a modal on editor mount when the per-user draft was saved before the latest deploy at the same path — i.e. a teammate deployed a new version while this user's draft was sitting. Two choices: discard the stale draft and pick up the deploy, or keep editing the older draft. DraftEditorModals computes the staleness from the timestamps each route threads in (script.created_at, flow.edited_at, app_version.created_at) and the "Load latest deploy" callback reuses the route's existing reset-to-deployed logic. Wired for script / flow / app / raw_app editors; trigger / resource / variable drawer editors follow a different pattern and aren't covered here. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * fix(drafts): deploy only wipes the deployer's draft, not everyone else's Script / flow / app deploys ran an unconditional DELETE on every draft at the path, so a teammate's deploy silently destroyed any other user's pending draft. After the wipe, the other user's tab kept auto-saving — re-creating the row at a NOW timestamp newer than the deploy — and StaleDraftModal never fired because draft_saved_at had been bumped past the deploy. Filter the DELETE to email = deployer (plus the legacy NULL row), so other users' drafts persist and the stale-draft prompt actually fires on their next reload. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * fix(drafts): surface save failures in AutosaveIndicator instead of pretending Saved postSave caught network errors with `console.error` and let the runner finish normally. The indicator read the saving → none transition as a successful save and flashed "Saved" even when the request had thrown. Track failed keys in a SvelteMap, expose `'failed'` as a new UserDraftSyncState, render "Save failed" in red with a CloudOff icon. Failure clears on the next successful save for the same key, or when recordRemoteSync seeds a fresh authoritative timestamp. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * fix(drafts): surface 'Save failed' inside the AutosaveIndicator popover too The popover used to repeat the cheerful "All changes are saved as a draft on the server..." copy even when the inline label said "Save failed", which read as contradictory. Add a red, text-xs warning at the top of the popover body when the sync state is `failed`, explaining that the latest edits didn't reach the server and that editing again retries the save. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * fix(drafts): surface the actual error message in the AutosaveIndicator popover Replace the generic "your latest changes did not reach the server" copy with the real failure detail. The syncer now stores the extracted message in the failures map (formatSaveError walks body / message / statusText) and exposes it via the state handle's `failureMessage` getter. Popover renders it in red, monospaced, scrollable so a long server traceback doesn't blow out the popover. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * fix(drafts): suppress Saving/Saved indicator during a reset-to-deployed discard A `value: null` POST is a discard, not a save, but it ran through the same runner the indicator watched — so resetting to deployed flashed "Saving..." → "Saved", reading as "your draft just landed" while we were actually wiping it. Track in-flight discards in a SvelteSet, expose a distinct `'discarding'` UserDraftSyncState, and the indicator stays quiet for it: no spinner, no label change, and the `discarding → none` transition deliberately skips the "Saved" flash. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * Revert "fix(drafts): suppress Saving/Saved indicator during a reset-to-deployed discard" This reverts commit 625a47c5d2. * fix(drafts): flush pending autosaves when the editor hook unmounts SPA navigation doesn't fire `pagehide`, so a debounced edit (up to maxDebounceMs old) silently disappeared when the editor was unmounted mid-typing. `UserDraft.useMany`'s onDestroy now walks every acquired entry and fires `UserDraftDbSyncer.flush(query)` before releasing, re-submitting the pending opts with `immediate: true`. The POST rides the runner's own lifetime and survives the component teardown. `use` / `useReactive` are thin wrappers around `useMany` so they inherit the flush automatically. Editors that don't go through the hook (sessions' `ScriptEditorView`, `AppJsonEditor`, copilot adapter, DraftBadge fork action) only call `UserDraftDbSyncer.save` for one-shot operations and don't need lifecycle flush. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * nit * feat(ui): Modal2 fixedHeight='adaptive' sizes the modal to its content The fixed-height steps force either wasted whitespace or clipped content for small dialogs. `adaptive` emits no height rule (still capped by max-h-screen-80) so the modal hugs its content. Use it in StaleDraftModal, which only has two lines of copy and a button row. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * feat(drafts): 'Create test drafts' button on the home page Dev/QA helper that seeds one per-user draft for every supported kind (script, flow, app, raw_app, trigger_schedule, resource, variable) at fixed u/{me}/draft_<kind> paths, so the draft surfaces (home badges, editors, stale-draft modal, others' drafts modal) can be exercised without hand-creating items. Re-clicking overwrites the same paths. Value shapes mirror what each editor's autosave writes, matching the backend list synthesizers that parse them back. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * fix(drafts): dedupe app list rows when a path holds both app and raw_app drafts The apps list LEFT JOINed draft with typ IN ('app', 'raw_app') for the is_draft flag — a path holding BOTH kinds for the same user (easy to hit: open a raw-app draft path in the regular app editor and its autosave writes the second kind) fanned the row out into two identical entries and crashed the home list with each_key_duplicate. Join a DISTINCT (path, workspace_id) subquery instead. Same dedup for the draft-only synthesis block via DISTINCT ON (path) keeping the most recently saved kind. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * feat(drafts): asterisk on resource/variable/schedule/trigger rows with own draft Add an is_draft flag to ListableVariable / ListableResource / ScheduleLight / BaseTrigger list rows — a scalar EXISTS subquery on the draft table for the authed email (no join, so no row fan-out), plus is_draft: true on the synthesized draft-only rows. The list pages (variables, resources, schedules, all trigger kinds) append `*` to the displayed name when set, mirroring the home page's convention. Also fixes draft-only resources never appearing on the resources page: the page always lists with resource_type_exclude=cache,state,app_theme (its tab split) and the synthesis gate bailed on any type filter. The gate now keeps synthesizing and applies resource_type / resource_type_exclude per-row against the draft JSON instead. list_triggers (trait default) takes an authed_email: Option<&str> — Some from the list endpoint, None from workspace export. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * Revert "feat(drafts): 'Create test drafts' button on the home page" This reverts commit 1f244a2a8b. * fix(drafts): P1 hardening — save authz, secret scrubbing, hot-path index 1. save_draft had no authorization check (a regression from the old create_draft's require_writer_of_path): any workspace member could plant drafts in another user's u/ namespace or unwritable folders, and those drafts get surfaced to every reader of the path (home circles, others'-drafts modal, View JSON / Fork). New require_can_write_path: admins; own u/ namespace; g/ namespace when in the group; f/ folders with the write/owner bit (with the same folder-claim refresh deploy endpoints use). Operators are rejected outright — they're excluded from every other draft surface. 2. Secret variable values were persisted in the draft table in plaintext. save_draft now blanks variable.value for is_secret drafts at write time (the editor never round-trips secret values anyway — it fetches with decrypt_secret=false), and a migration scrubs rows persisted before the guard. 3. fetch_other_drafts_users runs on every get-by-path request with (workspace_id, path, typ) and no email predicate — neither partial unique index covers it, so it seq-scanned a table that accumulates per-user autosaves across all workspaces. Add a plain btree index; it also serves get_draft_for_user's IS NOT DISTINCT FROM lookup. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * fix(drafts): Ctrl/Cmd+S flush narrates via the indicator, not a toast The "Draft saved" toast fired even with the network down — flush never rejects (postSave catches errors internally and routes them to the failures map), so the success branch always ran. Drop the toasts from the script / flow / raw-app Ctrl+S handlers; the AutosaveIndicator already narrates the flush truthfully (Saving... → Saved / Save failed in red). Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * fix(drafts): Ctrl/Cmd+S always flashes Saved in the indicator After dropping the toast, an explicit Ctrl/Cmd+S with nothing pending (the common case — autosave already landed everything) gave zero feedback: flush() no-ops when pendingSaveOpts is empty and no state transition fires. flush() now bumps a reactive per-key counter on completion (no-op path included), exposed as flushCount on the state handle; the AutosaveIndicator flashes "Saved" on the bump when the pipeline is idle. Real flushes keep narrating through Saving... → Saved / Save failed as before. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * ui(drafts): Ctrl/Cmd+S replays the green backdrop flash on the indicator Decouple the one-shot light-green → transparent backdrop from the load hint label: triggerFlash() owns the keyed span (mounted only while the animation runs), and both the on-mount hints and the Ctrl/Cmd+S confirmation route through it. The flush bump fires after the POST lands, so a real flush flashes too — not just the no-op path. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * feat(drafts): 'Create test drafts' button on the home page Dev/QA helper that seeds one per-user draft for every supported kind (script, flow, app, raw_app, trigger_schedule, resource, variable) at fixed u/{me}/draft_<kind> paths, so the draft surfaces (home badges, editors, stale-draft modal, others' drafts modal) can be exercised without hand-creating items. Re-clicking overwrites the same paths. Value shapes mirror what each editor's autosave writes, matching the backend list synthesizers that parse them back. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * fix(drafts): Ctrl/Cmd+S reaches the raw-app flush from every editor surface The raw-app window keydown handler never fired in practice: the file editor is a VS Code workbench in a same-origin iframe (keydowns don't cross documents) and the inline-script / YAML Monacos swallow Ctrl+S via addCommand. Two hooks: - attach a capture-phase keydown listener inside the iframe document on each load (no preventDefault — VS Code's own save still runs, we flush the pending autosave alongside it); - Editor.svelte / SimpleEditor.svelte re-broadcast their swallowed Ctrl+S as a `wm-monaco-save-shortcut` window event, which RawAppEditor listens for. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * fix(drafts): editing a draft-only item opens create mode prefilled from the draft Variable / resource / schedule / trigger editors treated every loaded path as deployed and routed saves through the update endpoints, which 404 for draft-only items ("Resource not found at name ..."). The get-by-path responses already mark the case (`no_deployed` from fetch_draft_only) — editors now flip to create mode when it's set: - VariableEditor / ResourceEditor: existedInitially = !no_deployed - ScheduleEditorInner + all 10 trigger editor inners: loadTrigger / loadSchedule return { overlay, noDeployed } and openEdit sets edit = !noDeployed The form opens prefilled from the draft and deploys via create, whose endpoints already delete the creator's draft on success. (The "Could not load schedule: Not Found" half of the report was a stale dev backend — getSchedule?get_draft=true verified working on the current build.) Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * fix(drafts): leading-edge draft saves for raw apps (no double debounce) Raw-app file changes reach the parent already coalesced — the UI Builder iframe holds a ~1s trailing debounce on its rebuild and only posts setFiles when it fires. The syncer then stacked its own 1.5s trailing window on top, so the draft landed ~2.5s after the user stopped typing. The debouncer now supports a leading edge (run immediately when the key is idle and cooled down; later schedules in the window coalesce trailing with the max-wait ceiling, mirroring the classic editor's first-keystroke-materializes-immediately logic), and raw_app saves opt into it. The app build keeps its own trailing debounce inside the iframe — only draft persistence is affected. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * ui(drafts): blue flash for load hints, green for save confirmations The backdrop flash now carries meaning: green = "your save landed" (Ctrl/Cmd+S), blue = informational on-mount hints ("Loaded from draft", "Others are working on this ..."). Color is passed as an inline CSS custom property the keyframe reads, so the single keyframe serves both variants. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * Revert "fix(drafts): leading-edge draft saves for raw apps (no double debounce)" This reverts commit 1b996fd73a. * feat(drafts): 'Enable auto-save' toggle in the AutosaveIndicator popover Browser-wide preference (default on, persisted in localStorage). While off, the reactive keystroke mirror never POSTs — saves marked `auto: true` park their latest opts in pendingSaveOpts instead of scheduling, and the unload keepalive flush is skipped, so nothing leaves the tab except explicit actions: Ctrl/Cmd+S flush (sends the parked latest content), discard / reset-to-deployed, fork, conflict overwrite. The indicator shows a muted cloud-off while disabled (the idle check-mark would otherwise read as "everything saved") and the popover copy explains the Ctrl/Cmd+S-only behavior. Re-enabling re-schedules every parked unsaved draft so edits made while off catch up immediately. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * Revert "feat(drafts): 'Create test drafts' button on the home page" This reverts commit fd7013b399. * feat(drafts): Review & Deploy covers variables/resources/schedules/triggers The drafts review page only assembled scripts/flows/apps from three paginated list endpoints, so drafts of every other kind were invisible. New GET /w/{ws}/drafts/list returns every draft of the authed user in one query over the draft table, with a per-kind draft_only flag (deployed-table EXISTS per kind); getDraftItems switches to it, which also drops the 3×N-page fan-out. CompareDrafts renders the new kinds (icon via a UserDraftItemKind → layout-Kind mapping, gray kind badge, list-page edit links for drawer-based editors), diffs them through a generic overlay GET, and deploys them by replaying the editor save: create/update for variables and resources, saveScheduleFromCfg for schedules, the per-kind save*TriggerFromCfg helpers for the ten standalone trigger kinds. Also fixes two paths stale since the draft_only column removal: draft-only flows/apps now deploy via create (update 404s — there is no row anymore), and discard always deletes the draft row (the old delete-the-item branch 404'd for the same reason). Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * feat(drafts): optimistic asterisk while editing in list-page drawers The `*` suffix on variable/resource/schedule/trigger rows came from the server's is_draft flag, which only updates on a refetch — editing an item in the drawer didn't mark its row until much later. New localDraftHints module (SvelteSet-backed): editors publish their dirty state (the same condition that shows the "You have unsaved changes" banner) and the 13 list pages OR the hint into the asterisk condition, so the suffix appears the moment the form diverges and clears on discard/teardown. Wired once in useTriggerDraftSync (covers the schedule editor and all ten trigger editors) plus VariableEditor and ResourceEditor. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * ee repo * fix(drafts): draft hints persist past editor teardown, re-sync on reopen Clearing the optimistic asterisk on drawer close was wrong: the divergence the editor observed is autosaved server-side, so the draft outlives the drawer and the asterisk should too. Hints are now corrected rather than expired — while an editor is settled on an item it publishes the observed truth in both directions (divergence sets, sitting at the deployed baseline clears), so a draft discarded from another tab loses its stale asterisk the next time the item is opened. No teardown cleanup anywhere. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * fix(drafts): list-page asterisk mirrors the editor's banner, not stale is_draft The asterisk was `is_draft || hint` — an OR can turn the asterisk on optimistically but can never turn it OFF, so after discarding a draft (or editing back to the deployed value) the stale server flag kept the asterisk until the next list refetch. Make the local hint a tri-state override instead: the editor publishes the live banner state (true/false) into a SvelteMap, and the list pages read `getLocalDraftHint(...) ?? is_draft` — the editor's observed truth wins over the stale server flag in both directions. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(drafts): autosaves equal to the deployed value delete the draft instead When the user edits back to exactly the deployed value, the reactive autosave mirror used to persist a baseline-equal copy — a useless draft row that kept `is_draft` (and the list asterisk) on after refetch. Add a `discardIfEqualTo` baseline getter to `UserDraft.useMany` specs: when the cell's value deep-equals the deployed baseline, the mirror POSTs `value: null` (delete) instead of the value. The variable and resource editors pass their `initialStates` baseline, guarded on `existedInitially` — draft-only/new items have no deployed copy, so equality must never delete their only data. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * Draft encryption for secret variables * fix(drafts): discardIf predicate + deploys clear the asterisk and draft row Two follow-ups on the baseline-equal-autosave-deletes change: 1. `discardIfEqualTo` (baseline getter + raw deepEqual) becomes `discardIf` (predicate). Raw deepEqual reported spurious diffs after a refresh: drafts round-trip through JSON, which strips undefined-valued keys, so a restored draft (`{}`) never compared equal to the freshly built baseline (`{ labels: undefined }`) and the delete never fired. The editors now pass the SAME comparison that drives their "unsaved changes" banner — a new exported `draftValuesEqual` (JSON-normalized deep equality) used by both — so the banner and the synced draft can never disagree. 2. Truly saving (deploying) clears the asterisk and the draft row: - variable/resource editors: replace post-deploy `UserDraft.remove` (blanks the cell to `undefined`, which reads as dirty and keeps the banner + asterisk on) with `discard` to the just-saved state, and refresh `initialStates`/`existedInitially` so the editor settles clean. - trigger editors: `useTriggerDraftSync.discard` publishes the hint off explicitly — after a deploy the editor's `deployed()` baseline is stale, so the hint effect alone would keep the asterisk on. - Review & Deploy page: `deployDraft`/`discardDraft` clear the hint. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * revert encryption just for the resources part * fix(drafts): required const DRAFT_KIND on TriggerCrud; deploy/delete cover raw_app The TriggerCrud::user_draft_item_kind() default matched on TRIGGER_TYPE and panic!'d on any unmapped string — a runtime crash on the first draft save for a trigger that forgot to map. Replace it with a required associated const DRAFT_KIND, so a missing mapping is a compile error. user_draft_item_kind() now just returns Self::DRAFT_KIND; every impl (OSS + EE) declares the const. Also fix the app deploy/delete draft cleanup to cover raw_app: raw apps deploy and delete through the same internal path, but the cleanup filtered typ = 'app' only, leaving raw_app drafts dangling (create_app_internal apps.rs:1465, update path apps.rs:2077) or un-archived on delete (apps.rs:1687). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(drafts): deleting an item wipes every user's draft, not just the caller's Scripts/flows/apps already wiped all users' drafts on delete, but resources/variables/schedules/triggers called delete_user_draft (caller-scoped), so a teammate's draft on the just-deleted item lived on forever — surfacing through fetch_other_drafts_users with no item left to deploy onto. Add delete_all_drafts_for_path (all emails + the legacy NULL row) and use it in every delete handler; keep delete_user_draft for the discard-my-own-draft flow where the item lives on. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * perf(drafts): skip other-drafts query on non-editor reads (get_draft=false) maybe_overlay_draft ran fetch_other_drafts_users (a usr join) on every get-by-path, including worker/CLI reads of MB-scale flows & apps that pass get_draft=false and never render the draft overlay or "others editing" surfaces. Gate the query behind get_draft — only editor reads pay for it. Reset-to-deployed editor reloads still get it (they pass get_draft=true). (Eliminating the serde_json::to_value materialization of the deployed payload needs WithDraftOverlay to become generic over T, which is folded into the get-by-path choreography refactor.) Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * refactor(drafts): single-source the kind→table mapping via deployed_table() The kind→table dispatch lived in three places that could drift: the TriggerCrud string-match (already replaced by const DRAFT_KIND), the table_for_kind access-check map, and a hand-written draft_only CASE in list_drafts. Add UserDraftItemKind::deployed_table() as the single source (plus an ALL enumerator). table_for_kind now delegates to it, and the list_drafts draft_only CASE is generated from it at runtime (table names come from the closed enum, never user input — no injection). Drift between the access check and the existence check is now impossible by construction. Webhook and the native triggers (poll/cli/nextcloud/google/github) map to None: they have no path-keyed backing table and aren't draftable, so they report draft_only=true and use a path-only access check. This also fixes a latent bug where table_for_kind mapped native kinds to native_trigger, which has no `path` column — the access query `SELECT 1 FROM native_trigger WHERE path = $1` would have errored. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * ee repo * fix(drafts): close variable draft-secret laundering oracle (sentinel + rehydrate) save_draft encrypts secret variable values with the workspace key, but the ciphertext was round-tripped to the client and the deploy endpoints decrypted whatever $encrypted: ciphertext the client submitted (variables.rs create/update). Any workspace member who can write a variable path could take an arbitrary workspace-key ciphertext (another user's secret draft via GET /drafts/get with only path-read, or a deployed secret's stored value) and submit it as their own secret variable's value — the server decrypted it and, since they own the path, they read the plaintext back. That bypasses the audited decrypt_secret permission. Fix: the ciphertext never leaves the server. get_variable swaps a draft secret's $encrypted: value for an opaque $draft_secret sentinel (both the draft overlay and the draft-only inner stand-in). On deploy the client sends the sentinel back and the server rehydrates the plaintext from the caller's OWN draft row — the only ciphertext it ever decrypts is one it encrypted for this exact (workspace, path, email). A raw $encrypted: submitted by a client is now rejected outright. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(drafts): don't clobber a secret draft when autosaving the $draft_secret sentinel After reload the client holds the $draft_secret sentinel for a secret variable (never the ciphertext). Editing some OTHER field (description, labels) triggers an autosave carrying value="$draft_secret" — and save_draft's encrypt_secret_variable_value, seeing a non-empty, non-$encrypted: string, encrypted the literal sentinel, overwriting the real ciphertext in the draft row and losing the secret. Treat the sentinel as "secret unchanged": restore the $encrypted: ciphertext already stored in this user's draft row instead of encrypting the placeholder (falling back to empty only if there's no prior ciphertext). The new lookup reuses the same query shape as the deploy- time rehydrate, so no new offline cache entry. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * Revert "$draft_secret" sentinel approach for variable draft secrets Reverts 339c259fce and b2c38ef407. Instead of round-tripping a sentinel and rehydrating server-side, we close the laundering vector more simply by disabling cross-user draft visibility for triggers/resources/variables (next commit) — an attacker can no longer read another user's secret draft ciphertext to launder it. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * feat(drafts): keep drafts private to their owner for resource/variable/trigger kinds Replaces the reverted $draft_secret sentinel: instead of laundering-proofing the ciphertext round-trip, simply don't expose other users' drafts for the drawer kinds (resource/variable/triggers). A viewer can no longer obtain another user's secret-variable draft ciphertext, so it can't be laundered into plaintext via deploy. UserDraftItemKind::shares_drafts_across_users() — true only for script/flow/app/raw_app. maybe_overlay_draft skips other_drafts_users for non-sharing kinds, and get_draft_for_user (View JSON / Fork) returns 404 for them. Own-draft load/save is unchanged. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * refactor(drafts): make the list-page asterisk hint a shadow of UserDraftDbSyncer The optimistic `*` hint was written by three open-editor publishers, so draft deletions that didn't go through an editor (banner discard, autosave-back-to-baseline, Review & Deploy) left a stale asterisk that a server refetch couldn't clear (the hint overrides is_draft). Move ownership to the syncer — the one choke point where a draft's existence actually changes: - postSave sets the hint on a saved write (value !== null) and clears it on a delete (null), so every syncer-routed delete clears it for free. - save() lights it optimistically when a real save is scheduled, so the asterisk still tracks the editor's banner without the debounce lag. The editors no longer SET the hint; they only CLEAR it when settled at the deployed baseline (so a draft discarded from another tab disappears on reopen). discardDraft drops its explicit clear (postSave covers it); deployDraft keeps one (it deletes server-side, bypassing the syncer). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * chore(migrations): fold draft index + secret scrub into the base sync migration Merge 20260610095349_draft_workspace_path_typ_index and 20260610100018_scrub_secret_variable_drafts into the base 20260528143710_draft_user_sync_schema migration (the index creation + secret-draft scrub in .up, the index drop in .down; the scrub stays irreversible). 20260609165313_remove_draft_only remains standalone. Verified the full chain applies and reverts cleanly on a fresh DB. (Rewrites an already-applied migration — existing dev DBs need a reset.) Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * refactor(drafts): promote the get-by-path draft choreography to one helper The "Some(deployed) → overlay / None+get_draft → draft-only / None → 404" dance was copy-pasted across the get-by-path handlers and had drifted (different 404 text, the trigger one missing the draft-only fallback at first). Promote it to windmill_common::overlay_or_draft_only<T>, which takes the deployed entity as Option<T> and a per-route not_found closure. Converts scripts, flows, apps, schedules, and triggers onto it. Resources keeps its own (it runs an async explain_resource_perm_error on the 404 path) and variables keeps its own (secret-decrypt logic interleaved with the draft fetch) — both genuinely diverge from the common shape. (The serde_json::to_value elimination via a generic WithDraftOverlay<T>, and the list-only draft synthesis dedup, remain as follow-ups.) Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * perf(drafts): serialize the deployed overlay payload in one pass maybe_overlay_draft materialized the deployed entity into a serde_json::Value tree (serde_json::to_value) and then serialized that tree again into the response — two passes plus a full Value allocation over what can be an MB-scale flow or app, on every get-by-path (including get_draft=false worker/CLI reads). Hold WithDraftOverlay.inner as a boxed erased_serde::Serialize trait object instead, so the deployed payload flattens straight into the response in one pass. The struct stays non-generic, so the helper and all seven handler return types are unchanged; only the deployed type now needs Send + 'static (already true — they're owned rows; added 'static to TriggerCrud::Trigger to say so). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * refactor(drafts): one helper for the draft-only list synthesis query The "draft rows at paths with no deployed counterpart" query was copy-pasted into the variable / resource / schedule / trigger list handlers, each hardcoding its own typ literal and NOT EXISTS table — a drift hazard. Promote it to windmill_common::fetch_draft_only_list_rows, which derives the absence-check table from kind.deployed_table() (the same single source as the access check and draft_only flag). Each handler keeps its own include_draft_only gating and per-type row mapping (genuinely entity-specific); only the shared SQL is deduped. The trigger handler's prior generated-SQL version is folded in too. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(drafts): route raw-app draft deploys through the raw-app endpoint [P1] deployDraft's raw-app guard was `kind === 'app' && rawApp`, but Review & Deploy passes `kind === 'raw_app'` (raw apps are their own DRAFT_KIND), so the guard never fired and the row fell into the visual-app branch. There `d.value` is undefined (a RawAppDraft has files/runnables/data, no `value`), so AppService.updateApp did a partial update — resetting policy to the publisher default, never bundling/deploying the files — the backend then deleted the user's raw_app draft rows, and the UI reported "deployed". The work-in-progress was destroyed without ever deploying. Route `kind === 'raw_app'` (or the editor's `app` + rawApp) through deployRawAppDraft. The now-unreachable `raw_app` arm of the visual-app branch is dropped. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(drafts): allow draft saves for item-level extra_perms writers [P1] require_can_write_path only accepted namespace rules (own u/, member g/, writable f/), dropping the item-level extra_perms check the old create_draft had. A user granted write on e.g. u/alice/script via the Share dialog could still deploy it (the update endpoints go through RLS) but could no longer save a draft — and because the editors autosave continuously with no permission gate, editing a shared item produced a persistent "Save failed: you don't have write permission" and Ctrl/Cmd+S failures. Add the item-level fallback: when a deployed row exists at the path, check its extra_perms for a write grant (every deployed table has extra_perms; the table comes from the closed deployed_table() mapping). Draft-only items have no row and stay governed by the namespace rules. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(drafts): pass rawApp on get-app for never-deployed raw-app drafts [P2] A raw app that has only ever been drafted has no `app` row, so get_app resolves the draft kind from the `rawApp` query param. getDraftDiffValues ("Show diff") and deployRawAppDraft both fetched with getDraft=true but without rawApp, so the backend looked up the visual-app draft kind, found nothing, and 404'd. Pass rawApp so the raw_app draft is found. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * feat(drafts): surface the localStorage→DB migration with toasts migrateUserDraftsToDb already uploaded legacy "userdraft/..." entries and cleared them on success (and runs after the v1→userdraft normalizer). Add the user-facing surface: when real legacy entries are detected, show an info toast "Migrating local storage drafts ..."; on a per-draft failure show an error toast "Could not migrate draft <path> in workspace <X>" with a "Delete draft" action that drops the stuck localStorage entry (otherwise it retries every mount). Unparseable junk is still cleared silently up front, so the toast only fires for genuine drafts. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * test(drafts): cover the autosave pipeline's pure-logic utilities [P2] The deleted draft tests left the new debouncer + coalescing runner — the core of the autosave pipeline — with zero coverage. Add vitest suites (16 cases) for debouncerByKey (debounce window, latest-task-wins, maxDebounceMs ceiling under a trickle, fresh-chain-after-fire, cancel, key independence) and coalescingRunner (immediate run when idle, coalesce burst to in-flight + latest, displaced-task drop, submitAndWait resolve/reject/displaced, cancel semantics, key independence). Broader replacement (save_draft conflict semantics + the require_can_* checks as backend integration tests) still outstanding. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * feat(drafts): add UserDraft.seed — a one-shot baseline load that never POSTs The page editors bracket their new-draft / deployed-baseline loads with stopSync + restartSync so the programmatic write isn't synced as the user's edit. Forgetting restartSync silently disables autosave for the session — the footgun behind the three divergent resume strategies the review flagged. `UserDraft.seed(kind, path, value)` is the scoped alternative: it sets the cell (all reactive readers update) and arms a single-shot `seedNextWrite` flag the sync effect consumes — adopting the value as the new baseline and skipping exactly that one POST, with no suspension to resume. Additive: stopSync/restartSync are untouched and still used for the writes that fan out across editor components (initContent cascades). Foundation for converting the editor bootstraps off the bracket. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * refactor(drafts): extract usePageDraftSync; convert the scripts editor onto it First step of unifying the four page editors' hand-rolled draft orchestration (three divergent handle-ownership models + an easy-to-forget recordRemoteSync). usePageDraftSync is the single model — the page analogue of useTriggerDraftSync — owning the re-keyed autosave handle, the live-editor-draft registry entry, recordRemoteSync (now a method, not a per-page ritual), seedBaseline (via UserDraft.seed), and draft removal. The scripts editor is converted as the reference adoption: its inline useReactive handle, live-editor-draft effect, recordRemoteSync, and the two UserDraft.remove calls now go through draftSync. The new-draft stopSync bracket stays (it spans ScriptBuilder's initContent cascade). Verified in a real browser against the dev stack: load fires no spurious save, a code edit triggers exactly one save_draft POST + a draft row, and the draft persists across reload. Flows / apps_raw / apps conversions follow. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * refactor(drafts): convert the flows editor onto usePageDraftSync Replace the inline useReactive handle + UserDraftDbSyncer.recordRemoteSync + UserDraft.remove with draftSync. effectivePath is omitted — flows register their live-editor-draft entry through FlowBuilder (liveEditorDraftStoragePath), so the composable doesn't double-register. The new-draft stopSync + armRestartOnFirstInteraction bracket stays (it spans FlowBuilder's seed cascade). flowStore reads/writes draftSync.draft. Verified in a real browser: load fires no spurious save, a summary edit triggers exactly one save_draft POST + a draft row, and the edit persists across reload. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * refactor(drafts): convert the apps_raw editor onto usePageDraftSync Replace the UserDraft.use handle + mirror, UserDraftDbSyncer.recordRemoteSync, and UserDraft.remove with draftSync. `path` is a mount-scoped plain `let` (the editor remounts per path), so the composable's useReactive re-keys only on workspace change — equivalent to the prior capture-once use(). effectivePath omitted (RawAppEditor owns the live-editor-draft entry); the new-draft stopSync + armRestartOnFirstInteraction bracket stays. Type-checked and behavior-equivalent (handle mechanism unchanged; the centralized recordRemoteSync/remove read the same `path`). Not browser-exercised here — no existing raw app in the dev workspace and the new-draft template-picker flow isn't scriptable quickly; scripts and flows (same composable) were verified live. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(drafts): remove app autosave at its canonical key after deploy/rename AppEditor keys the app autosave on the URL draft path and passes it down as userDraftPath, but AppEditorHeader's post-deploy cleanup re-derived the key from the just-typed deploy path (createApp) / the live $appPath (updateApp) instead. For a new app the autosave lives at u/{user}/draft_{uuid} while the typed path is the user's chosen name, and a rename leaves the autosave at the original key — so removing at path/$appPath missed the real draft row and orphaned it. Use the canonical userDraftPath AppEditor already provides. This is the "children re-derive the UserDraft key" fragility from the review, addressed without giving apps a page-level handle — apps deliberately lets AppEditor own the handle so the entry is destroyed on unmount (a page handle would keep it alive and reintroduce spurious autosaves on every /edit visit). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * test(drafts): integration tests for save_draft conflict semantics + authz [P2] Replaces the deleted drafts.rs (which targeted the removed /drafts/create API) with tests for the new surface: - save_draft upsert → stale-last_sync conflict (rejected, value unchanged) → force overwrite → delete, the optimistic-concurrency contract. - require_can_write_path: own namespace allowed, another user's namespace rejected, operators rejected. - the item-level extra_perms fallback — a user granted write on a deployed item can save a draft on it (regression test for the authz drop). - cross-user draft privacy: GET /drafts/get is 404 for the drawer kinds (variable/resource/triggers), not blocked for script/flow/app. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * chore(sqlx): refresh offline cache after the main merge The merge auto-combined both branches' additions inside the resource get-by-path query_as! (our draft_only/is_draft columns + main's folder_labels(...) inherited_labels), producing query text neither branch had cached — so the offline build failed for it. Regenerate the entry (rename to the new content hash) and refresh a re-described workspace query. Feature-gated/EE entries the local prepare can't compile are left as committed. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * ee repo ref * chore(system_prompts): regenerate for draft_only/is_draft trigger schema fields The openapi.yaml trigger/schedule schemas gained draft_only + is_draft, but system_prompts/generate.py wasn't rerun, failing the freshness check. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * refactor(drafts): defer save_draft write authz to RLS via a FOR UPDATE probe require_can_write_path re-implemented the item-level extra_perms write rule in Rust (SELECT extra_perms + get_perm_in_extra_perms_for_authed) — a third copy of rules whose canonical home is the RLS policies, and the exact lane that regressed once already. Replace it with an RLS write-probe: `SELECT 1 FROM {deployed_table} WHERE path/workspace ... FOR UPDATE` through UserDB. Postgres applies UPDATE policies to rows locked via FOR UPDATE, so a returned row means the canonical policies (see_own / see_member / folder-write / see_extra_perms_*_update / admin_policy) would let this user UPDATE the row — no write rule re-implemented, no drift possible. The probe's row lock is released by the immediate commit. The claim-based namespace checks stay, evaluated FIRST: they read the same JWT claims RLS does (so outcomes are identical), they spare the autosave hot path a DB round-trip for the common own-namespace case, and they are the entire check for draft-only paths — where no deployed row exists, so there is structurally nothing for RLS to evaluate. The u/own + folder-owner part now goes through the shared windmill_api_auth::require_owner_of_path instead of bespoke code. Adds a read-only-grant test case (extra_perms value false): the row is visible under the SELECT policy but FOR UPDATE filters it under the UPDATE policy — pinning the semantics the probe relies on. All 4 draft integration tests pass. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * chore: point ee-repo-ref at the EE branch merge (has DRAFT_KIND consts) ee-repo-ref was set to main's EE commit (d45b9a6) while the EE branch was unpushed; building OSS (which requires const DRAFT_KIND on TriggerCrud) against that EE ref fails with E0046 on every EE trigger impl. The EE branch head e936e9a — the merge of d45b9a6 into the EE remove-workspace-drafts branch, carrying the DRAFT_KIND consts — is now pushed; point at it. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(drafts): ignore permissioned_as fields in the unsaved-changes comparison The schedule cfg carries permissioned_as / preserve_permissioned_as — run-as deploy directives, not user-edited draft content — and the editor round-trips them asymmetrically (preserve_… is rebuilt as !!cfg.permissioned_as on load but `|| undefined` on build), so the banner comparison could report a phantom diff. Extract the normalization into a shared normalizeDraftForCompare (JSON round-trip + a DRAFT_COMPARE_IGNORED_FIELDS list with the two fields) and use it from BOTH comparators: draftValuesEqual (variable/resource banner + discardIf) and useTriggerDraftSync's cfgDiffers (schedule and trigger banners, the persist-effect's at-baseline discard, restore) — one ignore-list, no way for the two to disagree. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * nit * fix(drafts): at-baseline discard is auto-gated and only fires with a draft Two related fixes to useTriggerDraftSync's persist-effect: 1. The reactive at-baseline discard bypassed the "Enable auto-save" toggle: with autosave off, value saves were parked (correct) but the discard's value:null still POSTed — so the editor never wrote drafts yet kept reactively DELETING them, and the only network traffic was discards. Thread `auto` through UserDraft.discard to the syncer; the persist-effect passes auto:true (parked for Ctrl/Cmd+S when the toggle is off), explicit discards (banner button, post-deploy cleanup, reset-to-deployed) stay ungated. 2. The discard fired unconditionally whenever the form sat at the deployed baseline — including a spurious value:null POST on every drawer open. Guard on cfgDiffers(h.draft, deployed): undefined on a fresh open (nothing to discard) and equal to deployed right after a discard (no repeat per cfg recompute). Verified live as a non-admin user on a schedule: toggle on → no POST on open, edit → one value save, revert → one discard; toggle off → zero POSTs (everything parked), banner still functional. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * feat(drafts): scope the "Enable auto-save" toggle to the page editors Add a canBeDisabled opt (default false) to UserDraft.use / useReactive / useMany specs, threaded through acquireEntry into the reactive mirror's save opts. The syncer's auto-save gate (and the pagehide-flush skip) now only applies to saves whose handle opted in: the four full-page editors — script / flow / raw app via usePageDraftSync, app via AppEditor's use() — which are exactly the surfaces whose AutosaveIndicator carries the toggle. Drawer editors (variables / resources / schedules / triggers) keep the default and always sync regardless of the toggle — previously a toggle flipped off in some browser silently disabled their autosave and the optimistic asterisk (both sit behind the same gate) with no toggle UI anywhere on those surfaces to explain it. Verified live: schedule edit with the toggle off now POSTs the value save (and the discard on revert); script editor with the toggle off still parks everything for Ctrl/Cmd+S. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(drafts): consume the import handoff stores in the new-draft bootstrap The /add pages used to read importStore / importFlowStore / importScriptStore / sessionStorage rawAppImport to seed the editor from "Import from YAML/JSON", "Build app" (from a script/flow), and the workflows-as-code import. Since /add became a pure redirect to /{kind}/edit/u/{user}/draft_{uuid}?new_draft=true, the writers kept firing but nothing consumed the payload — every import landed in an empty editor. Consume them (one-shot read + clear) in the four edit pages' new_draft branches, layering the imported content over the empty template with path kept '' so the friendly-name generation still runs: - scripts: $importScriptStore spread over the empty script (non-empty content also keeps ScriptBuilder's template bootstrap from overwriting it — that cascade is gated on content == ''). - flows: $importFlowStore spread over the empty flow. - apps: $importStore — wrapped exports ({summary, value, policy}) and bare App values, mirroring main's /add. - raw apps: $importStore then sessionStorage rawAppImport (the full page reload for cross-origin isolation drops in-memory stores); honored only when the payload carries files (rendering gates on them), skipping the framework picker; otherwise the template seed. Verified live: "Build app" from a script lands on /apps/edit with the canvas seeded from the script instead of an empty editor. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * chore(drafts): remove dead delete_user_draft + its stale doc [C4] The doc claimed item delete handlers call it, but those all moved to delete_all_drafts_for_path (an item delete is for everyone); the caller-scoped discard goes through the save_draft route with value:null. That left delete_user_draft with zero callers (OSS and EE) — remove it and its orphaned sqlx cache entry, and reword the contrast note on delete_all_drafts_for_path. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * chore(migrations): retire the sync_drafts-era index comment + right-size it [C6] The draft_user_sync_idx comment described the deleted sync_drafts polling endpoint (editors polling created_at ranges every 2-10s) — that design was replaced by recordRemoteSync + save_draft last_sync, and nothing range-scans draft.created_at anymore. Since this migration only exists on this branch, fix it before it ships: the index's real consumer is GET /drafts/list (workspace_id + email equality, ORDER BY path), so swap the vestigial trailing created_at for path (rows come back in output order) and rename to draft_user_listing_idx. Chain re-verified on a fresh DB. (Byte-for-byte migration edit — dev DBs that already applied it need a reset, as with the earlier consolidation.) Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(drafts): discardDraft awaits the delete POST before refetching [I5] UserDraftDbSyncer.save resolves at enqueue time for debounced saves, so discardDraft's await finished ~1.5s before the value:null POST and the invalidateWorkspaceDrafts refetch re-listed the just-discarded draft. Use immediate: true (resolves after the POST lands), matching every sibling delete-then-refetch path. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(drafts): replace stale draft_only gates in the builders [I6] draft_only was dropped from the get-by-path wire shape (the column is gone; overlays carry no_deployed instead), so these four reads were always undefined: - ScriptBuilder "Exit & See details" gate and TriggersEditor's isDeployed treated every draft-only script as deployed → now keyed on savedScript.no_deployed like the sibling reads right next to them. - FlowBuilder's deploy path never took the direct-save branch for draft-only flows (no deployed version exists to compare against), and "Exit & see details" was offered for draft-only flows (404 details page) → both now keyed on the newFlow prop (driven by no_deployed), which the rest of the file already uses. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(drafts): Ctrl/Cmd+S flushes the draft in the low-code app editor [I7] The app editor's keydown handler swallowed the shortcut with a bare preventDefault() — every other page editor flushes the pending autosave (UserDraftDbSyncer.flush) so the AutosaveIndicator narrates Saving... → Saved and parked edits (autosave toggle off) actually persist. Wire the same flush, skipped in the AI session pane where no UserDraft handle exists. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(drafts): AI tool strings no longer describe drafts as localStorage [C2] The copilot tool results/messages still told the model drafts were "saved to local storage" / "a browser-only local draft" — drafts are per-user rows in the server-side draft table now. Misleading the model about the storage medium produces wrong explanations to users (e.g. "your draft will be lost if you clear your browser data"). Reword all occurrences to "draft" / "per-user draft (saved server-side)". Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(openapi): drop stale draft_only request props, fix OtherDraftUser, regen deref [D4][C5] - The create-script (NewScript), createFlow, createApp and createAppRaw request bodies still documented draft_only — the backend request structs no longer read it, so an older CLI sending draft_only: true is silently ignored and fully deploys. Remove the property from the spec so generated clients can't offer it. (Response-side draft_only on the Listable* rows stays — the list synthesis populates it.) - UserDraftOverlay.other_drafts_users item schema declared email and a required draft_saved_at; OtherDraftUser serializes only username (nullable for the legacy row — emails never leave the server). Align the schema. [C5] - Regenerate openapi-deref.yaml/.json (served at runtime via include_str!) — they still advertised getScriptByPathWithDraft and the deleted draft surface, and now carry the drafts/save_draft routes. Frontend gen client regenerated; check:fast clean. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(sessions): stop session pane from clobbering server-side raw-app drafts [P1] loadRawApp seeded the session runtime from result.value (the deployed payload), ignoring the .draft pocket returned by the get-by-path overlay. The subsequent UserDraft.save then POSTed deployed content with no last_sync recorded, silently overwriting the user's server draft. Now the no-draft branch consumes result.draft when present (matching the flow/script branches) and records draft_saved_at via recordRemoteSync so later session saves are conflict-checked instead of treated as fresh. Also corrects the header and aiDraft-branch comments that claimed the overlay merges drafts into top-level fields — it never does. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * fix(rust-client): pass new get_draft arg to variable_api::get_variable getVariable gained a GetDraft query parameter (per-user draft overlay), so the generated client fn takes a sixth argument. Verified with the same generate+check pipeline CI runs (rust-client/dev.nu --check). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * nit: Workspace fork mention * fix(drafts): don't leak other_drafts_users on draft-only private kinds [P2] fetch_draft_only built the other_drafts_users list unconditionally, while the deployed-overlay path gates it on shares_drafts_across_users. For the drawer kinds (resource/variable/triggers) drafts are private to their owner, so a draft-only GET was the one route that still told a viewer who else has a draft at the path. Apply the same kind gate. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * perf(drafts): probe a single row in the RLS write-probe [P2] The script table keeps one row per version at the same path, so the FOR UPDATE probe locked the entire version history and serialized against concurrent deploys. LIMIT 1 locks one row — any UPDATE-policy visible row proves writability (same pattern as scripts.rs's latest-version lock). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * fix(drafts): consume the /add?param= seeding intents in new_draft branches [D2] The /add routes' redirect preserves query params, but the edit pages' new_draft branches only consumed the YAML/JSON import stores — every other intent the old /add pages handled landed in a blank editor: - scripts: ?hub= and ?template= forks (with locked language and a `<source>_fork` path suggestion), ?wac=python|typescript (WAC editor template + language), ?lang=, ?initial_args= (URL form), and the base64-JSON #hash payload (run page "Fork", workspace_settings handler-template buttons; WAC detection restored for imports too) - flows: ?hub= (preprocessor placeholder replacement + env-variables panel), ?template=/?template_id=, ?fork=true (fork_flow localStorage / window.opener handoff), #state, ?tutorial= - apps: ?hub= (fromHub inputs panel), ?template=/?template_id=, ?tutorial= The redirect itself also dropped the URL hash — SvelteKit forbids url.hash in load, so it forwards window.location.hash (correct for all hash producers: they arrive as full page loads via window.open / target=_blank). Seeding priority and toasts mirror main's /add pages. Verified live: hub/template/wac/hash/fork intents for scripts and flows, hub for apps (dev hub returns empty payloads, code path confirmed via toast + inputs panel); no autosave POSTs fire during seeding. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * fix(drafts): LS→DB migration no longer clobbers fresher server drafts [P2] The one-off localStorage migration POSTed every entry with force: true, unconditionally overwriting whatever the user had since saved server-side from another browser. It now passes the LS copy's lastWrittenAt as last_sync (epoch 0 when absent), so the server's conflict rule arbitrates: empty slot → insert; server draft fresher → conflict, LS copy dropped; LS copy fresher → upload wins. Verified all three outcomes against the live save_draft endpoint. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * refactor(raw_apps): drop banned $bindable(default) on template picker open [P2] `open = $bindable(false)` on an optional prop is the AGENTS.md-banned pattern (the default masks the undefined state). The only caller always binds a boolean, so `open` is now a required prop with a plain `$bindable()`. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * fix(drafts): fork others' drafts via the import handoff, not an eager save The Fork actions (OtherUsersDraftsModal + DraftBadge popover) saved the fetched draft server-side immediately and navigated to the fork path, which surfaced three problems: a server draft existed before the user edited anything, the Path widget treated the slot as an existing item ("Only the owner can change the path"), and the value's draft_path kept the source path while the URL said X_owner_fork. Forking now routes through the same one-shot import handoff as the "Import from YAML/JSON" actions (new shared forkDraftToImport helper): stash the value in the kind's import store, navigate to /add, and let the new_draft branch seed a brand-new own item — nothing saved until the first real edit, fresh renamable path, no source identity riding along. The editPathFor/currentUserUsername plumbing that only served the old flow is removed from both fork surfaces and their callers. The new_draft branches also clear the previous path's draft-presence state (otherDraftsUsers, loadedFromDraft, stale-draft timestamps) — the page component is reused across same-route navigation, so forking from an editor with collaborators used to carry the "Others are working on this" hint onto the fresh draft. Verified live: fork of a legacy draft seeds content+summary on a fresh u/{user}/draft_{uuid} slot with zero save_draft requests and no leftover collaborator hints. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * refactor(drafts): replace deprecated Popover with meltComponents Popover - Migrate from old Popover.svelte to meltComponents/Popover.svelte - Convert to new trigger/content snippet pattern with openOnHover=true - Maintain hover behavior with debounceDelay=100 - Add key to visibleUsers each block for Svelte 5 compliance * feat(drafts): seed forked drafts with the source path in the forker's namespace Forking u/admin/myflow as guest now seeds the Path widget with u/guest/myflow instead of a random friendly name — everything after the source path's first two segments is kept, so f/folder/my/flow becomes u/guest/my/flow. The re-homed path travels from forkDraftToImport to the new_draft branches as a ?seed_path= param (the redirect preserves query params; plain ?path= would be eaten in transit by ScriptBuilder's legacy collab-param cleanup, which deletes path/collab from the live searchParams object). The script editor also passes initialPathChosen for any seeded path — MetadataGen fires onChange for a non-empty summary at mount, and the summary→path auto-slug would otherwise overwrite the explicit seed (hub/template forks and URL-hash payloads included). Verified live: forking a draft on u/admin/hard_working_script seeds path u/admin/hard_working_script (with the "path already used" warning), keeps the drafted summary/content, and still fires no save_draft until the first edit. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * fix(drafts): DiffDrawer "Restore deployed" actually discards the draft [P1] All four restoreDeployed implementations POSTed the delete through the debounced pipeline and reloaded with getDraft defaulting to true: the reload's draft write re-entered the autosave mirror (the one-shot seed guard was consumed on first load), and debouncerByKey displaced the queued value:null with the new save — the delete never reached the server and the editor re-rendered the draft it was told to discard. They now funnel through runResetToDeployed (the stopSync-bracketed delete the AutosaveIndicator reset already uses) with each page's proven reset body (getDraft: false reload), so the suspension mutes the mirror while the delete flushes and sync re-arms on first interaction. Also fixes the raw-app drawer navigating to the visual app editor (/apps/edit) instead of /apps_raw/edit [P2]. Verified live on the script editor: Restore deployed issues exactly one save_draft ({value:null} answered status=saved), the server row is gone, and the editor re-renders the deployed content. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * fix(drafts): deploying a draft-only item reliably deletes its draft Two bugs left the slot draft (u/{user}/draft_{uuid}) alive after a successful deploy: - RawAppEditorHeader.createApp removed the draft at the just-typed deploy path instead of the URL slot key (the visual header documents exactly this trap), orphaning the real row for every draft-only raw-app deploy. - Everywhere else the delete went through bare UserDraft.remove, which only QUEUES the value:null in the per-key debouncer. Editors that stay mounted through the post-deploy navigation (AppEditor, RawAppEditor — and timing-dependently the script/flow builders' post-deploy draft_triggers mirror) keep mirroring their working value, and one such write displaces the queued delete with a fresh save — observed live: deploying a new visual app re-saved the full grid value at the slot right after deploy. New discardDraftAfterDeploy helper (userDraftToast.ts) applies the same bracket runResetToDeployed uses: stopSync to mute the mirror, remove + immediate flush so the displacement window closes, re-arm on first interaction. Wired into the script/flow pages' onDeploy and both app headers' create/update paths (session-pane guards preserved). Verified live for all three kinds: draft-only deploy issues the value:null (status saved), the slot row is gone, and no post-deploy save re-creates it. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * fix(drafts): forks-compare deploy clears drawer-kind drafts too The script/flow/app deploy endpoints delete the deployer's draft server-side, but the drawer kinds' (variable / resource / schedule / triggers) create/update endpoints never touch the draft table — their editors discard client-side after a save. deployDraft replayed the save but not the discard, so "Deploy n drafts" on /forks/compare deployed those kinds correctly and left the drafts listed forever. deployDraft now issues the canonical value:null delete (immediate) for the drawer kinds after a successful save. Verified live: deploying a draft-only variable from /forks/compare creates the variable and the draft row is gone. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * fix(drafts): StaleDraftModal "Load latest deploy" actually discards the stale draft [P2] The modal invoked onLoadLatestDeploy directly — the draft = undefined write queued the delete and the reload's deployed-payload write displaced it, overwriting the stale draft with a deployed-identical copy (is_draft stuck on, asterisk persists, modal can't re-fire since draft_saved_at moved past the deploy). All four pages now run the callback through runResetToDeployed, same as the DiffDrawer restore. Verified live: stale-draft scenario → Load latest deploy → exactly one value:null POST, draft row gone, editor renders the newer deploy. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * fix(drafts): don't acquire a sync entry for empty-path specs [P2] The read-only historical-hash view (/scripts/edit/x?hash=...) computes draftPath '' but useMany still acquired a live entry at ws/script/ — every edit mirror-POSTed to /drafts/save_draft/script/ (unroutable), populating the failures map and pinning the AutosaveIndicator on "Save failed" with a retry per debounce window. Empty-path specs now get a detached local-only handle: bind: works, nothing syncs — which is what usePageDraftSync's doc always claimed. Verified live: editing in the hash view fires zero save_draft requests. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * fix(drafts): no spurious conflict after bfcache restore of a flushed page [P2] flushOnPageHide advances the server rows with unreadable keepalive POSTs and leaves lastSyncMap stale — correct when the document dies, wrong when bfcache resurrects it: the next autosave carried the pre-flush last_sync and the server rejected the user's own write as a conflict, opening DraftSyncConflictModal. The flushed keys are now remembered and dropped from lastSyncMap on pageshow with event.persisted, so the first post-restore save takes first-push semantics against this document's own flush. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * ui(drafts): draft asterisk sits on the trigger row's main title The draft hint rendered at the end of the secondary path line (u/admin/item*) on the http/websocket/nats/kafka/email trigger lists — easy to miss. It now renders at the end of the row's bold title, and on the azure/gcp lists it moves from mid-title (after the path, before the topic suffix) to the end of the line. mqtt/postgres/sqs/schedules already had it on the title. Verified visually on the HTTP routes list. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * fix(drafts): trigger editors save the FIRST edit, not the second Three interlocking fixes in the trigger autosave path: - The entry's one-shot first-write seed guard (skipNextWrite) was never consumed for trigger entries: the drawers don't write the cell on open (the form holds the state, unlike variables/resources which pass a defaultValue), so the guard stayed armed and silently swallowed the user's FIRST edit — banner on, no asterisk, no save until a second change. maybeRestore now seeds the cell with the post-load baseline (server draft overlay if any, deployed otherwise) via UserDraft.seed, consuming the guard without POSTing. - Guard hygiene in the cell's sync effect: a programmatic write consumes BOTH one-shot guards, and a no-op write (same serialization — e.g. the trigger pages fire openEdit twice per row click, re-seeding the same value) defuses a lingering seedNextWrite instead of leaving it armed to eat the next real edit. - The at-baseline auto-discard is now deferred + revalidated (600ms): with the cell seeded, the double-openEdit churn transiently shows form-at-deployed + cell-holds-draft and an immediate discard deleted the server draft on open; the recheck skips the transient state while a genuine user revert still discards. Verified live on the HTTP route editor: open-with-draft restores the draft with zero POSTs, the very first field edit saves, and reverting the form to the deployed value deletes the server draft. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * ui(drafts): underscore-separated uuids in draft slot paths u/{user}/draft_{uuid} now uses underscores instead of dashes in the uuid — path segments elsewhere in Windmill are [a-zA-Z0-9_] words and downstream consumers treat '-' as a foreign character. Nothing parses the uuid back, so existing dashed slots stay valid. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * fix(drafts): cascade draft cleanup on bulk-delete and rename Drafts have no SQL FK to their underlying items (only to password.email), so deletion and rename must cascade programmatically. Two gaps remained: - Bulk delete of variables/resources did not wipe per-user drafts at the deleted paths (single delete already did via delete_all_drafts_for_path). Cascade them — including the linked resource/variable rows the bulk delete fans into — so no orphaned draft-only rows survive. - Renaming a variable/resource/trigger left the per-user draft stranded at the old path. Add delete_own_draft_for_path and clear the deployer's own (+ legacy NULL) draft at the old path on rename, mirroring the script/flow/app rename path; teammates keep theirs (StaleDraftModal). Variable/resource renames also move the linked counterpart, so both kinds' drafts at the old path are cleared. Schedules have no rename path. Note: sqlx offline cache not yet regenerated for the new/changed queries. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * fix(drafts): surface legacy NULL-email drafts and migrate pathless /add keys Legacy workspace-scoped drafts (pre-per-user rows + the remove_draft_only migration, all email IS NULL) stopped showing up because every per-user lookup matched only email = self. Match (email = self OR email IS NULL) everywhere a draft is surfaced or opened, with the owned row taking precedence (DISTINCT ON / ORDER BY email NULLS LAST): the home drafts list, the script/flow/app/drawer draft-only list syntheses, and the get-by-path overlay/fallback. The localStorage->DB migration also dropped pathless legacy keys (userdraft/w/{ws}/{kind}/ with no path) — the new-item /add autosave — because parseKey rejected an empty path, leaving them stranded in LS. Mint a fresh u/{user}/draft_{uuid} slot for those (same convention as the editors' /add redirects) so they migrate as regular draft-only items. Note: sqlx offline cache not yet regenerated for the changed macros. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * chore(sqlx): regenerate offline cache for draft cascade + legacy-draft queries Adds the offline entries for the queries changed in the two preceding draft fixes (delete_own_draft_for_path, the maybe_overlay_draft/fetch_draft_only NULL-email fallback, and the script/flow/app draft-only syntheses). Also forwards the `http_trigger` feature from windmill-api-openapi to windmill-store: that crate imports `try_get_resource_from_db_as` unconditionally, but the fn is cfg-gated behind a trigger feature, so the openapi targets failed to compile in isolation (e.g. `--all-targets` under resolver 2) — which blocked `cargo sqlx prepare`. The feature was already present transitively in whole-workspace builds; this just makes it explicit where the symbol is used. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * fix(drafts): resolve own draft owner in the admins workspace The draft-owner surfaces (home-page badge, "others' drafts", View JSON / Fork) resolve a draft's email to a username via the `usr` table. The `admins` workspace has no `usr` rows — there a user's "username" IS their email — so the join missed every owner and returned NULL, which the badge renders as "Legacy workspace draft". A user editing a deployed item in `admins` thus saw their OWN draft plus the genuine legacy NULL-email row both labelled "Legacy workspace draft" (the reported duplicate). Add the identity fallback `COALESCE(u.username, CASE WHEN workspace_id = 'admins' THEN email END)` to the script/flow/app draft_users aggregations and fetch_other_drafts_users, and accept username==email in get_draft_for_user. The genuine legacy row keeps username NULL (its email is NULL, so the CASE yields NULL too), so it alone reads "Legacy workspace draft" while the user's own draft now reads "<email> (you)". Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * fix(drafts): keep "See others' drafts" after reset-to-deployed other_drafts_users is only computed by the backend when getDraft is true (the cross-user lookup is skipped otherwise). Reset-to-deployed reloads with getDraft:false, so the editors were overwriting the known list with the empty response — hiding the "See others' drafts" button until a full page reload recomputed it. Discarding one's own draft is independent of other users' drafts, which are untouched on the backend. Only assign otherDraftsUsers on a getDraft:true load. Applied to the script, flow, app and raw-app editors. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * disable fork for operators * Path reactivity issue * docs(drafts): tighten draft-feature comments and drop dead code The draft feature accumulated many multi-paragraph comments that risked code-comment drift. Compact them to the AGENTS.md bar (constraints not narration, state-once, no drafting-history), de-duplicating the repeated draft_users / cascade / draft_only-synthesis rationale to one canonical version per theme with terse cross-references elsewhere (~1300 fewer lines). Also fixes three stale/contradictory comments surfaced while trimming: - the operator authz note claimed operators are "excluded from every draft surface", contradicting require_can_read_path (they can read some drafts, never write) — reworded to match the code; - a migration comment named a non-existent index (draft_user_sync_idx); - a syncer comment documented the wrong map-key separator. Removes notifyDraftLoaded (orphaned exported helper, no callers). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * refactor(drafts): rename save_draft route to /update for CRUD consistency The draft write route was POST /drafts/save_draft/{kind}/{path}, which stutters with the /drafts prefix and uses a non-house verb. Rename it to POST /drafts/update/{kind}/{path} (operationId saveDraft -> updateDraft) to match the codebase's CRUD convention (/list, /get/{path}, /update/{path}). /list and /get/{kind}/{path} already matched and are unchanged. Updates the handler, openapi spec + dereferenced bundles, the two DraftService callers, the hand-built keepalive page-unload URL (it bypasses the generated client, so it wouldn't be caught by regeneration), and the integration tests. Response status values ("saved"/"conflict") are unchanged, so there is no behavior change. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> --------- Co-authored-by: windmill-internal-app[bot] <windmill-internal-app[bot]@users.noreply.github.com> Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com>	2026-06-15 10:23:16 +02:00
Ruben Fiszel	765f50c474	feat: folder-level label inheritance for scripts, flows and jobs (#9524) ... * feat: folder-level label inheritance for scripts, flows and jobs Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix: use SECURITY DEFINER folder_labels() for RLS-consistent inheritance Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * feat: extend folder label inheritance to apps, resources, variables, schedules Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> --------- Co-authored-by: Claude Fable 5 <noreply@anthropic.com>	2026-06-11 07:49:02 +00:00
Ruben Fiszel	a6a5600833	feat(flow): support worker tag override on AI agent steps (#9513) ... * feat(flow): support worker tag override on AI agent steps * refactor: drop ineffective tag passthrough in nested agent tool path * chore: regenerate openflow-derived system prompt artifacts	2026-06-10 13:54:05 +00:00
hugocasa	fad1a549d9	feat(otel): connect jobs to the inbound distributed trace (#9456) ... * feat(otel): propagate inbound W3C traceparent to job spans Capture the inbound traceparent header at the run endpoints (WebhookArgs::to_args_from_format) into a reserved _wm_traceparent arg key (gated on OTEL_TRACING_ENABLED), riding the args jsonb like _ENTRYPOINT_OVERRIDE. At pickup, create_span_with_name attaches a span link from the job's worker span to the originating distributed trace, so a job triggered by an instrumented service is connected to the caller's trace while keeping its UUID-derived trace id (trace-by-job-id unaffected). The link/parse logic lives in the EE otel modules; this OSS side only captures the header and calls the (no-op outside EE) hook. Companion EE PR required. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * chore: bump ee-repo-ref to inbound-trace-propagation EE branch Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * docs(agents): don't attribute work to specific customers in repo content Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(otel): relocate job + script spans into the inbound trace Builds on the captured _wm_traceparent: the worker job span is re-parented on the inbound caller context, the script subprocess's TRACEPARENT env is the inbound context (so its spans join the caller's trace), and the context is propagated to flow steps so the whole flow relocates. Carried to the worker via a new LogContext.inbound_traceparent field. Non-inbound jobs are unchanged. Adds a relocation integration test. Companion EE PR required. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * chore: bump ee-repo-ref to inbound-trace-propagation relocate commit Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(otel): harden inbound traceparent capture Address review feedback: - strip any caller-supplied _wm_traceparent from args/extra before stashing the header-captured value, so the reserved key is Windmill-controlled only - valid_w3c_traceparent: reject version ff and require lowercase hex, so we don't forward an inbound header that downstream OTel parsers would reject - clarify that the capture helper does not validate the W3C format (done at use) Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * chore: update ee-repo-ref to 2c7964460327fab5e3a27c0f74b8d6f26ab7f79a This commit updates the EE repository reference after PR #604 was merged in windmill-ee-private. Previous ee-repo-ref: 8fc04fb105dc49769205f7174d551a0d134d1bec New ee-repo-ref: 2c7964460327fab5e3a27c0f74b8d6f26ab7f79a Automated by sync-ee-ref workflow. --------- Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Co-authored-by: windmill-internal-app[bot] <windmill-internal-app[bot]@users.noreply.github.com>	2026-06-05 16:50:07 +00:00
hugocasa	f2f0812a04	feat(flows): opt-in to include the stopping step's result in early-stop errors (#9446) ... * feat(flows): early stop can include the stopping step's result in the raised error When a step uses Early Stop with "Raise an error message if stopped", the flow result was entirely replaced with a static error object ({"error": {"name": "EarlyStopError", "message": "..."}}), discarding the stopping step's own output. This made it impossible to stop+fail a flow while preserving the data the step produced (e.g. an API that returns HTTP 200 with a userErrors payload). Add an opt-in `error_include_result` flag on StopAfterIf. When enabled on the raise-error path, the raised payload becomes {"error": {...}, "result": <step result>} instead of dropping the result. Default is false, so existing behavior is unchanged. The option is threaded through the worker's stop-after-if handling (including stop_after_all_iters_if for loops/branchall) and exposed in the flow editor's Early Stop panel. Fixes WIN-2012 Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * test(flows): cover early-stop error_include_result payload shaping Add a regression test asserting that a step using Early Stop with a raised error message and error_include_result=true fails the flow while preserving the step output as {"error": {..}, "result": <step result>}, and that with the flag off the result is the bare {"error": {..}} object. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * refactor(flows): nest early-stop step result inside the error object Embed the stopping step's result under `error.result` rather than as a top-level sibling of `error`. This keeps the flow result shape as `{ "error": { .. } }` — identical to a normal error — so consumers that key off the top-level shape (single `error` key) keep working, while the data is still preserved for those that look inside the error object. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(flows): always include the stopping step's result in early-stop errors Drop the opt-in `error_include_result` gate. Since the step result is nested inside the error object (`error.result`), the top-level result shape stays `{ "error": .. }` — identical to a normal error — so consumers that detect or parse failures by the top-level shape are unaffected. Gating it added schema surface, plumbing, and a UI toggle for no real compatibility benefit. Now, whenever a step early-stops with a raised error message, the flow fails and the raised error embeds the stopping step's own result under `error.result` (aggregated iteration results for loops/branchall). This reverts the `StopAfterIf.error_include_result` field, its threading, the OpenAPI/generated-client surface, and the editor toggle; the "Raise an error message" tooltip now notes that the step result is included. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(flows): gate early-stop result inclusion behind opt-in flag Re-introduce the per-step `error_include_result` flag (default off) instead of always embedding the step result. Although nesting the result under `error.result` keeps the result *shape* backward-compatible, it does not address data exposure: a failed flow's result is propagated to synchronous webhook callers, the flow's failure module, and the workspace/global error handler (commonly a Slack/email/outbound-webhook notifier). Always including the step output would surface previously-redacted intermediate data to all of those sinks for every existing error-stop flow. Gating keeps the existing behavior (bare `{ "error": .. }`) as the default and only embeds `error.result` when the flow author explicitly opts in, matching the original issue's intent. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(flows): omit error_include_result when false; refresh generated prompts - Add `skip_serializing_if = "is_false"` to `StopAfterIf.error_include_result` so serialized flows are byte-identical when the flag is off. Fixes the `flowmodule_serde` round-trip test (cargo_test) and avoids churn on existing flows. - Regenerate `system_prompts/auto-generated/` and `cli/src/guidance/skills.gen.ts` for the new OpenFlow `error_include_result` property. Fixes check-freshness. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * test(flows): cover error_include_result for the loop "stop after all iters" path Add a regression test for the stop_after_all_iters_if branch, where `nresult` already holds the aggregated iteration results — confirming `error.result` carries each iteration's output (distinct from the per-step fallback path). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>	2026-06-04 18:57:05 +00:00
Ruben Fiszel	f0301b1605	feat(flows): preserve step/subflow worker tags under a custom-tagged flow (#9375) ... * feat(flows): preserve step/subflow worker tags under a custom-tagged flow A flow running on a custom worker tag force-propagates that tag to every descendant step, script and nested sub-flow, overriding their own declared tags. This made it impossible to route a specific step or sub-flow to a different worker group. The new opt-in FlowValue.preserve_step_tags lets a step that declares its own non-empty tag run on it; untagged steps still inherit the flow tag. Defaults off to preserve existing behavior. * chore: regenerate system prompts for preserve_step_tags Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * feat(flows): nest preserve_step_tags toggle under flow worker tag setting The toggle only affects routing when the flow has a custom worker tag, so show it as a sub-setting of the Worker Group tag picker, visible only once a tag is set, instead of as a standalone option. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * fix(flows): allow step worker tag picker when preserve_step_tags is enabled When a flow defines a worker tag, the per-step tag picker was replaced by a read-only "Flow's WG" label. With preserve_step_tags enabled the step's own tag is honored, so the picker must remain editable in that case. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * fix(flows): propagate preserve_step_tags to branch and loop bodies payload_from_modules built the synthetic RawFlow for branch/loop bodies with a default FlowValue, dropping preserve_step_tags. Tagged steps inside a branch or loop therefore still inherited the parent flow tag even with the flag enabled. Thread the flag through to the synthetic FlowValue so the behavior is consistent for nested containers. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * fix(flows): clear preserve_step_tags when flow worker tag is removed Avoids the flag lingering as invisible state after the flow tag (and its toggle) are removed. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * fix(flows): repair preserve_step_tags propagation to branch/loop bodies The previous commit added flow.preserve_step_tags at the payload_from_modules call sites but the parameter and FlowValue field were not actually threaded through (a failed edit left the function unchanged), so the crate did not compile. This completes the change: payload_from_modules takes preserve_step_tags and sets it on the synthetic FlowValue for branch/loop bodies. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * fix(flows): complete preserve_step_tags propagation to branch/loop bodies Previous two commits left windmill-worker uncompilable: payload_from_modules received flow.preserve_step_tags at its call sites but the parameter and the synthetic FlowValue field were not actually added. This adds the parameter, sets preserve_step_tags on the synthetic FlowValue, and threads flow.preserve_step_tags through all five call sites. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * fix(flows): clear preserve_step_tags whenever the flow worker tag is removed The flag was only reset when the Worker Group toggle was switched off, not when the tag was cleared directly in the picker (or via the YAML editor), leaving preserve_step_tags=true as invisible state with the advanced badge still reporting it active. Move the cleanup into the reactive block that already tracks the flow tag so every clear path is covered. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>	2026-05-30 12:10:16 +00:00
hugocasa	b0c3b01d31	fix(cli): preserve user drafts on sync push and permissioned-as (#9381) ... CLI deploys (sync push, set-permissioned-as) went through the same create/update endpoints as a UI "deploy from draft", which delete the draft at that path. That silently wiped teammates' in-progress drafts on every push. Add a transient skip_draft_deletion deploy flag (mirroring deployment_message) that the CLI sets; the backend then skips the DELETE FROM draft for scripts, flows, and apps. UI deploys are unchanged. Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>	2026-05-30 10:02:11 +00:00
Ruben Fiszel	bd05bcadde	fix: validate entrypoint override to prevent worker code injection (GHSA-wxjq-w5pj-jqhx) (#9204) ... Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>	2026-05-18 09:01:05 +00:00
Ruben Fiszel	26a41df3e3	chore: use concat! for SCRIPT_COLUMNS to avoid \ continuation whitespace	2026-05-04 08:44:15 +00:00
Ruben Fiszel	1174d7d77f	refactor: replace SELECT * with explicit column lists (#9010) ... * refactor: replace SELECT * with explicit column lists Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * chore: update sqlx offline query cache * chore: update sqlx offline query cache * chore: update sqlx offline query cache with EE support * chore: update sqlx offline query cache, no deletions * chore: update sqlx offline query cache after rebase * fix: correct column names in explicit script query lists - concurrency_limit → concurrent_limit (matches DB column name) - runnable_settings → runnable_settings_handle (matches DB column name) Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * fix: add missing delete_after_secs column to script queries Also add integration test covering all explicit-column export queries. Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * test: add workspace export integration test covering all explicit-column queries Covers tarball_workspace (folder, script, resource, resource_type, variable, schedule, usr, group_) and the mcp_oauth_client SELECT query from windmill-mcp. Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * test: add tarball export integration test covering all explicit-column queries Single test creates one of each entity type and exercises every runtime-checked explicit-column query in tarball_workspace. Uses archive_type=tar to avoid zip feature-gate in CI. Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * docs: fix stale verification step and CI contradiction in update-sqlx skill - Regenerate current_files.txt after EE cache restoration so step 4 reports accurate diff - Scope "Never use SQLX_OFFLINE=true" to local prepare (CI legitimately uses it) Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * docs: remove Co-Authored-By from commit skill template * refactor: extract SCRIPT_COLUMNS const to single source of truth Replaces 5 duplicated 44-column lists with a shared const in windmill-types. Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>	2026-05-04 08:39:45 +00:00
Ruben Fiszel	0c22f52b46	feat: support assigning a worker tag to app inline scripts (#9002) ... * feat: support assigning a worker tag to app/raw-app inline scripts Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * fix: omit empty tag field from inline script raw_code payload Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * style: shrink tag popover width --------- Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>	2026-05-01 17:21:17 +00:00
hugocasa	c95642863e	feat: support restart from steps inside BranchOne, ForLoop, Subflow (#8955) ... * feat: support restart from steps inside BranchOne, ForLoop, Subflow Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * fix: preserve original job kind in nested restart, support expanded subflow steps Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * fix: read selected iteration from graph state for nested ForLoop restart Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * feat: iteration selectors per ForLoop in restart popup, more nested restart tests Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * refactor: extract useNestedRestartState composable Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * test: cover deployed-subflow + FlowDependencies path in nested restart Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * chore: update sqlx prepare cache Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * fix: detect BranchOne/ForLoop ancestors inside expanded subflows for nested restart Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * fix: hide restart button for non-restartable steps (parallel containers, untaken branches) Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * fix: address review feedback on nested restart PR - preview FlowRestartButton: hide nested case (chain UUIDs aren't resolvable in preview path; users can use the run page for nested restart instead) - branchOneAncestorMatchesOriginal: be permissive when status isn't reachable (don't hide the button for BranchOnes nested deeper than top-level) - worker_flow.rs: apply nested_restart_payload swap on the is_simple ForLoop fast path too, so simple iterations don't bypass restart spawn interception - FlowStatusViewer: reset expandedSubflows cache on jobId change; drop $bindable({}) banned pattern for the new prop - API resolver: validate the leaf step exists before returning (fail-fast) - doc fix: branch_or_iteration_n is 0-based, not 1-based - selectedJobStepIsTopLevel reset on early-return in composable - comment iterationCounts collision caveat - new HTTP-level integration tests covering the API endpoint contract: happy path (top-level + nested), unknown step, out-of-range iteration, parallel-loop rejection Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * revert: remove unreachable nested-restart swap on is_simple ForLoop fast path The swap is unreachable in valid flows: `is_simple_modules` requires the body to be a single `script` / `rawscript` / `flowscript` (per `FlowModule::is_simple`), none of which spawn flow-kind children. Any nested-restart chain targeting a leaf inside such an iteration is rejected by the API at leaf validation. Even if a chain reached the worker via `JobPayload::RawFlow.restarted_from`, the resulting `RestartedFlow` would fail to push (script kind isn't a flow kind). Replaced the swap with an explanatory comment so the next reader knows why the symmetry with the non-simple path was deliberately not added. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * fix: handle undefined expandedSubflows + tighten branchOne match check Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>	2026-04-28 20:00:03 +00:00
hugocasa	d6c642b170	feat: add Azure Event Grid triggers (#8888) ... * feat: add Azure Event Grid triggers (EE) Introduces a new enterprise trigger kind `azure` that supports three modes via a single unified trigger type: - basic_push: Azure Event Grid basic — custom topics, system topics (Storage, Resource Manager, Key Vault, etc.), domains (push only) - namespace_push: Event Grid Namespace topics (CloudEvents over HTTP push) - namespace_pull: Event Grid Namespace topics (HTTP pull with lock-token ack/reject for dead-lettering) Auth uses a Service Principal resource (tenant_id, client_id, client_secret, subscription_id). Subscriptions are created in CloudEvents 1.0 schema so the push webhook handler and the pull listener share one payload parser. Backend - New crate `windmill-trigger-azure` (OSS stubs + EE impl symlinked from windmill-ee-private) - Migration `azure_trigger` table with CHECK constraints enforcing mode/columns coherence - `TriggerKind::Azure`, `JobTriggerKind::Azure`, `DeployedObject::AzureTrigger` variants - Push route `/api/azure/w/{workspace}/*path` handles classic Event Grid SubscriptionValidation handshake and CloudEvents 1.0 abuse-protection OPTIONS handshake - Optional inbound JWT validation (audience check only for v1) - Feature flag `azure_trigger` propagated through windmill-api, windmill-store (resource helper), and added to ee_core Frontend - `triggers/azure/` editor with mode toggle (basic/namespace-push/ namespace-pull) and per-mode config (topic ARM id / namespace + topic name / subscription / filters / push auth / pull options) - Registered in icon map, display names, save functions, badge, wrapper, editor, add-trigger menu OpenAPI - `AzureTrigger`, `AzureTriggerData`, `AzureMode`, `AzureSubscriptionMode`, `AzureDeliveryConfig`, `TestAzureConnection` schemas; `/azure_triggers/*` endpoints; client regenerated Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * chore: update ee-repo-ref to eaa7c3a9cb37a9ccc93f10a2535d929365acd2d8 This commit updates the EE repository reference after PR #541 was merged in windmill-ee-private. Previous ee-repo-ref: 9689014e8c12c36c1059fd8fa5758d550b8b8bc9 New ee-repo-ref: eaa7c3a9cb37a9ccc93f10a2535d929365acd2d8 Automated by sync-ee-ref workflow. * feat(azure-trigger): secret-auth push, ARM discovery, capture isolation, CLI + parity Frontend: - Split mode selector into Namespace/Basic + Pull/Push - ARM resource dropdowns (namespaces, Basic topics, namespace topics) populated from the service principal; cascade with stale-selection reset on SP / edition change - Remove stale authenticate toggle + audience input (server-managed push_auth_config has replaced them) - Azure listing page: "Create from template" button; "Also delete Azure subscription" toggle in the delete modal; simplified trigger label falling back to path - AzureCapture.svelte: "Test subscription name" with -wm-capture suffix - CompareWorkspaces.svelte: wire Azure for fork/compare - Drop Trigger-deployed/event-loss warning (capture subscription is isolated with -wm-capture) Backend: - Shared-secret push auth (see EE crate for detail) - JSONB push_auth_config column (renamed from delivery_config), #[serde(skip)] so clients/CLI/exports never see it - Drop redundant enabled column; mode supersedes - Azure capture infra: AzureTriggerConfig + set_azure_trigger_config + azure_payload route + TriggerKind::Azure arm; PT15M queue TTL on capture subscriptions so they bound storage after tab close - Granular ACLs, users offboarding, trash, git-sync deployed-object: all include azure_trigger CLI: - Add azure to TRIGGER_TYPES, pushObj dispatch, getTypeStrFromPath, trigger commands (get/update/create/list/template), sync delete switch + regex; e2e test for `trigger new --kind azure` - system_prompts: SCHEMA_MAPPINGS + schema_names include AzureTrigger; auto-generated/* regenerated Skill: - .claude/skills/adding-a-trigger/ checklist covering every file that needs editing when wiring a new trigger type (learned from this PR) ee-repo-ref bumped to b0e490cbf3724b7b64c6a5b010e3bdf24acd873c. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * fix(azure-trigger): ci — ShareModal Kind + regenerated system_prompts - frontend/src/lib/components/ShareModal.svelte: add 'azure_trigger' to the Kind type so the listing page's "Permissions" action compiles (ts2345 — caught by npm_check on CI, missed by fast-check locally). - system_prompts/auto-generated/: regenerate to drop the stale delivery_config / AzureDeliveryConfig fields from the Azure schema (check-freshness on CI). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * refactor(azure-trigger): use workspace constant_time_eq crate Drop hand-rolled constant-time compare in favour of the workspace constant_time_eq crate (same one used by http_trigger_auth). ee-repo-ref bumped to 9659382d47286e7f7f66d01b6f5dd8d4ed34848b. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * fix(azure-trigger): pass placeholder + disabled via inputProps `TextInput`'s `placeholder` and `disabled` go through its `inputProps` prop — CI's `npm run check` caught the stale top-level passing that `npm run check:fast` missed. Align with the DefaultEmailConfigSection pattern. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * fix(azure-trigger): correct LATEST_GIT_SYNC_SCRIPT_PATH version to 28213 The hub deploy of the azure-aware sync-script is version 28213, not 28214. Backend was pinning a non-existent hub script, which broke the git_sync_e2e suite (every deploy's sync step 404'd). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * fix(azure-trigger): add azure_triggers to token scope selector + skill - windmill-api/src/token.rs: `build_trigger_scope_domains` was missing `("azure_triggers", "Azure Event Grid")`, so the CreateToken UI's scope selector didn't surface azure_triggers:read/write. Backend already had `ScopeDomain::AzureTriggers` wired (scopes.rs), this just exposes it. - .claude/skills/adding-a-trigger/SKILL.md: capture both scope-related files under the hardcoded-arrays section so future triggers don't miss the UI surface. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * docs(adding-a-trigger-skill): clarify token.rs scope effect Not a regression — nothing was working before. Skipping TRIGGER_DOMAINS just means the scope works via API/CLI but has no UI checkbox. * docs(adding-a-trigger-skill): trim token.rs bullet * fix(azure-trigger): regen openapi-deref + swap textarea for TextInput - Run build_openapi.sh to regenerate openapi-deref.{yaml,json} with the 12 azure_triggers paths + schemas. These files are served by the runtime (include_str! in windmill-api/src/lib.rs) to external SDK consumers; without this regen the new endpoints wouldn't be advertised. - Replace the raw <textarea> for event type filters with the design-system TextInput in textarea mode (frontend/CLAUDE.md bans raw HTML elements). Addresses cubic + claude PR review items. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com> Co-authored-by: windmill-internal-app[bot] <windmill-internal-app[bot]@users.noreply.github.com>	2026-04-23 16:30:18 +00:00
centdix	9a60ff2e77	feat: add ai agent conversation output control (#8915) ... * feat: add ai agent chat output flag Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * fix: suppress ai agent tool chat messages Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * refactor: rename ai agent conversation output flag Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * feat: expose ai agent conversation output toggle Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * fix: gate ai agent chat tab by chat mode Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * chore: regenerate system prompts Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * fix: address ai agent chat review feedback Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>	2026-04-23 14:13:20 +00:00
hugocasa	4f998cc231	feat: add GitHub as a native trigger service (#8856) ... * feat: add GitHub as a native trigger service Add GitHub webhooks as a native trigger, allowing users to trigger scripts/flows from repository events (push, PR, issues, etc.) via OAuth-based webhook management. Backend: - DB migration adding 'github' to native_trigger_service, TRIGGER_KIND, and job_trigger_kind enums - Full External trait implementation: create/update/delete/get webhooks, per-trigger sync verification, webhook payload preparation - Paginated repos endpoint (up to 1000 repos) - OAuth flow with admin:repo_hook and read:user scopes Frontend: - GitHub trigger form with repo picker and MultiSelect event selector - Workspace integration settings with setup instructions - Trigger badge, editor, and wrapper integration - GithubIcon updated to support size/class props (matching other icons) - Hub template reference for starter scripts Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: show GitHub in sidebar when triggers exist Add github_used to the getUsedTriggers endpoint so the sidebar picks up GitHub as an active trigger kind. Also document this step in the native- trigger skill so future services don't miss it. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * feat: on-demand GitHub repo search instead of bulk fetch Replace the upfront pagination through all repos with a debounced search flow: load 30 most-recently-updated repos by default, then query GitHub's /search/repositories API (scoped to the authenticated user via user:@me and restricted to name matches via in:name) as the user types. Frontend uses runed's Debounced + resource to wire the Select's filterText to the backend query with 300ms debouncing. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: request `repo` OAuth scope to list private GitHub repos `admin:repo_hook` grants webhook management but not repo listing — so /user/repos and /search/repositories returned only public repos. Switch to `repo` (full repo scope, which is a superset and also covers webhook management). Users who already connected GitHub need to disconnect and reconnect to pick up the broader scope. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * revert: fetch all GitHub repos upfront instead of searching on demand Revert the debounced search flow — paginate through /user/repos (up to 1000) on form open. Simpler UX: repos are all there from the start, the Select's built-in client-side filter handles finding one. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: typed 404 detection + add GitHub flow template reference Replace fragile e.to_string().contains("404") matching with a proper http_error_status helper that downcasts through anyhow to the typed HttpRequestError and reads the StatusCode. Also wire the hub flow template (id 80) into NATIVE_TRIGGER_SERVICES. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: update GitHub script template hub ID to 28202 Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: align GitHub trigger with Nextcloud/Google patterns Addresses review feedback from Claude and cubic. Backend: - `delete()` now only swallows NotFound (DB missing row) and 404 (API webhook already deleted); non-404/DB errors propagate so callers know cleanup failed. Matches Nextcloud's delete pattern exactly. - `get_owner_repo_from_db` returns `Result<Option<(String, String)>>` instead of an error on missing row (matches Google's delete flow). Frontend: - `loading: boolean` (required) + `$bindable()` with no default — matches Nextcloud, satisfies CLAUDE.md banned-pattern rule. - Wrap `loadRepos()` in `$effect` reacting to `$workspaceStore` so repos load once the store is available and refresh on workspace switch. - Replace raw `fetch('/api/.../native_triggers/github/repos')` with the generated `NativeTriggerService.listGithubRepos(...)` typed client. Adds `/repos` route + `GithubRepoEntry` schema to openapi.yaml. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>	2026-04-17 17:01:55 +00:00
Ruben Fiszel	362ae248fe	fix: per-branch concurrency key for promotion-mode git sync (#8844) ... * [ee] fix: per-branch concurrency key for promotion-mode git sync jobs Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * chore: bump ee-repo-ref for per-branch concurrency key Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * chore: update ee-repo-ref to 7bc0fdb8647268c7afa67b4b0bed69c897eaf92a This commit updates the EE repository reference after PR #537 was merged in windmill-ee-private. Previous ee-repo-ref: b933874649a63c5266a33360a95e3c163acc6b5f New ee-repo-ref: 7bc0fdb8647268c7afa67b4b0bed69c897eaf92a Automated by sync-ee-ref workflow. --------- Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com> Co-authored-by: windmill-internal-app[bot] <windmill-internal-app[bot]@users.noreply.github.com>	2026-04-15 20:50:18 +00:00
Ruben Fiszel	c57c769dea	feat: add CI test scripts with auto-trigger on deploy (#8736) ... * feat: add CI test scripts with auto-trigger on deploy Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: fix annotation parser early return and handle renames correctly Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: move CI test results to top of script/flow detail pages Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: improve CI test results spacing, icon, and remove pass label Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * feat: support one-line annotation and use script/path format Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * feat: move CI test trigger logic to EE Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * feat: move CI badge next to New badge and add deduplicated CI summary Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * feat: add CI test e2e tests and fix nullable column annotations Add integration tests for CI test annotation parsing (creates/removes ci_test_reference rows) and the CI test results API (single + batch endpoints). Add backend test for auto-trigger on deploy (private+python). Fix sqlx LEFT JOIN LATERAL nullable column annotations in get_ci_test_results and get_ci_test_results_batch queries — sqlx cannot infer nullability from LATERAL subqueries, causing runtime decode errors when no matching job exists. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix build/sqlx * fix * feat: CI test improvements and templates - Fix windmill-dep-map/private feature propagation in worker, api-scripts, and api-flows Cargo.toml so CI test triggers actually fire in EE mode - Clone ci_test_reference rows during workspace fork - Add polling to CiTestResults component (refetch every 3s while running) - Add running state and auto-refresh to ForkWorkspaceBanner CI summary - Add yellow "CI test" badge on script list rows and detail page - Fix Library badge border color (remove indigo border override) - Add CI Test TypeScript and CI Test Python templates in ScriptBuilder - Update sqlx offline cache - Add debug tracing for CI test trigger in worker_lockfiles Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: add missing children prop to WorkspaceDeployLayout Fixes svelte-fast-check type error when passing named snippets as children content inside the component tag. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: address PR review feedback - Remove empty wrapper divs around CiTestResults, move mb-4 into component - Add batch endpoint size cap (max 200 items) - Add ON DELETE CASCADE to ci_test_reference workspace FK (new migration) - Downgrade CI test trigger logs from info to debug - Fix false-positive polling: only treat status='running' as running, not null status (CiTestResults, CompareWorkspaces, ForkWorkspaceBanner) - Fix test numbering in integration tests Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * chore: update ee-repo-ref to latest EE commit Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * chore: update ee-repo-ref to d9d68c2406df0b59f413ea0b2cb24780a9817d04 This commit updates the EE repository reference after PR #516 was merged in windmill-ee-private. Previous ee-repo-ref: d7ccd9b86da99ec056a0e8708e3637d64290387a New ee-repo-ref: d9d68c2406df0b59f413ea0b2cb24780a9817d04 Automated by sync-ee-ref workflow. * fix: treat queued jobs (job_id set, null status) as running Jobs that have been pushed but not yet picked up by a worker have a job_id but null status. Treat these as 'running' to avoid showing misleading 'pass' badges or '0 passing'. Tests that were never triggered (no job_id, null status) remain neutral/hidden. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com> Co-authored-by: hugocasa <hugo@casademont.ch> Co-authored-by: windmill-internal-app[bot] <windmill-internal-app[bot]@users.noreply.github.com>	2026-04-09 17:21:36 +00:00
Ruben Fiszel	2d18a68099	feat: add scheduled job deletion with configurable retention period (#8753) ... * feat: add scheduled job deletion with configurable retention period Extends delete_after_use with delete_after_secs to enable configurable retention periods for job args/result/logs. At completion, jobs can be scheduled for future deletion via a new job_delete_schedule table, processed by a monitor task. Supports per-script, per-flow, and per-flow-step configuration. Backward compatible. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * feat: add integration tests, revert query! macros, fix review issues - Add integration tests for resolve_delete_after_secs, schedule_job_deletion, flow-level and module-level delete_after_secs, backward compat - Revert sqlx::query() back to sqlx::query!() macros for compile-time safety - Regenerate sqlx offline cache - Fix FlowModule/NewScript/FlowValue constructions in all test files - Fix autoscaling_ee.rs for updated script_path_to_payload return type Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * chore: update ee-repo-ref.txt for autoscaling_ee fix Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: gate cleanup_scheduled_job_deletions behind enterprise feature Prevents dead_code warning (which CI treats as error via -D warnings) when compiling without enterprise feature. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * chore: regenerate sqlx cache after merge with main Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: address review feedback on scheduled deletion - Monitor: roll back transaction on any cleanup error so schedule rows survive for retry on next cycle (instead of best-effort then discard) - Migration: add FK with ON DELETE CASCADE to job_delete_schedule.job_id to prevent orphan rows when jobs are deleted through other means - Simplify bool-to-Option conversion with .then_some(true) Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * refactor: stop setting delete_after_use alongside delete_after_secs No mixed-version deployment scenario exists, so delete_after_secs alone is sufficient. The backend's resolve_delete_after_secs handles (None, Some(secs)) correctly without needing delete_after_use set. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * refactor: remove delete_after_use from public API surface Remove delete_after_use from OpenAPI spec, API client, runtime client, and workspace export. Only delete_after_secs is exposed going forward. The field remains in Rust backend types with #[serde(skip_serializing)] for backward-compatible deserialization of existing scripts/flows that were saved with delete_after_use: true. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * chore: update ee-repo-ref to 1d4b7a31fc115d6aba8640f7cd3fd5a01abe6806 This commit updates the EE repository reference after PR #519 was merged in windmill-ee-private. Previous ee-repo-ref: 9eba09a13b778caafc6ae65098b90e53c91984d3 New ee-repo-ref: 1d4b7a31fc115d6aba8640f7cd3fd5a01abe6806 Automated by sync-ee-ref workflow. * fix: regenerate system prompts, remove unused import - Regenerate auto-generated system prompts after openflow schema change - Remove unused serde_json::json import in test file (CI -D warnings) Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: insert dummy v2_job row in schedule tests for FK constraint The job_delete_schedule table has a FK to v2_job, so tests need a real v2_job row before inserting into the schedule table. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * chore: trigger CI re-run * fix: remove heavy flow integration tests to avoid CI worker contention The flow integration tests spawn workers that compete for CPU with the existing relock_skip tests under --test-threads=10, causing consistent 60s timeouts in CI. Keep only the lightweight unit tests and DB integration tests. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: restore correct ee-repo-ref for our branch The ref was overwritten to main's EE ref during a rebase. Restore to our branch's EE commit that includes the autoscaling tuple fix. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * chore: retrigger CI on fresh runner * fix: remove FK constraint from job_delete_schedule to unblock CI The FK with ON DELETE CASCADE to v2_job may have caused performance overhead during test DB setup (each sqlx::test creates a fresh DB with all migrations). Remove the FK — orphan schedule rows are harmlessly cleaned by the monitor. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * ee-ref --------- Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com> Co-authored-by: windmill-internal-app[bot] <windmill-internal-app[bot]@users.noreply.github.com>	2026-04-08 04:15:28 +00:00
Yoaquim Cintrón	7bf6ac2b69	fix: enrich OTEL spans with job_kind, trigger_kind, trigger, created_by, and script_hash (#8718) ... Add five new attributes to the `job` and `job_postprocessing` tracing spans so that OTEL-consuming backends (Sentry, Honeycomb, Datadog, etc.) can filter and group telemetry by how a job was triggered and what type it is. New span attributes: - `job_kind` — Script, Flow, AppScript, AIAgent, Preview, etc. - `created_by` — the user or system identity that queued the job - `trigger_kind` — schedule, webhook, kafka, http, sqs, etc. - `trigger` — the schedule/trigger path (when applicable) - `runnable_id` — the id of the runnable that ran Also adds `JobKind::as_str()` for a consistent lowercase string representation, following the same pattern as `ScriptLang::as_str()`. Existing attributes (job_id, workspace_id, script_path, language, tag, flow_step_id, parent_job, root_job) are unchanged. Note: the EE `full_job` span in `otel_ee.rs` and the log records emitted by `job_logger_ee.rs` would also benefit from these attributes. This PR covers only the public-repo spans; a follow-up EE change would propagate the same fields to logs and the full_job span.	2026-04-05 14:11:22 +00:00
Ruben Fiszel	c4c9ef5fd7	feat: add optional labels to scripts, flows, apps, schedules, triggers (#8609) ... * feat: add optional labels to scripts, flows, apps, raw apps, schedules, and triggers Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: update sqlx cache, make labels optional in openapi, regenerate system prompts Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * feat: add minimal labels input UI to script, flow, and schedule editors Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: reduce gap between summary and labels input Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * feat: add labels to script/flow detail pages and summary/path popover Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: move labels inside SummaryPathDisplay trigger for clickable area, reduce gap Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: display labels inline to the right of summary, not below Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: increase gap between summary and labels Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * feat: add labels to resources/variables, make labels nullable, add home page label filter badges Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * feat: add labels to workspace export/import, resources, variables + test coverage Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: make migration idempotent, regenerate sqlx cache after merge Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: pass labels in script create and flow create/update API calls Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * feat: add labels input UI to resource and variable editors Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: remove negative margin from LabelsInput to prevent overlap Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: add top and left margin to LabelsInput for better spacing Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: reduce left margin on LabelsInput Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: widen label input to w-32 Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: use inline-flex so LabelsInput doesn't stretch full width Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: remove flex-wrap so label input stays on same line as badges Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * feat: add label filter presets to resources, variables, and schedules search Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: use max-w-32 on label input to prevent stretching Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: pull labels closer to summary with negative top margin Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: increase negative margin to pull labels even closer to summary Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: pass labels in schedule create/update API calls Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: use COALESCE to preserve existing labels when not provided in schedule/flow update Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: add labels to CreateResource, EditResource, CreateVariable, EditVariable in OpenAPI spec Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * feat: display label badges on resource and variable list pages Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * feat: display label badges on schedule and all trigger list pages Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * feat: add folder and label presets to schedules search filter Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: apply user_folders_only filter on all workspaces including admins Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * feat: add label presets to resources and variables search filters Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: derive folder presets from loaded items, not all workspace folders Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: add label query parameter to resource and variable list endpoints in OpenAPI Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * feat: display label filter badges inline with folder filters on home page Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * Revert "feat: display label filter badges inline with folder filters on home page" This reverts commit 6767a50aa6. * feat: support comma-separated label filters (allowMultiple) in all list endpoints Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: append label presets with comma for allowMultiple filters instead of duplicating key Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: hide label presets that are already in the comma-separated filter value Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: replace unsafe manual SQL ARRAY construction with parameterized queries, add labels to ScriptWDraft Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: complete down migration, add labels to Resource/Variable OpenAPI schemas, remove type cast, add label length validation Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: add labels field to Schedule test fixture Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: add labels field to Rust client struct constructions Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: regenerate sqlx cache with --all-features for EE builds Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * chore: regenerate sqlx cache and package-lock after merge with main Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: squash two migrations into one, use IF NOT EXISTS for idempotency Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: track label changes in SummaryPathDisplay to enable save button Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: use JSON string comparison for label dirty tracking in popover Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: navigate to script by path after save from popover to load new version Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: update initialLabels after save so subsequent label changes enable save again Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: use onchange callback for label dirty tracking instead of derived comparison Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: reload script by path after label save to fetch new version Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * feat: propagate script/flow labels to jobs at push time Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * feat: show script/flow labels on runs page, merge with wm_labels for completed jobs Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: change job labels type from JSONB to text[], show labels on job detail page, fix type mismatch Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: add labels to QueuedJob struct, fix get_job queries to return v2_job.labels Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: replace +Label text with icon only Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * feat: add tag icon before labels on job detail page Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: move tag icon inside badge on job detail page Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: use blue badge with tag icon in RunBadges, remove duplicate labels from JobDetailHeader Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: set icon position to left so tag icon renders in badge Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: render Tag icon inline in badge children instead of via icon prop Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: retry icon prop with small badge and position left Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * feat: add hover tooltip showing "Label: X" on job label badges Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * feat: include v2_job.labels in runs page label filter and broad search Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * chore: regenerate sqlx cache and system prompts after merge with main Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: add labels to EE JobPayload constructions, regenerate sqlx cache with --all-features Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: regenerate sqlx cache CE-only (without EE symlinks that cause conflicts) Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: update remaining wm_labels JSONB queries to use text[] merge expression Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: simplify job labels to just read v2_job.labels (wm_labels already merged at completion) Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: consistent label badge spacing with gap-0.5 wrapper and px-0.5 on badges Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: add labels: None to test utils JobPayload construction Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: add labels to all test fixture JobPayload/NewFlow/EditApp constructions, regenerate sqlx cache Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: fix vertical content shift by fixing container and input height to h-5 Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: npm_check errors - unused imports, combinedItems order, flow.labels type, badge px-1 padding Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: remove unused FolderService imports, fix label badge alignment in RunBadges Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: restore deleted service imports in variables page, remove empty loadFolders Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * chore: trigger CI with updated ee-repo-ref * chore: update ee-repo-ref to merged EE companion PR Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * chore: trigger fresh CI run for updated ee-repo-ref * fix: match label badge size with other badges in RunBadges using {large} prop Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: remove icon from RunBadges label badge to fix vertical alignment Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: shorten "Job kind" to "Kind" in run badges Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: add small inline tag icon (10px, -mt-px) to label badge without disrupting height Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: add "Label: X" hover tooltip to all label badges, show hidden labels on +N hover Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * feat: add tag icon and "Label: X" tooltip to home page label filter badges Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: show LabelsInput even when path is hidden in ResourceEditor Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * feat: add labels input to new resource creation drawer (AppConnectInner) Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * iterate * fix: add LabelsInput to all resource creation steps in AppConnectInner Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: reduce LabelsInput top margin from -mt-3 to -mt-1 Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: increase negative margin to -mt-2 for tighter spacing Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: split the difference with -mt-1.5 Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: adjust to -mt-1 for label spacing Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: per-site label spacing via class prop instead of global negative margin Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * feat: make label badges clickable to toggle label filter on resources, variables, schedules Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: use proper array indexOf for label filter toggle, set undefined correctly on removal Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: use delete instead of undefined to properly clear label filter Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * feat: add /labels/list endpoint and autocomplete dropdown to LabelsInput Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: use inline preventDefault for Svelte 5 event handling Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * feat: add "Create new" option in label autocomplete, regenerate sqlx cache with update_sqlx.sh Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * feat: add GIN indexes on labels column for all 16 tables Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: remove CONCURRENTLY from GIN index creation in migration Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * test: add comprehensive label coverage for pull, edit, removal across all item types Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: simplify job label filters to only use v2_job.labels, remove wm_labels back-compat Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * test: add integration tests for job label propagation, display, and filtering Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: address PR review findings — missing labels in fetch_script_for_update, app rename, escape key bug - Add `labels` to SELECT in `fetch_script_for_update` to prevent lost labels on script clone - Pass `labels` in app branch of `moveRenameManager.ts` so app renames preserve labels - Clear `inputValue` before `adding = false` in LabelsInput escape handler to prevent accidental label add via onblur - Fix `test_job_label_filter` to complete jobs via SQL (label filtering only works on completed jobs) - Add `test_wm_labels_from_result_merged_with_static_labels` integration test using Bun Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>	2026-04-03 17:39:32 +00:00
Ruben Fiszel	a46aa641f9	feat: add R language support (#8263) ... * feat: add R language support Add R as a new supported scripting language in Windmill, following the same pattern used for Ruby. Includes: - Backend: ScriptLang::Rlang enum variant, DB migration, tree-sitter-r parser crate with tests, WASM parser binding, R executor with NSJail sandboxing, job dispatch and signature parsing - Frontend: language picker, R icon, syntax highlighting, editor bar insertions (Sys.getenv, get_variable, get_resource), schema inference, init code template, BETA badge - CLI: .r extension mapping, sync support, bootstrap template R scripts use `main <- function(...)` syntax, jsonlite for JSON serialization, and system curl for the Windmill client helper. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * feat: add R package resolution and installation Parse library()/require() calls from R scripts to extract dependencies. Resolve versions from CRAN, cache lockfiles in pip_resolution_cache, and install packages to a shared R library cache. The run step sets R_LIBS_USER so installed packages are available to the script. - Parser: parse_r_requirements() extracts package names from AST - Executor: resolve() generates lockfile, install() installs from CRAN - Worker lockfiles: wire up R resolve for dependency jobs Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * feat: add nsjail sandboxing for R resolve and install phases Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * fix: fix R get_variable/get_resource and add sandbox annotation + e2e tests Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * fix: fix R arg inference with JS fallback parser and get_variable/get_resource Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * fix flake * nsjail * nits * fix: R install improvements - suppress verbose output, flat lockfile logging, Dockerfile R support, rlimits - Suppress renv verbose output during resolve and install (controlled by #verbose annotation) - Filter renv from install list (already loaded, causes noisy restart message) - Log compact "resolved N packages" instead of full renv.lock JSON - Add R (r-base, r-cran-renv) to DockerfileFull and DockerfileFullEe - Use disable_rl for nsjail install config (R compiles from source) - Reduce default concurrency from 20 to 5 - Add rlang to openflow.openapi.yaml - Fix MainArgSignature (no_main_func -> auto_kind) after main merge Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * final * fix: remove accidental R install from multiplayer Dockerfile Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * fix: remove R from Windows build and DockerfileExtra Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * fix: rename R migration to avoid timestamp collision with trigger_filter_logic Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * all * fix: R install improvements - suppress verbose output, flat lockfile logging, Dockerfile R support, rlimits Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * fix: add clear error when Rscript binary is missing Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * fix: fix type errors in R fallback parser, use format! in wrap(), add R system prompts Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com> Co-authored-by: pyranota <pyra@duck.com>	2026-04-01 06:11:37 +00:00
Ruben Fiszel	da8886be85	feat: add configurable preview job tag override in default tags settings (#8649) ... * feat: add configurable preview job tag override in default tags settings Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: skip re-tagging for FlowPreview jobs when preview override is active Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>	2026-03-31 18:59:23 +00:00
Ruben Fiszel	f40cdaf434	fix(cli): app push crash, lint path, push --message, run validation, history timestamps (#8585) ... * fix(cli): app push crash, lint entry point, push --message, run arg validation, history timestamps Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix(cli): update sqlx cache and fix second history query missing created_at Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * chore(cli): regenerate system prompts after new CLI options Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>	2026-03-28 14:33:49 +00:00
Ruben Fiszel	71549c3db0	fix: resolve parent_hash race condition in sync push with auto_parent (#8545) ... * fix: resolve parent_hash race condition in sync push with auto_parent During concurrent sync push operations (parallel CLI groups or separate CI pipelines), multiple requests could read the same remote script hash and both try to create a new version with the same parent_hash, causing "the lineage must be linear" errors. Adds an opt-in `auto_parent` field to the create_script API. When set, the backend resolves the parent_hash to the current head script at that path within the transaction, atomically. This eliminates the client-side race window where the parent could change between read and write. The CLI now sends `auto_parent: true` when updating existing scripts, so sync push is resilient to concurrent deployments. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: add missing auto_parent field in clone_script NewScript initializer Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * fix: add advisory lock to serialize concurrent auto_parent script creates Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * sqlx * fix: add sqlx anchor for CE-only user count query Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>	2026-03-26 18:14:10 +00:00
Ruben Fiszel	167084a0eb	feat: show groups and notes in flow status viewer (#8535) ... Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>	2026-03-26 08:18:01 +00:00
Diego Imbert	5d1c54d9b3	feat: Debounce node (#8324) ... * Debounce node works * sqlx prepare * sqlx prepare * fix: address PR review issues for flow node debouncing - Add sibling check in parent-walking loop to avoid killing branchall siblings - Remove stale .sqlx cache files from earlier iterations - Remove single-variant FlowNodeDebounceResult enum, use Result<()> - Parse flow value once in version guard, recurse into nested modules - Fix Svelte reactivity when switching selected flow modules - Fix Tab indentation in FlowModuleComponent - Use integer types in OpenAPI spec for debounce fields Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * ee repo ref * nit sqlx * add Debouncing: None * ee repo ref * ee repo * sqlx update * fix: reject node-level debouncing inside branches (branchall/branchone) Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * Revert "fix: reject node-level debouncing inside branches (branchall/branchone)" This reverts commit fa4820dde2. * ee repo * sqlx prepare * sqlx prepare * feat: add MIN_VERSION_SUPPORTS_NODE_DEBOUNCING (1.658.0) version guard Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * docs: mark node-level debouncing as EE only in openflow schema Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * fix: guard node debouncing against parallel steps (len > 1) Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * generate system prompts * system prompts * chore: update ee-repo-ref to c04f3851c03758662e4936ff4b6e71bc56dbae7e This commit updates the EE repository reference after PR #451 was merged in windmill-ee-private. Previous ee-repo-ref: d140bb8944dfe3efb23cf8c12f556eacf30e2f87 New ee-repo-ref: c04f3851c03758662e4936ff4b6e71bc56dbae7e Automated by sync-ee-ref workflow. --------- Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com> Co-authored-by: windmill-internal-app[bot] <windmill-internal-app[bot]@users.noreply.github.com>	2026-03-24 09:47:41 +00:00
Pyra	9643006f1e	feat(cli): better stale scripts detection #3 (#8480) ... * fix Signed-off-by: pyranota <pyra@duck.com> * reduce tests Signed-off-by: pyranota <pyra@duck.com> * update Signed-off-by: pyranota <pyra@duck.com> * fix Signed-off-by: pyranota <pyra@duck.com> * update Signed-off-by: pyranota <pyra@duck.com> * WIP: stash changes after merge with origin/main * Delete backend/parsers/windmill-parser-wasm/Cargo.lock * reset cargo.toml * feat(cli): integrate dependency tree into generate-metadata command - Add isDirectlyStale field to DependencyNode for staleness tracking - Update addScript to accept itemType, folder, isRawApp, isDirectlyStale - Update propagateStaleness to use isDirectlyStale field instead of parameter - Handlers now determine staleness and pass it to tree.addScript - generate-metadata calls propagateStaleness() and populates staleItems from tree - Pass legacyBehaviour=false and tree to handlers during generation phase 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * fix(cli): store originalPath in tree for correct handler invocation Scripts need the path with extension to be passed to the handler. Added originalPath field to DependencyNode to track this. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * fix parsers Signed-off-by: pyranota <pyra@duck.com> * rever sqlx removal * update sqlx * feat: make py-imports parser WASM-compatible and add as separate WASM package Gate heavy deps (sqlx, windmill-common, async-recursion, toml, pep440_rs, tracing) behind cfg(not(wasm32)). Make parse_code_for_imports, parse_relative_imports, NImport, and ImportPin public. Remove duplicate import_parser from parser-py (reset to origin/main). Add py-imports-parser feature to windmill-parser-wasm and py-imports target to build.nu. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * safer return * update * fix: CLI metadata fixes - folder filter, staleness detection, WASM py-imports setup - Fix lazy_static cfg gating for WASM compatibility (split into separate blocks) - Fix folder argument filter to match specific file paths (not just directories) - Fix staleness detection to use checkHash with conf (includes module hashes) - Convert relative_imports_skip tests from Deno to bun APIs - Add windmill-parser-wasm-py-imports to CLI and build-npm dependencies - Relax module stale test to not require per-module change detail in output Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * fix: restore temp_script_refs parameter in parse_python_imports Re-adds the temp_script_refs parameter that was lost when resetting py-imports crate to origin/main. This enables resolving relative imports from not-yet-deployed scripts during CLI lock generation. * fixes * extend testsuit * update ee repo ref * fix: diff endpoint bytea cast, upload only mismatched scripts - Add POST /scripts/raw_temp/diff endpoint to batch-compare local content hashes against deployed versions using Postgres sha256() - Use convert_to(content, 'UTF8') instead of content::bytea to avoid failure on scripts containing backslash sequences (e.g. \n) - CLI now diffs all scripts against deployed, uploads only mismatched ones - propagateStaleness no longer deletes non-stale nodes (needed for diff) - Suppress verbose log.info messages during metadata generation - Add E2E tests for locally modified and unpushed helper scripts Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * rework * sqlx * fixes * add index * expand tests * fix flows * archive script before executing * disable tests for ci * skip Python-dependent E2E tests on CI Tests requiring the python backend feature are skipped when CI_MINIMAL_FEATURES=true since CI builds with zip-only features. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * fix: make flow fixture lock optional and reset nonDottedPaths after tests Flow fixtures no longer emit an empty lock file by default. The lockContent parameter controls whether a lock: "!inline ..." line appears in flow.yaml. This prevents flows from appearing "up-to-date" when they should be processed by generate-metadata. Also adds afterAll to reset setNonDottedPaths(false) so global state doesn't leak between test files when run together. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * debug: add error logging in withTestBackend to diagnose CI failures Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * debug: add --bail 1 to CI test runner to show full error on first failure Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * debug: include CLI stdout/stderr in assertion message for workspace deps test Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * fix: set WMDEBUG_FORCE_V0_WORKSPACE_DEPENDENCIES in test backend The workspace deps feature requires workers to report their version, but in test/CI there are no separate workers (standalone mode). The version check fails because workers haven't had time to ping yet. Setting this env var bypasses the version check. Also reverts --bail 1 from CI workflow now that the root cause is fixed. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * debug: add --bail 1 to Windows CI and assertion messages for Windows failure diagnosis Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * fix: replace TEMP_SCRIPT_REFS_PLACEHOLDER in bun builder tests The loader.bun.js now includes a TEMP_SCRIPT_REFS_PLACEHOLDER that must be replaced before execution. The builder tests were missing this replacement, causing all 6 bun_builder_tests to fail. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * fix: use cdirFwd in Windows loader filterLoad regex Raw cdir (with backslashes) interpolated into RegExp causes \r to become carriage return and \w to become word-char, so filterLoad never matches main.ts. This prevents replaceRelativeImports from running, leaving bare relative imports like "./script_b" in the bundled output, which scanImports then misparses as package ".". Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * fix: Windows filterLoad regex + graceful fallback for old backends - Fix filterLoad in loader.bun.windows.js to match both native backslash and forward-slash paths from Bun's resolver by escaping cdir for regex - Wrap uploadScripts in try/catch so generate-metadata degrades gracefully when the backend lacks /raw_temp endpoints (locks use deployed versions) - Add TODO for missing TEMP_SCRIPT_REFS support in Windows loader Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * debug: add loader/builder debug logging for Windows CI diagnosis Temporary console.log statements to understand: - What path Bun passes to onLoad for main.ts - Whether filterLoad regex matches - Whether replaceRelativeImports fires - What the bundled output contains - What imports scanImports extracts Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * chore: trigger CI for cli path Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * chore: trigger CI via workflow file change Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * Add TEMP_SCRIPT_REFS to Windows loader, use .ts extensions in test imports - Add TEMP_SCRIPT_REFS_PLACEHOLDER support to loader.bun.windows.js (mirrors loader.bun.js) so CLI lock generation can resolve imports from locally-modified scripts on Windows - Use .ts extensions in all test relative imports to work around the Windows filterLoad regex bug (replaceRelativeImports doesn't fire on Windows, so extensionless imports fail) - Remove unused uploadSucceeded variable Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * Remove debug logging from loader_builder.bun.js Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * Remove windmill-parser-wasm-py-imports from frontend package.json This dependency is only needed by the CLI, not the frontend. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * debug: add temp_script_refs logging for Windows CI investigation Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * ci: remove --bail 1 from Windows CLI tests Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * fix: normalize backslashes in folder filter treePath lookup (Windows) On Windows, item.path (originalPath) uses backslashes but tree keys use forward slashes. The isRelevant filter's touchesFolder call passed the unnormalized path to traverseTransitive, which couldn't find the node. This caused cross-folder importers to be excluded from generate-metadata when a folder argument was specified. Also removes debug logging from previous commit. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * Update cli-tests.yml * fix: normalize backslashes in strict-folder-boundaries warning message (Windows) Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * chore: update ee-repo-ref to fe8f0d1d7448464c98474d994e6492c0a45e8e38 This commit updates the EE repository reference after PR #467 was merged in windmill-ee-private. Previous ee-repo-ref: 03e6eaf950776c96b9581848a583af9ad735be60 New ee-repo-ref: fe8f0d1d7448464c98474d994e6492c0a45e8e38 Automated by sync-ee-ref workflow. * revert cli-tests.yml --------- Signed-off-by: pyranota <pyra@duck.com> Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com> Co-authored-by: windmill-internal-app[bot] <windmill-internal-app[bot]@users.noreply.github.com>	2026-03-23 18:20:19 +00:00
hugocasa	efb4a27d51	fix: replace email with permissioned_as for triggers/schedules (#8439) ... * refactor: replace email with permissioned_as for triggers/schedules Add a new `permissioned_as` column (format: `u/{username}`, `g/{group}`, or raw email) to all trigger tables and schedule. This value is used directly for job permission checks, removing the need for email lookups when creating/updating triggers. - Migration: add permissioned_as to all 9 trigger tables + schedule, drop email from trigger tables (schedule keeps it for backwards compat) - Backend: resolve_email() (async, DB) -> resolve_permissioned_as() (sync) - Email cache: get_email_from_permissioned_as() with quick_cache for places that still need email (fetch_api_authed, schedule backwards compat) - Frontend: rename email/preserve_email -> permissioned_as/preserve_permissioned_as in deploy data and OpenAPI schemas - Tests updated for new field names and u/{username} format Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix sqlx/build * update ee ref * refactor: simplify resolve_edited_by to always use authed username Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix compile + migration * update ref * test: add trigger trait method tests for permissioned_as queries Add tests that call TriggerCrud and Listener trait methods directly to verify dynamic SQL correctly references the permissioned_as column. Covers get_trigger_by_path, list_triggers, set_trigger_mode, and fetch_enabled_unlistened_triggers for all trigger types. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * update sqlx * fix: use permissioned_as directly for schedules and fix audit RLS for groups - Schedule: permissioned_as only set on create, not on edit/set_enabled - Schedule: stop reading email column, use get_email_from_permissioned_as - Triggers: use fetch_api_authed_from_permissioned_as instead of edited_by - Triggers: rename listener fields for clarity (username -> edited_by) - Fix audit author username for group permissioned_as (g/test -> group-test) to match session.user, preventing RLS policy violations on audit_partitioned - OpenAPI: remove permissioned_as/preserve_permissioned_as from EditSchedule - Add backwards-compat comments for schedule email writes Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * chore: regenerate system prompts for permissioned_as field Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix build * refactor: generalize onBehalfOf naming, add permissioned_as to EditSchedule - Frontend: rename onBehalfOfPermissionedAs -> onBehalfOf with comments explaining it carries emails for flows/scripts and permissioned_as for triggers/schedules - Frontend: rename getOnBehalfOfEmail -> getOnBehalfOf, getOnBehalfOfPermissionedAsForDeploy -> getOnBehalfOfForDeploy, customOnBehalfOfEmails -> customOnBehalfOf - Backend: add optional permissioned_as/preserve_permissioned_as to EditSchedule with COALESCE (only updates when provided) - Backend: add on_behalf_of audit log for schedule edit - Backend: remove unused resolve_on_behalf_of_permissioned_as - Tests: remove email assertions from schedule update test (email is just backwards compat, only permissioned_as matters) Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: preserve email column when permissioned_as is preserved on schedule edit Derive email from the preserved permissioned_as via cache lookup instead of always writing authed.email. This keeps the email column consistent with the old behavior for backwards compat with old workers. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: update deploy UI labels from "edited by" to "run as" for triggers Triggers now use permissioned_as (not edited_by) for permissions, so update the deploy UI wording to reflect this. Also update wm_deployers group description to mention schedules and permissioned_as. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: use u/username format for custom trigger/schedule deploy selection When picking a custom user for trigger/schedule deployment, store u/${username} (permissioned_as format) instead of the email. Flows/scripts continue to use email format for on_behalf_of_email. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: show u/username format for "me" option in trigger deploy selector Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * refactor: simplify OnBehalfOfSelector to return the right format per kind OnBehalfOfSelector now handles the email vs permissioned_as format internally based on kind: - triggers: returns u/username, displays u/username in all options - flows/scripts/apps: returns email, displays username The onSelect callback now takes (choice, value?) where value is already in the correct format. Parent components just store it directly without needing to know about the format difference. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: always show u/username format in OnBehalfOfSelector for all kinds Display is now consistent: all kinds show u/username in the selector. The returned value still differs (email for flows/scripts, u/username for triggers) since the backend APIs expect different formats. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: replace email with permissioned_as in http_trigger test insert The email column was dropped from trigger tables in the migration. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: review fixes — migration, app policy, capture cleanup, naming - Migration: remove DEFAULT '', use nullable → populate → SET NOT NULL - App policy: set both on_behalf_of and on_behalf_of_email for all choices - OnBehalfOfSelector: return OnBehalfOfDetails {email, permissionedAs} instead of ambiguous value - Remove unused email field from Capture struct and query - Rename getSourceEmail/getTargetEmail → getSourceOnBehalfOf/getTargetOnBehalfOf - Rename test functions from preserve_email to preserve_permissioned_as Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: add permissioned_as to all test schedule INSERTs Since the migration no longer uses DEFAULT '', all INSERTs must explicitly provide permissioned_as. Updated test fixtures and schedule_push tests. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: strip permissioned_as from exports/sync, fix OpenAPI required field - Add permissioned_as to workspace export strip list (like edited_by) - Add permissioned_as to CLI TriggerFile Omit list - Fix TriggerExtraProperty.required: email → permissioned_as - Regenerate frontend and CLI types Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: remove accidentally committed generated files These directories are gitignored and should not be tracked. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * chore: regenerate system prompts for permissioned_as schema changes Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: remove permissioned_as from CLI TriggerFile Omit list Already stripped in workspace export, no need to also omit from the type. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: optimize email cache key and revert TriggerFile Omit change - Use single concatenated string for cache key instead of (String, String) tuple - Remove permissioned_as from CLI TriggerFile Omit (already stripped in export) Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: zero-allocation email cache lookups using Equivalent trait Use a borrowed EmailCacheKey(&str, &str) for cache lookups via quick_cache's Equivalent support. Only allocates (String, String) on cache miss for insert. This is called on every trigger fire and schedule push. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: add permissioned_as to Schedule required fields in OpenAPI spec The backend always returns permissioned_as (non-optional String), so the schema should reflect that. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: handle group- prefix in migration UPDATE statements edited_by can be 'group-{name}' for group-owned triggers/schedules. The migration now correctly maps these to 'g/{name}' format instead of incorrectly producing 'u/group-{name}'. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * Revert "fix: handle group- prefix in migration UPDATE statements" This reverts commit 0971392b38. * fix: use superadmin email to resolve permissioned_as in schedule migration For users upgrading from older versions where edited_by may not reflect the actual schedule owner, check if the email belongs to a superadmin and look up their username. Otherwise fall back to edited_by. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: fall back to superadmin email when not in workspace usr table If the superadmin isn't a member of the workspace, use their email as raw permissioned_as instead of falling back to edited_by. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: always update permissioned_as and email on schedule edit Consistent with pre-refactor behavior where email and edited_by were always updated on every edit. permissioned_as is now always set (to editing user or preserved value), removing the COALESCE that previously preserved it when not provided. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * feat: add schedule permission tests and centralize group prefix constants Tests: schedule create/update for normal user, workspace admin, and superadmin not in workspace. Verifies schedule fields (email, permissioned_as, edited_by) and pushed job fields (permissioned_as, permissioned_as_email). Constants: centralize "u/", "g/", "group-" as PERMISSIONED_AS_USER_PREFIX, PERMISSIONED_AS_GROUP_PREFIX, USERNAME_GROUP_PREFIX. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: use @unknown.windmill.dev for synthetic email fallback Prevents privilege escalation: a user with username like 'superadmin_secret' would get superadmin via the synthetic email matching SUPERADMIN_SECRET_EMAIL. Using a different subdomain avoids any collision with hardcoded @windmill.dev emails. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * update ee ref * sqlx * chore: regenerate system prompts after main merge Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * chore: update ee-repo-ref to bda51bc33bcb573659e7ff07d0a23ff6e23b8148 This commit updates the EE repository reference after PR #468 was merged in windmill-ee-private. Previous ee-repo-ref: 8cf1802f8fe183f430830590b4f3172a50207843 New ee-repo-ref: bda51bc33bcb573659e7ff07d0a23ff6e23b8148 Automated by sync-ee-ref workflow. --------- Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com> Co-authored-by: windmill-internal-app[bot] <windmill-internal-app[bot]@users.noreply.github.com>	2026-03-20 16:28:38 +00:00
Ruben Fiszel	31d6660d56	feat: script module mode with CLI sync, preview, and WAC UI improvements (#8380) ... * feat: add script module mode with folder model for Bun and Python Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * fix: add missing modules field to RawCode in bun_executor Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * sqlx * feat: enrich WAC templates with checkpoint and replay semantics Add prominent comments explaining that all computation must happen inside task/step/taskScript or it will be replayed on resume/retry. Clarify that waitForApproval does not hold a worker and that approve/reject URLs are available in the timeline step details. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * fix(cli): script module sync idempotency, per-module hash tracking, and preview support - Fix pull→push idempotency: use `??` instead of `||` for module lock field so empty strings are preserved (matches API's `lock: ""`) - Add per-module hash tracking in wmill-lock.yaml following the flow inline script pattern (SCRIPT_TOP_HASH + per-module subpath hashes) - Selective module lock regeneration: only regenerate locks for modules whose content actually changed, not all modules - Use unfiltered rawWorkspaceDependencies for module hashes to match what updateModuleLocks passes to fetchScriptLock - Show changed module names in stale script output for clarity - Add module support to `script preview` command: read modules from __mod/ folder and pass them in the preview API request - Add preview tests for taskScript pattern (flat and folder layout) - Update test assertion for module stale detection output Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * feat(frontend): WAC UI improvements — reorder templates, module tab rename, import consolidation - Reorder WAC template buttons: TypeScript before Python in ScriptBuilder, CreateActionsScript, and CreateActionsFlow - Remove dropdown items from +Script button (simplify to direct link) - Move "Import Workflow-as-Code" to +Flow dropdown with dedicated drawer - Add module tab rename: pencil icon on hover opens popover with validation, fixed-width icon container prevents layout shift Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * feat: remaining module-mode changes from working branch - Backend parser updates for WAC detection - CLI sync/types updates for raw app path and module support - Frontend UI polish (Dev.svelte, ScriptRow, script hash page) - Test fixture updates Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * test(cli): add test for module modification detection in generate-metadata Verifies that modifying a single module file re-triggers stale detection and only the changed module is listed, not all modules. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix(backend): critical fixes from PR review - Fix hardcoded dev path in bun_executor.rs WAC v2 wrapper — use "windmill-client" import instead of absolute filesystem path - Fix missed no_main_func → auto_kind rename in parser TS test - Add modules column to clone_script SQL (windmill-common and windmill-api-workspaces) so cloned scripts retain their modules - Add modules: None to RawCode structs in worker tests - Restore complete sqlx cache (merge main's cache + our new queries) Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix(backend): fix clone warning treated as error in CI Change `.clone()` on double reference to `*k` dereference in scripts.rs hash implementation. Update sqlx cache with new query hashes from modified clone_script SQL. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix(frontend): use published parser wasm versions for CI build The local file:// paths for windmill-parser-wasm-py and windmill-parser-wasm-ts don't exist in the Cloudflare Pages build environment. Revert to published npm versions (1.655.0). Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix(frontend): update parser wasm packages to 1.657.2 Use newly published windmill-parser-wasm-ts and windmill-parser-wasm-py v1.657.2 which include auto_kind/WAC detection changes. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix(frontend): regenerate package-lock.json for npm ci compatibility Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix(frontend): use main's lockfile as base, update only parser wasm packages Regenerating package-lock.json from scratch pulled different dependency versions causing svelte-check type errors. Instead, start from main's lockfile and only update the two changed packages. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix(backend): add modules column to fetch_script_for_update query The Script<SR> struct has a modules field (FromRow), but fetch_script_for_update didn't SELECT modules, causing a runtime error "no column found for name: modules" when the worker processed dependency jobs. This was the root cause of the relock_skip test timeout. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix(backend): fix script module execution for Python and Bun - Fix modules not passed through job queue: inject _MODULES into PushArgs.extra when pushing Code jobs so worker can extract them - Fix Python module imports: use relative imports (from .helper) and add sys.path.insert for module directory in wrapper - Fix Python tests: use relative imports and empty lock to prevent pip from resolving module names as packages - Add local file check in Bun loader for module resolution - Ignore Bun module test (bundle mode loader integration tracked separately) - Add missing modules column to fetch_script_for_update query Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix(backend): remove unnecessary empty lock in Python module tests Relative imports (from .helper) are not parsed as pip packages, so the empty lock workaround is not needed. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix(backend): fix module execution for Python and Bun — all tests pass Python modules: - Use relative imports (from .helper import greet) since scripts run as packages - Add sys.path.insert for module directory in wrapper to ensure local modules take precedence over pip packages with same name Bun modules: - Use bundled output (./out/main.js) as wrapper import when modules are present — the bundled output has module content inlined by Bun.build, avoiding runtime loader resolution issues - Add local file check in loader.bun.js onResolve to short-circuit API URL resolution for module files on disk Job queue: - Inject _MODULES into PushArgs.extra when pushing Code jobs so the worker can extract them at execution time Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * refactor: address PR review — simplify, fix correctness, remove dead code Critical fixes: - Replace all CLI `no_main_func` references with `auto_kind` (string) to match the backend migration and API changes - Remove duplicated `compute_python_module_dir` in worker.rs, use the canonical version from python_executor.rs High priority: - Auto-create `__init__.py` in intermediate directories for nested Python modules so imports like `from .utils.math import add` work without users manually creating __init__.py files - Remove redundant `sys_path_insert` — relative imports use Python's package system, not sys.path Medium: - Fix lock file base name extraction: use regex to strip only the final extension (`.replace(/\.[^.]+$/, '')`) instead of `indexOf(".")` which breaks for files like `helper.test.ts` Simplification: - Remove dead `{#if false}` Popover block in ScriptEditor.svelte - Guard loader.bun.js local file check to only run for relative paths (matching the Windows loader pattern) - Add clarifying comment on Bun dual mechanism (build + run phases) - Add maintenance comment on manual Hash impl for NewScript Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: final review fixes — stale cleanup, baseName, auto_kind export - Fix sync.ts baseName extraction using indexOf(".") → regex (same fix as script.ts/metadata.ts, missed this instance) - Add stale module file cleanup in writeModulesToDisk: removes files from __mod/ that are no longer in the modules map before writing, fixing the pull→push cycle that couldn't delete modules - Log warning when _MODULES serialization fails in job push instead of silently dropping modules - Use strict equality (===) for auto_kind comparison - Exclude auto_kind from workspace export — it is auto-detected by the parser at deploy time from script content Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix(cli): remove auto_kind from push, comparison, and metadata auto_kind is auto-detected by the parser at deploy time, so the CLI should not send it, compare it, or write it to script.yaml. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: remove erroneously added backend/backend/.sqlx directory Duplicate .sqlx cache was committed at the wrong nested path. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: address PR review feedback + fix CI dead_code warning Frontend (ScriptEditor.svelte): - Fix switchToMain() missing lastSyncedCode update — prevents stale code sync on external changes while editing a module tab - Fix formatAction saving module code to main script's localStorage draft — now saves main code when on a module tab - Fix non-null assertion on inferModuleLang in renameModule — fall back to original language instead of force unwrap - Remove redundant activeModuleTab truthy check in runTest CLI (script.ts): - Clean up empty directories after removing stale module files in writeModulesToDisk Backend: - Add path traversal guard in write_module_files — reject module paths containing ".." - Fix dead_code warning on auto_kind field in workspace export struct Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * feat(frontend): improve auto_kind UX + address review findings - Rename "Include without main function" toggle to "Include library scripts" in script list (ItemsList.svelte) - Update NoMainFuncBadge: "No main" → "Library" with clearer tooltip - Filter module file extensions by main script language — Python scripts only allow .py modules, TypeScript only .ts, etc. - Split flushModuleState into flushModuleContent (no UI side-effect) and flushModuleState (flush + reset tab), reducing duplication - Dynamic placeholder and hint text in add module popover based on main script language Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>	2026-03-17 01:20:09 +00:00
Ruben Fiszel	b31a475c88	feat: add workspace dependencies support for powershell (#8395) ... * feat: add workspace dependencies support for powershell Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * feat: improve workspace deps editor UX for powershell Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * feat: add powershell workspace deps support to CLI Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>	2026-03-16 15:37:28 +00:00
Ruben Fiszel	f67b8159ad	warn about missing <clear /> in nuget config and make description optional (#8281) ... Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>	2026-03-09 13:11:16 +00:00
Ruben Fiszel	5f0ef936d1	feat: add sandbox annotations, volume mounts, for AI sandbox starting with claude (#8058)	2026-03-05 06:19:51 +00:00