76 Commits

Author	SHA1	Message	Date
hugocasa	c09de594b6	feat: version resource values with history, diff and restore (#10596) ... * feat: version resource values with history, diff and restore Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix: record resource versions in a trigger so direct writes are covered Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix: show the selected version's value and tighten history write access Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * perf: gate resource version recording in trigger WHEN clauses Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * feat: clear a resource's past versions, and address review nits Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix: restore the displayed version and keep author attribution on pooled writes Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix: scope history to the selected workspace and gate clearing on ownership Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix: gate restore on write access and clearing on the signed-in workspace Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * refactor(frontend): share the version-history row between script and resource drawers Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * perf: trim resource version history in the monitor sweep, not on write Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix(frontend): match the script versions drawer shell for resource history Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * perf(frontend): highlight version values instead of mounting monaco Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix(frontend): match the script drawer's code preview presentation Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix: rank version trim in one windowed pass instead of a correlated delete Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * refactor(frontend): treat the newest version as current by position Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * perf: gate the resource version trim to an hourly sweep Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix: unnest the version row action and correct the trim cadence docs * perf: cap the history listing and use sets for reference lookup * feat: warn when a resource is written more than 60 times a minute * fix: lower the resource write advisory to 20 per minute * fix: discard stale history loads and never diff against an unread value * fix: correct the write advisory boundary and document the eviction lock * fix: read history and the live value from one snapshot * refactor: read the drawer's diff baseline from versions, not the live resource * fix: open the history drawer with no version selected * fix: disarm the clear confirmation and clear the pane when the selection moves * fix: explain the missing diff and drop a guard that can no longer fire --------- Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>	2026-08-10 21:32:18 +02:00
Ruben Fiszel	fb82748296	fix: make on_behalf_of control permissions for scripts and flows (#10438) ... * fix: make on_behalf_of control permissions for scripts and flows Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix: inherit the recorded on-behalf-of identity when a preserving deploy omits it Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix: keep an omitted permissioned_as from re-versioning an unchanged script Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix: derive the on-behalf-of principal from the email and reject mismatched pairs Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix: stop workspace deploys from carrying a source-workspace principal Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * docs: correct the onBehalfOfPermissionedAs param doc Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * test: pin that workspace deploys never carry a source-workspace principal Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * docs: correct the omitted-principal contract and refresh generated prompts Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix: keep external-superadmin principals on email-only redeploys Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix: scope the recorded principal to its workspace and prefer real accounts Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix: carry the recorded principal correctly through drafts and set-permissioned-as Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix: sweep draft identity pairs on email change and offboarding Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix: leave group identities alone when sweeping a user's email Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix: treat only g/ without an email as a group, and match the offboard preview Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix: stop the group guard from skipping rows with no recorded principal Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * docs: state the group guard once instead of restating it Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * refactor: make the permissioned_as the only stored on-behalf-of identity Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * perf: skip resolving the on-behalf-of address for sync clients that discard it Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix: address the local review of the identity refactor Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix: resolve the on-behalf-of identity coherently across clones, offboarding and no-op deploys * test: pin that a fork keeps only the on-behalf-of identities that resolve in it * fix: decide a principal prefix-first everywhere and canonicalize bare addresses * fix: prefix a slash-containing address so a reader cannot take it for a group * fix: read an address as a username before the group- convention * fix: rewrite the canonical principal when an account's address moves * fix: keep the address form of a principal to accounts without a usr row * fix: reject an identity a job row cannot carry and read it uncached at dispatch * fix: count characters against the job identity width and cap the backfill * refactor: name the script/flow principal on_behalf_of, as apps do * docs: state the caller-must-authorize contract on the identity resolvers * fix: keep writing on_behalf_of_email until every worker reads the principal * fix: err high on the compatibility version and document the last resolver * fix: keep the compatibility address current through identity mutations * fix: carry the compatibility address with the principal on every copy path * chore: re-pin the EE ref to the companion branch merged with EE main * fix: key the dbt retry lookup on the stored principal * fix: keep a mixed-version address recoverable through a fork * fix: read a round-tripped address uncached so a redeploy is not rejected * fix: refuse an email change that would make a principal unenqueueable * chore: update ee-repo-ref to ac3d7d015296f041ae44ab6bc4953485f44d36e4 This commit updates the EE repository reference after PR #704 was merged in windmill-ee-private. Previous ee-repo-ref: 219b0b03905a1a0028054b3a4985724e77d09036 New ee-repo-ref: ac3d7d015296f041ae44ab6bc4953485f44d36e4 Automated by sync-ee-ref workflow. --------- Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com> Co-authored-by: windmill-internal-app[bot] <windmill-internal-app[bot]@users.noreply.github.com>	2026-08-01 20:37:21 +02:00
Ruben Fiszel	032300e28e	feat: run dbt projects as a first-class Windmill runtime (#10326) ... * fix: mount only the engine in the dbt jail, reject shadowed and malformed args Review round 42. The jail mounted the whole dbt cache directory, whose siblings of the engine are `repos/` and `packages/` — other workspaces' private checkouts and package trees, kept apart by cache key rather than by permissions. A jailed project could read them. It now mounts the engine's own directory, which the provisioner names; verified from inside the jail that `repos/`, `packages/` and `state/` are invisible while the engine stays usable. A `{{ placeholder }}` may no longer take the name of a run argument this runtime defines. It was silently dropped from the signature, so a descriptor like `value: "{{ select }}"` deployed and then could not be run at all: the built-in `select` is an array and the interpolation needs a scalar. Refused at parse, so the deploy says so. A `vars` override that is not an object is refused rather than ignored. Argument-schema validation is opt-in, so a string or an array silently ran the descriptor's own vars — against a different schema or alias than the caller asked for. `select` and `exclude` already refused theirs. * feat(dbt): the project is the script's module bundle, not a git checkout A dbt script now carries its whole dbt project as its module bundle. The descriptor is the script content; `<script>__dbt/` holds the project verbatim, so importing an existing project is `cp -r` plus `wmill sync push`, and the worker materialises the bundle into the job directory instead of cloning. Backend - `prepare_project` writes the script's modules and requires `dbt_project.yml` at the bundle root. `checkout`, the git-ssh command, the clone cache and the repository resource are gone, along with `repo`, `project`, `ref` and `git_ssh_identity` on the descriptor. - Run identity and the package cache key take a `project_digest` (sorted SHA256 over the bundle) where the commit used to sit, so an edited project cannot resume a previous run's `run_results.json` or reuse its `dbt_packages`. - The per-run graph re-ingest is now gated on `vars` placeholders and `$var:` env alone. - `capture_dependency_job` takes the script's modules so a dependency job, which has no generic module-writing step, materialises them itself. - `dbt deps` caching strips the git remote from every package it cached, not just the tree root: `packages.yml` can render a token into a `git:` URL. - `git_clone.rs` is dropped and `ansible_executor.rs` returns to its own copy of the clone helpers. CLI - `wmill sync pull` keeps a dbt script's lock beside its folder rather than inside it, so the folder holds nothing but the project. - Directories dbt generates (`target-path`, `packages-install-path`, `clean-targets` and the usual defaults, read from `dbt_project.yml`) are excluded from the bundle, from the sync diff and from staleness hashing. - A module-only edit now pushes its parent dbt script and is reported as a changed module rather than passing unnoticed. * fix: keep a script's modules in the worker's file-system cache The first fetch of a script version reads the database and carries its modules; every later fetch imports from the worker's cache directory, whose `RawScript::import` hard-coded `modules: None` and whose `export` never wrote them. A worker restart therefore started running the script without its own files, silently — for a dbt script, without its project, which fails with "carries no project"; for any other script with a module bundle, with the imports missing. `modules.json` is now written on every export and required on import, so an entry written by an older version fails to import and is refetched rather than serving a stripped script for as long as the directory lives. Also derives a dbt run's `project_digest` from the bundle the run actually carries: `handle_dbt_job` was passing `None`, which collapsed every project in a workspace onto one digest and let `dbt retry` resume a different project's `run_results.json`. * fix(dbt): give every phase the script's environment, bound the cache copies `dbt deps` ran without the script's environment variables on an unsandboxed worker, so a `packages.yml` resolving a private package URL through `env_var()` could not see them while the package cache key was still built on their digest. `with_invocation_env`, applied at three of the four call sites, is folded into `dbt_command` so no phase can be added without it, and `DBT_TARGET_PATH` is set after both environments rather than before. The package cache copies ran through a bare `Command::output()`: the tree is the project's, so a cancelled or timed-out job held its worker slot until `cp` finished. Both the restore and the publish now run under the job poller like every other phase. * fix(dbt): only offer commands whose writes match the graph, honour packages-install-path `dbt_command: run` is dropped from the allowed overrides. Asset dispatch fires a script's deploy-time writes on any successful job, and `dbt run` covers models only, so a project with seeds or snapshots notified consumers of relations the invocation left stale. That is the same reason `test` was already excluded. Narrowing what a run touches is `select`/`exclude`, which scope the graph too. `dbt deps` writes to the project's `packages-install-path`, so a project that moved it got no package cache at all: the publish found nothing at `dbt_packages` and every job resolved its dependencies over the network again. The path is read from `dbt_project.yml` and validated as project-relative, since both cache copies are rooted at it. Also states the sidecar's mutator contract at the module level: the dbt manifest tables carry no RLS and grant `windmill_user` full access, so a user-scoped transaction is not enforcement and every caller must have verified write access to the script itself. CLI: a module file is now grouped with its parent script for the push. Left in a group of its own it got its own `alreadySynced`, so a push touching several files of one bundle deployed the script once per file; the resulting versions raced, and the asset graph could end up describing none of them. * fix(dbt): seed a project for browser-created scripts, refuse a no-op retry A dbt script created in the browser only got a descriptor, and the runtime refuses a script whose bundle has no `dbt_project.yml`, so the advertised Create → Deploy → Run path always failed its dependency job. New dbt scripts now start with a project that builds: pointing `profile.resource` at a warehouse is the one edit, and growing it is `wmill sync pull` plus a local editor, which is where dbt development happens. `dbt retry` builds its graph from the previous run's error, fail and skipped nodes alone, so retrying an all-green run selected nothing and wrote nothing — and a job that succeeds having written nothing still dispatches every deploy-time write, waking every downstream consumer for relations no one touched. Refused, with the reason. CLI: a configured `target-path` or `packages-install-path` may be nested (`build/target`), and `clean-targets` has a block form as well as an inline one. Both are now parsed, and the exclusion compares the project-relative path rather than the top-level segment, so a nested generated tree no longer lands in the bundle and no longer makes a local `dbt run` look like a project change. * fix(dbt): lock a project once, find the parent on either path separator A dbt script's modules are its dbt project, not helper code with dependencies of its own, so the generic per-module lock loop is skipped for it: the parent lock already ran `dbt deps` and `dbt parse` over the whole project. Locking each file separately re-materialised the bundle and re-invoked dbt once per file, so a project of N files paid N project-sized passes and a large one timed the deploy out. The 13-file fixture went from 14 relock passes to 1. `pushParentScriptForModule` searched the raw path for `__dbt/`, so on Windows, where the folder is spelled `__dbt\`, a module-only edit returned without deploying its parent while the caller still recorded the file as synced. It now goes through `getScriptBasePathFromModulePath`, which normalizes separators. Also drops the last of the external-repository wording from the descriptor's module docs and from the `codebase` rejection a user can hit. * feat(dbt): infer the run form locally, keep test-only retries from cascading `windmill-parser-wasm-yaml` 1.770.0 carries `parse_dbt`, so the browser and the CLI derive a dbt script's run arguments from its descriptor instead of waiting for the deploy to hand back a schema. Pins bumped in both. `generate-metadata` was rewriting a dbt script's `lock` field on every run: a dbt lock comes from the dependency job on a worker, so nothing generates it locally and the resolved `!inline` reference was left inlined into the metadata or blanked. It is restored instead, and a push straight after `generate-metadata` is a no-op again. A retry now needs a failed node that materialises something. `dbt retry` builds its graph from error, fail and skipped nodes, and with `test_behavior: after_all` a failing test is what `run_results.json` ends up describing — so the retry reran tests, wrote nothing, succeeded, and still dispatched every deploy-time write. The dbt badge's destination is deterministic: writers outrank readers, and among several writers of one relation (which the backend permits) the smallest id wins, rather than whichever write edge arrived last. * feat(dbt): browse the project and read a run's per-node result Two views a dbt user expects and that the generic script surfaces do not give. **The project.** A dbt script's editor gains a Project tab beside its descriptor: the module bundle as the tree dbt itself expects, each file read-only with syntax highlighting. The existing module tab strip is a flat row built for a couple of helper files and does not survive a real project; a 13-file fixture already overflows it. Directories sort before files so it reads like the checkout on disk, and an empty bundle explains the `cp -r` instead of showing a blank pane. **The run.** `DisplayResult` renders a dbt invocation's per-node breakdown above the raw payload: totals, then a table of node, kind, target relation, rows and time, with failures and warnings sorted first and carrying their message. The data was already structured; it was being shown as JSON to scroll and PASS/WARN counts to find in the log. On a failed run the same JSON rides in the error message after the exit-status line, so it is parsed back out — that is the case worth rendering, since the failing node is what the user came for. * docs(dbt): say that profile.resource is what buys the asset graph The starter descriptor described `profile.resource` as the thing rendered into profiles.yml, with the project's own file as an equal alternative. It is not equal: the resource PATH is the warehouse's identity in the asset graph, so a project bringing its own profiles.yml runs fine and silently gets no assets, no lineage and no cascade. The deploy already says so in its log; now the descriptor a user starts from says it too, before they choose. * fix(dbt): authorize a resource used only for asset identity, clean up after failed installs A descriptor setting both `profile.profiles_yml` and `profile.resource` took its connection from the project's file but returned the resource path as the graph's warehouse identity without ever reading it. A script editor could therefore publish `table://<any resource>/...` writes, and wake that warehouse's subscribers, while connecting somewhere else. The resource is now read on that path too — reading is what authorizes it — so the combination keeps working for the case that wants it (keep your own profiles.yml, still get lineage) and fails closed otherwise. Provisioning cleaned up its staging directory only on the paths someone remembered, so a run of failed or cancelled first-use installs accumulated venvs, tarballs and installer scripts until the worker's disk was gone. All three engines now hold their scratch paths in a guard that removes them on drop, which is the one exit every path takes, cancellation included. Frontend: `partial success` is dbt's word for a node that built but whose tests failed, counted in `totals.error` and redone by a retry, so it ranks with the failures instead of rendering green with its message hidden. And the run panel now keys off the worker's engine discriminator rather than `{nodes, totals}`, which is a shape an ordinary script can return. Both pinned by unit tests on the extracted `parseDbtRun` helpers. * feat(dbt): show a run's models on the run page The run page is where you land on a running job, and until now it showed a dbt run as streaming text: the per-node table only renders once the job has produced a result, and the graph that moves per model lived on the pipeline page you had to navigate to. A Models section now sits above the result, scoped to the running script's own relations and its `ref()` lineage, polling while the job is in flight so nodes move as dbt walks the DAG. No `resolveGraph`: that merges drafts and live editor buffers into the persisted graph, and a run page has neither. * feat(dbt): retry failed nodes automatically, and from any worker **Node-level retry, in the job.** `retry_failed_nodes: {attempts, delay_seconds}` rebuilds only what a failed build left failed or skipped, before the job reports failure. dbt confines a failure to its own subtree and `dbt retry` resumes exactly that set, so a transient warehouse error costs those nodes rather than the project. Doing it in-job is what keeps the state question out of it: the previous attempt's `run_results.json` is still in the job directory, so there is nothing to persist and no worker to land back on. This is the granularity astronomer-cosmos gets from one Airflow task per model, without the ~6x that per-model tasks measured. A retry's `run_results.json` names only the nodes it redid, so it overlays the accumulated results rather than replacing them: the job's result has to be every node the job touched, or the nodes that succeeded before the retry settle no materializations. Pinned by a test. **Durable retry state.** `run_results.json` is now saved to `dbt_run_state` as well as the worker's local cache, so an explicit `dbt_command: retry` works from any worker of the group rather than only the one that failed. Only the results are stored: `dbt retry` also needs `manifest.json`, roughly sixty times larger and growing with the project (732 KB against 12 KB on the six-node fixture), but the manifest is a pure function of the project files, vars and env, all of which the stored identity already pins, so a worker restoring from the database re-derives it with a `dbt parse` of about a second. * fix(dbt): restore the sqlx cache, make retries cancellable and path-aware **SQLx cache.** A `cargo sqlx prepare` deleted 750 entries, including the enterprise queries CI needs under `SQLX_OFFLINE=true`, and the check that was supposed to catch it reported zero losses because it was run from `backend/` with a `backend/`-prefixed path, so its baseline was empty and it failed open. All 750 are restored; the branch now adds 19 and deletes none, and `SQLX_OFFLINE=true cargo check` passes. **Retry backoff observes cancellation.** `canceled_by` is only written by the job poller, which does not run between attempts, so re-reading it reported the state as of the failed attempt and missed every cancel issued during the wait — the whole window the check exists to cover. The wait now reads `v2_job_queue.canceled_by` each second, and the job's deadline is honoured before starting another dbt process. **Retry state follows its script.** `dbt_run_state` is path-keyed like the manifest sidecar but, unlike it, nothing regenerates it: a rename moves the row so a resumable failure survives, while archive and delete clear it, so a script later created at that path cannot inherit a stranger's failure and its arguments. **CLI.** `table` joins ducklake and s3object in the local graph's auto-trigger kinds, matching `is_auto_trigger_kind` and the frontend's set; without it a local graph and the generated docs omitted a cascade edge the deploy has. * fix(dbt): carry only the project files a bundle can hold, and say what it drops Exploring real and edge-case projects surfaced three frictions, all in the import path a user hits first. **A binary file broke the push, opaquely.** dbt projects carry images under `docs/`, stray `.DS_Store` files and occasionally a parquet seed. Read as text they become mojibake, and a NUL among them is rejected by Postgres with `unsupported Unicode escape sequence` — which `wmill sync push` then reported as success, exiting 0 with the script never created. Binary files are now detected the way `git` detects them, by a NUL in the first 8000 bytes rather than by extension, and skipped with the reason. **The size guard the docs promised did not exist.** Now it does: 5 MB per file, which only ever catches a committed dataset. Real dbt code is about 500 bytes median and 1.9 KB at p90. **Skipped files became a permanent phantom diff.** The push dropped them while the sync diff still offered them, so every push reported changes no push could resolve. One predicate now answers for the push, the staleness hash and the diff. Verified on a project with unicode filenames and content, CRLF endings, an empty model, an ephemeral model, a disabled model, a `.md` docs block, an extensionless README, six levels of nesting, a 7.6 MB seed and a PNG: it pushes, round-trips byte-for-byte through pull, deploys to 7 dbt nodes and 6 `table://` assets (ephemeral and disabled correctly absent), and runs green. * fix(dbt): resolve dbt-core against the adapter, settle partial success, unify status **Adapters could not be provisioned.** The 1.x engine pinned `dbt-core` to a fixed version independent of the adapter, but several adapters cap below it: `dbt-mysql` at `~=1.7`, `dbt-oracle` and `dbt-databricks` below 1.12, and `dbt-salesforce` has no package at all (it exists only inside Fusion). Those projects failed at provisioning with a uv resolver dump. The install now asks for a range and lets the adapter choose, and records what the resolver picked so the lock pins a version that adapter can take. The floor is the CLI this runtime invokes: resolving down to dbt-core 1.7 produced a working venv that then failed with `No such option '--target'`, which is worse than not resolving. An adapter with no release in range now fails naming itself and pointing at `dbt-core-2x` or `fusion`, instead of a resolver dump. Salesforce is refused up front with the reason. **`partial success` left a model stuck on `Running`.** It is dbt's word for a node that built and then failed its tests, and it was already treated as a failure when counting totals and deciding a retry — but the two sites that settle the RELATION fell through to "says nothing", so the tailer's `Running` was never replaced and a finished job showed a model still building. Six status comparisons had drifted apart, two folding case and four not, while dbt-core 1.x echoes the author's casing and 2.x uppercases; they are now one classifier. **Agent workers.** The durable retry state and the cancellation poll both need a database, which an agent worker reaches only through the API. The automatic node retry is refused there rather than running a wait it could not interrupt, and the docs say "any worker with a database connection" instead of overclaiming. Also clears `dbt_run_state` when a path stops being a dbt script, and moves `run_identity`'s contract onto `run_identity` from the digest helper below it. * feat(dbt): show the transform behind a model on the run graph The run page's graph carried a node for the script itself and drew every relation as a bare table. Both were wrong for that page: the graph there is already scoped to one script, so a node standing for it distinguishes nothing (on the pipeline page it separates one project from another, which is why it exists), and dbt's own DAG node is the model — the SQL and the relation it writes are one thing, so a graph of relations alone leaves out what a reader came to see. The script node is dropped, and selecting a model now shows its SQL underneath the canvas with its file path and materialization, read-only, the same view the pipeline details pane gives. * feat(dbt): move the graph with the run The worker has always recorded a state per relation as dbt walks the DAG — `running` when a model starts, `materialized` or `failed` when it ends — but nothing rendered it: the graph response carries what a relation IS, not what a particular run is doing to it, so the canvas had nothing to show and a running job looked identical to a finished one. `assets/run_progress/{job_id}` returns that state for one job, the run page polls it beside the graph, and the asset node carries a spinner or its outcome. Errors and retries need nothing extra: a failed node writes `failed`, and an in-job retry rewrites the same row, so the node returns to `running` and on to its new outcome by itself. `materialized_partition` holds a relation's CURRENT state keyed by relation, so filtering on `job_id` returns exactly what this run last touched — which is the question a run page asks, and why a superseded older run shows nothing. * feat(dbt): a dbt project is not a data pipeline Deploying a dbt script marked it `auto_kind = 'pipeline'`, which enrolled it in pipeline membership: the folder became a Pipeline entry on the home page, the script folded into it, and `/pipeline/<folder>` opened a canvas holding the project's whole model DAG next to the pipeline's own scripts. A folder holding both then read as two projects in one editor, and the pipeline editor offered to author transforms that are in fact authored in a local `dbt run` loop and pushed as the script's bundle. A dbt script is now never a pipeline member, and the pipeline canvas drops the dbt script node. Its models stay, with their `ref()` lineage: the relations are what a downstream pipeline script reads, and dropping them would break the cascade from a dbt run — the point of giving dbt models `table://` identity. Also drops a screenshot committed to this branch by accident. * fix(dbt): authorize run_progress through the job, drop dbt from the local graph `run_progress` read `materialized_partition` through `user_db` on the assumption that RLS would scope the rows. That table has RLS disabled and no policies, so any workspace member could pass a job id and read that run's relation paths, row counts and error text. It now joins `v2_job`, which does carry per-user policies, so a caller who cannot see the job sees nothing — the same pattern `v2_job_completed` reads need. Verified as a plain member: the old query returned 6 rows for another user's run, the new one returns 0, while the job's owner still sees all 6. The CLI's local graph still forced `in_pipeline` on every dbt script, so `pipeline docs --local` and `pipeline dev` kept presenting a dbt project as a pipeline the deploy no longer enrolls. It now skips them, matching the server. A dbt descriptor has no asset parser locally, so nothing is lost: its models come from the manifest the deploy derives. Declares `run_progress` in openapi.yaml so the frontend uses the generated client instead of a handwritten fetch; the generated `status` union also replaces a hand-rolled string mapping. * fix(dbt): drop the dbt node from the CLI's deployed pipeline views too `pipeline dev` and `pipeline docs` (without `--local`) read `/assets/graph` directly. That endpoint is asset-usage driven rather than membership driven, so it returns a dbt script like any producer — and both commands render every runnable, so a dbt project still showed up as a pipeline script there after the local builder stopped emitting one. `hideDbtRunnables` mirrors the frontend's projection of the same payload. It is generic over the graph shape so the bounded-cascade view (`BCGraph`, a narrower type over identical JSON) passes through without a cast. The relations stay: they are what a downstream pipeline script reads, and the node is what attributes them to a producer for every other consumer of the endpoint, so the filter belongs in the views rather than the query. * fix(dbt): narrow a selective run's cascade, settle the finished run graph Review-round fixes. A `select`/`exclude` run builds part of the project, but asset dispatch reads the deploy-time write set for the whole script, so a run selecting one model woke the subscribers of every other. Dispatch now intersects that set with the relations the run actually recorded as materialized, scoped to dbt because it is the only producer whose write set is decided per run. A run that recorded nothing still dispatches everything, so an agent worker whose reconciliation failed cascades as before. Verified both ways: `select: [extra_model]` no longer wakes the `fct_orders` subscriber, and a full run still does. `hideDbtRunnables` keyed its removal set on path alone while the graph keys runnables by `(usage_kind, path)`, so a flow sharing a path with a dbt script lost its node, edges and triggers too. Both copies now key on the pair. The run graph never took a final reading when a job finished, so the last state shown was whatever the tick before completion saw. Only `dbt-core-1x` streams node events; the other engines record every relation during end-of-run reconciliation, so their finished graph showed nothing until a reload. `DbtNodeOutcome::Inconclusive` collapsed statuses the tally has to tell apart, so two sites re-lowercased the status beside the classifier and `no-op` landed in `totals.error` — a clean run reporting an error in its own result. Split into Warn / Skipped / NoOp / Unknown so every site falls out of one match; `no-op` is kept out of the retry set, which dbt spells as error / fail / skipped. Also: reattach two doc comments to the items they describe, and correct the engine-distribution table — only dbt-core-2x is baked into the images, 1.x is a per-adapter venv provisioned on first use, and the default is compiled in rather than an instance setting. * fix(dbt): make the model chip inert where its project node is not on the graph The canvas passed `onDbtSelect` unconditionally, so the chip always rendered `cursor-pointer` and hover-highlighted — but the owner map is empty on both graphs this feature added, since the run page carries no runnables and the pipeline page hides the dbt node. The chip advertised a click that resolved to nothing. It now takes its handlers only when the relation has an owner on this graph, so it stays live on the surfaces that do show the project node. `classify_status` and `DbtNodeOutcome` were `pub` in a private module with no caller outside the file, unlike every neighbour. * fix(dbt): take the cascade's write set from the run's own result The previous narrowing read `materialized_partition`, which was wrong twice. That table keeps one row per relation and the newest writer takes `job_id`, so two overlapping runs over the same model erase each other's claim to it: the earlier job would dispatch a subset of what it built, or none of it. And an empty row set was read as "recording failed, dispatch everything" when it is also a real answer. A `select` matching no model, or one resolving to tests only, exits 0 having built nothing — and then woke every consumer of every model in the project, which is the opposite of what the narrowing exists to do and is reachable by a typo in a run argument. The run now reports the relations it materialized in its own result, which is immutable and per job. Absent means the producer said nothing (a job from before the field, a non-dbt producer) and the whole deploy-time set dispatches as before; present-and-empty means it built nothing and dispatches nothing. Verified on all three: an unmatched selector builds nothing and wakes nobody, a selector naming one unsubscribed model wakes nobody, and a full run wakes the subscriber. Also indexes `materialized_partition (workspace_id, job_id)` -- the run page polls that shape every 2s and no existing index leads with `job_id` -- corrects the selective-cascade section of the design doc, which still described the old deploy-time behavior, and reattaches `buildLocalPipelineGraph`'s doc comment. * docs(dbt): attach the CLI JSDoc to its function, correct the index rationale The `hideDbtRunnables` JSDoc ended up documenting the type declared beneath it — made while fixing the same mistake one function down. The migration's comment credited the cascade with a `job_id` lookup that the same commit replaced with a read of the job's own result. The run page's poll is the only reader keyed on that column. * fix(dbt): refuse graph publication for a removed script, allow test-only retries An archived or hard-deleted script could still republish its graph: the publication guard filtered `deleted` but not `archived`, and treated a missing row as "nothing newer exists" rather than "nothing left to publish for". A dependency job or dynamic run finishing after the removal put the asset, provenance and subscription rows back with nothing left to clear them. `dbt_command: retry` refused a run whose only failures were tests, which is precisely what `test_behavior: after_all` produces. That restriction existed because a successful job dispatched its whole deploy-time write set, so a test-only retry would have woken every consumer for relations no one touched — the cascade now dispatches what the run reports materializing, so it wakes nobody and the restriction only blocked a legitimate retry. * fix(dbt): gate run progress behind the job-read check, not RLS alone The endpoint joined `v2_job` so RLS would decide visibility, which it does — but `require_job_read_access` adds two things RLS does not: a scoped token's `if_jobs:filter_tags` restriction, and the app-embed cutoff that stops untrusted app JS from inheriting the viewer's broader job access. A scoped or embed token could therefore read relation names, statuses, row counts and errors for jobs the ordinary job endpoints deny it. That helper is private to `windmill-api`, which depends on `windmill-api-assets` rather than the reverse, so the endpoint moves to the job routes instead of the check being duplicated. It is job-scoped anyway: `/w/{ws}/assets/run_progress/{job_id}` becomes `/w/{ws}/jobs/run_progress/{id}`, and the frontend follows the generated client. * feat(dbt): a dbt run does not trigger downstream runs dbt orders its own DAG, so a cascade only ever adds one thing: waking a Windmill script that reads a mart. That edge is narrow, and only half of it can even be expressed — nothing outside dbt can declare a `table://` write, since `// materialize` accepts DuckLake targets only, so an ingestion script cannot wake a dbt project. Against that, dispatching correctly is not cheap. A run's `select` can build any subset of the project, so the deploy-time write set is not what ran; using it wakes consumers of relations the run never touched, and narrowing it needs a per-job record of what was built. The per-relation state table cannot supply one (it keeps a single row per relation stamped with the last writer), and the result field added for it made a run's own output carry the cascade's bookkeeping. So `asset_dispatch` returns early for `ScriptLang::Dbt`, before the producer gate. dbt still materializes, records per-model state and publishes its graph: models, `ref()` lineage and live run progress are unchanged, and a `# on table://<mart>` reader still renders beside the model it reads. It simply does not fire. Wiring it up later means deciding what a selective run should notify, which is the actual work. Verified: a full run of a 6-model project succeeds and starts nothing, where it previously triggered its subscriber; the run page still reports all 6 relations and the folder graph still carries 14 tables and 8 ref() edges. * fix(dbt): remove the cascade surface, settle stranded models, fix nested __mod Stopping dispatch left its surface behind. `table://` was still an auto-trigger kind, `persist_ingest` still derived subscriptions from a manifest's reads, and the deploy still accepted `# on table://` — so the canvas drew cascade arrows into scripts nothing could wake. All three are gone: the kind no longer derives, the ingest only deletes rows earlier versions wrote, and the deploy refuses the annotation with a message saying why rather than persisting a silent no-op. `DescriptorTriggers` went with them; every field it parsed was cascade config. A model marked `running` by the live tailer was never settled when the run did not finish: reconciliation only revisits nodes `run_results.json` names, and a cancelled or timed-out run has none for the model in flight, so the finished job showed a relation building forever. It is now settled on every exit path. Verified by cancelling a run mid-flight: 3 models `running` before, 3 `failed` after, none stranded. `getScriptBasePathFromModulePath` took the first matching suffix rather than the outermost boundary, so `proj__dbt/models/legacy__mod/a.sql` resolved to `proj__dbt/models/legacy`. dbt owns its directory names verbatim, so a folder ending `__mod` is legal inside a project, and a module-only sync would have looked for a descriptor that is not there and skipped the deploy. * fix(dbt): colour a finished run's models from its own result `materialized_partition` keeps one row per relation stamped with whichever job wrote it last, so reopening a run showed only the models no later run had touched since — down to none for an old run, which reads as a broken page rather than as stale data. Reproduced: a 6-model run reported 6 relations, then a second run rebuilt one shared model and the first reported 5. A finished run already carries the answer. Its result lists every node with a status, and the graph carries each asset's dbt `unique_id`, so the two join directly — no path derivation, nothing stored twice, and nothing a later run can overwrite. The endpoint stays for the live window, where the result does not exist yet, and as the fallback for a run that never produced one (cancelled or killed, whose relations the worker settles in the table instead). `relationOutcome` mirrors the worker's `classify_status` so the colour drawn over a record agrees with the record: `warn`, `skipped` and `no-op` leave the relation untouched and stay uncoloured, as do tests and analyses, which match no asset. Verified in the browser on the run whose model had been stolen: all six relations green again, both sources correctly uncoloured. * docs(dbt): record why only dbt-core 1.x has live per-model progress `emits_node_events()` reads as "the Rust engines produce no node events", which is false and would close off the option. They produce exactly the same events; they put them on the console and ignore `--log-format-file json`, which both accept. Measured on 2.0.0-alpha.5 and fusion 2.0.0-preview.202: 15 node events each on stdout, 0 in the file log, for a three-model project. Taking them means owning the job log's presentation to work around a flag that is documented and simply unimplemented, so the note records the measurement, the sample event, and that flipping the predicate is the whole change once either engine honours it. * fix(dbt): give HighlightCode a dialect-agnostic sql language `npm run check` had three errors the fast check does not reach: `"sql"` is not a value `HighlightCode` accepts. Every SQL dialect it knows maps to one grammar, but a dbt model is compiled by whichever adapter the project targets, so naming a dialect would be a guess — `sql` is now a value in its own right. `langOf` was typed `string` and returned `markdown`, `python` and `text`, none of which the component accepts either, so a dbt project's YAML and Python files rendered unhighlighted. It now returns the component's own prop type, which is what caught them, and `undefined` for what has no grammar rather than a name that silently means the same thing. Verified in the project panel: SQL 22 tokens, YAML 27, where YAML was plain. * fix(dbt): stop failing no-op models, drop table triggers client-side, keep cross-selection edges The sweep that settles a run's stranded relations was marking `no-op`, `warn` and `skipped` models FAILED on successful runs: reconciliation reports those nodes without settling their record, so they were indistinguishable from a model the run never reached. It now excludes every relation the run accounted for, so only the genuinely abandoned ones are settled. `table` was removed from the backend's auto-trigger kinds but left in both client mirrors, so the editor and `pipeline dev`/`docs` kept drawing cascade arrows the deploy will not create. `isModuleEntryPoint` scanned for the first `__mod/`, the same bug its sibling just had: a `legacy__mod/script.ts` nested in a dbt project — dbt owns those names verbatim — read as that script's entry point. Both now anchor on the outermost boundary. A script selecting a model whose parent another script builds dropped the parent entirely, so no `dbt_edge` could reach it and the two relations sat on the graph unconnected. The parent is now kept as an endpoint and recorded as a READ, since this script does not build it — splitting a project across selections only composes if the seam still draws. * fix(dbt): don't double-run after-all tests, count only models a script builds An `after_all` run whose test phase failed saves a `run_results.json` holding tests alone. Retrying it reran exactly those tests — and then the test phase ran the whole suite again, appending a second copy of every result: duplicate ids in the run table, doubled totals. A retry whose saved results are tests alone IS the test phase, so the suite is not run after it, and the two phases now merge by node id rather than concatenating. Keeping a selection's unselected parents as nodes made them count toward the `×N` badge, whose tooltip says "materializes N models" — a script selecting one mart claimed the staging models upstream of it, and the number grew with the seam. The count now comes from the relations the script writes. That change also made the cross-selection read block dead, with a comment asserting the inverse of what now happens; it is removed, and the test that covered it still passes on the new arm. The test I added landed between a neighbouring test's comment and its `#[test]`, orphaning the attribute so that test stopped running. Two display fixes: the run page no longer shows a relation's SQL when the provenance belongs to another project that materializes the same relation, and the editor no longer draws an explicit `# on table://` arrow the deploy refuses. The starter descriptor no longer promises the removed cascade. * fix(dbt): clear untouched models, reject unknown descriptor fields A `no-op` model was left `running` forever on a successful run. The previous attempt at this stopped the sweep marking such models FAILED but gave them no terminal state instead, so they simply never settled. Reconciliation now returns what it settled and what the run reported but did not build, and the two get opposite treatment: a relation the run left untouched has its row DELETED, which is what the finished run's own result says about it (`relationOutcome` colours a `no-op` nothing), so the live and settled views agree; only a relation the run never reached at all is failed. The descriptor accepted unknown fields, so `selcet:` was ignored and left an empty selection — building the whole project — and a misspelled `target` fell back to the profile's default. It rejects them now. That immediately caught two of our own test fixtures still passing `repo:`, a field removed with the git path, which is exactly the class of mistake it exists to stop. `isDbtModulePath` matched `__dbt/` anywhere in a path, the third site with that bug: `foo__mod/vendor/x__dbt/a.ts` read as a dbt project file, and the push then looked for `foo.script.yaml` and could skip the edit. A verbatim dbt bundle dropped any file named `*.lock` before it reached the module map, so an authored `uv.lock` never deployed and the unmodified-project round trip quietly lost it. The exclusion now applies only to `__mod` bundles, where `.lock` really is the script's own lockfile — in the walker that hashes modules too, or a change to such a file would not register as one. Also: `langOf` fell back to `undefined`, which HighlightCode resolves to TypeScript rather than to no highlighting, so seeds and Markdown were coloured as code; and four comments still gave the removed cascade as the reason for sharing an asset node, which is now lineage. * fix(dbt): retry failed tests too, anchor the last __dbt path check `retry_failed_nodes` only ran after the model phase, which fails before the `after_all` test phase exists — so a project whose models built and whose tests failed got no retry at all, exempting exactly the failure mode that separate phase produces. The loop is now a function, called after both phases. `isDbtGeneratedPath` matched `__dbt/` anywhere, the fourth site with that bug: `foo__mod/vendor/x__dbt/target/a.ts` counted as generated dbt output, so `ignoreF` excluded an ordinary module file and a module-only edit never deployed its parent script. `wmill sync push` still dropped an ADDED or DELETED `.lock` three branches before the module arm, so the earlier fix only covered a first push: adding a `uv.lock` to a deployed project was reported as a change forever and never applied, and deleting one left it deployed. Editing worked, which is what made the round trip look whole. Also removes a duplicate `#[test]` that was double-registering a test and detaching its neighbour's comment, and rewrites seven comments that still gave the cascade as the reason for behaviour that now serves lineage only. * fix(dbt): bound the excluded-file read, keep the retry budget job-wide `isBundledModuleFile` read a file in full before deciding it was too big or binary, so a project sitting next to a multi-gigabyte parquet seed loaded the whole thing only to reject it. It now takes the size from `stat` and reads at most the 8 KB the NUL check needs: a 191 MB file is rejected in 0.0ms at 82 MB RSS. Calling the retry helper after both phases gave each its own `attempts` budget, so a job could spend double what the descriptor asked for — the bound exists because every attempt is a real dbt invocation holding a worker slot. The budget is now the job's, spent across whichever phases fail, and the field says so. Extracting that helper had also placed it between `#[allow(clippy:: too_many_arguments)]` and `run_dbt`, taking the attribute off the 12-argument function it was written for. * fix(dbt): actually spend the retry budget `retry_failed_nodes` looped on `while *remaining > 0` and never decremented it, so a failing job reissued `dbt retry` — logging "attempt 1 of 3" each time — until the job's deadline instead of `attempts` times. The decrement existed briefly and was lost when the function was re-extracted by hand. Claiming and counting are now one operation, `claim_attempt`, because keeping them apart is exactly how the bound goes missing: the loop cannot iterate without spending the budget. Its test is bounded by its own `for` rather than by the function under test. An earlier version collected `std::iter::from_fn(|| claim_attempt(..))`, which against a non-spending `claim_attempt` is an infinite iterator — it allocated until the machine died. A test for a loop bound must fail an assertion when the bound regresses, not consume the host: it now reports `[1, 1, 1, …]` against `[1, 2, 3]` in 0.00s. * fix(dbt): ask before reading, not after Bounding `isBundledModuleFile` did nothing for the bundle builder, which read the whole file into memory and only then asked whether to keep it — so a multi-gigabyte seed beside a project was still loaded in full just to be skipped. The predicate is now consulted first, and the read happens only for files the bundle actually carries. * feat(dbt): animate the ref() edges feeding the model being built The nodes moved during a run but the edges did not, so the graph showed where dbt had got to without showing it flowing there. Reuses the canvas's existing rule rather than adding a second one: an edge animates when it touches what is happening. For a pipeline that is the running script; for dbt the unit of work is the model, so a `ref()` edge animates while its target builds. Same `animated` field, same visual language, no new styling. Verified mid-run on a 7-model project: of six `ref()` edges only the two feeding the model then building were animated, and none once the job finished. * feat(dbt): show what each model wrote, and say when its SQL is another project's Three things a reader wanted from the run graph and could not get. Row counts: the worker already records one per relation and `run_progress` already returned it, but the graph used only `status` and dropped the number. A model that built green having emitted zero rows is the failure that looks like a success, so the count is on the node. The relation's fully-qualified name, copyable: there is no table browser to open, so the next best affordance is the exact identifier to paste into a SQL client. It is parsed with `splitRelation`, which honours quoting the way the worker's `split_relation` does — splitting on every period renders `"wh"."analytics.v2"."orders"` as a relation `orders` in a schema `v2`, which does not exist. And when two projects materialize one relation, the graph keeps a single provenance winner, so the losing project's node carries the other's model. The SQL was already suppressed there — correctly, it is not this run's code — but silently, which reads as a dead click. It now says so. * fix(dbt): a finished run's graph is the models it built, not today's project `/assets/graph` is the current deploy, so an old run's graph drifted with the project: a model added after it appeared as though the run had built it, and the older the run the wronger the picture. A finished run's node set now comes from its own result, which named exactly what it touched. Sources survive the filter regardless — dbt never lists them in `run_results.json` because it does not build them, but they are the upstream the run read, and dropping them would leave the models hanging. The graph is still the current deploy's, so a model renamed or deleted since cannot be drawn at all. Rather than a silently shorter graph, the count is stated above it. Verified by adding a model after a run: the old run renders 7 models without it, a fresh run renders 8 with it. * feat(dbt): preview a model's rows with `dbt show` There was no way to see the data behind a node — only its SQL and its row count. `dbt show` selects from a model and returns rows, and every engine ships it, so the preview needs no adapter code of ours: no connection path, no dialect-correct quoting, no type coercion for ten warehouses. It runs against the profile the run already renders. It is a `dbt_command` rather than a new endpoint, so it inherits the whole job path — authorization, isolation, cancellation, logs, engine provisioning — and `limit` joins the run form beside it. That the allowlist can admit it at all is a consequence of dropping the cascade: while a successful job dispatched its deploy-time write set, a command that wrote nothing woke every consumer for relations nothing had touched. Read-only, and treated as such: no graph republish, no materialization records, no retry state, no test phase. Captured rather than streamed, like `dbt ls` — these rows are the result, not commentary, and the job-log writer is what `NO_LOGS_AT_ALL` discards. Verified: `{"dbt_command":"show","select":["stg_customers"],"limit":3}` returns three rows; a preview leaves `materialized_partition` untouched (62 → 62, 0 rows for the job); `clean` is still refused by the allowlist. * feat(dbt): preview a model's rows from the graph, and keep our locks out of dbt projects The run page could show a model's SQL and how many rows it wrote, but not the data. Selecting a model now offers "Preview rows", which runs the script with `dbt_command: show` and renders the result as a table. Explicit rather than on-select: a preview is a job, so it costs a worker slot and the engine's start-up, and previewing on every click would spend both on mere navigation. Sources are excluded — dbt shows what a model SELECTs, and a source is not one. Also: `updateModuleLocks` was the one module helper that never learned about verbatim bundles, so it walked a dbt project writing `foo.lock` beside `foo.sql`. None of those files is a Windmill script needing a lockfile, and the bundle promises to round-trip the project byte-for-byte — our artifacts have no business in it. Verified in the browser: selecting `stg_customers` and previewing returns the columns `id`/`src` and five rows from the warehouse. * fix(dbt): keep a run's models when another project owns their provenance Scoping a finished run's graph to the ids it named dropped relations whose provenance winner belongs to a different project — so a run of a project sharing a schema showed 3 of the 6 models it had built. An id that was never this run's package cannot be judged against its result, so it is kept: the relation IS one the run wrote, and hiding it understates the run. The same rule applies to the "no longer in the project" count, which otherwise reported deletions that were only provenance collisions. Previews are now cached per model and survive the selection moving. One was thrown away whenever the reader clicked elsewhere, which for a job costing a worker slot and an engine start-up meant re-running it to see it again — and the run continues in the background, so leaving and returning finds the rows there. The spinner also never span: `startIcon` takes the icon and its classes separately, so the animation has to be passed alongside. How long it took is shown with the rows. A preview is a job, and its cost should not be something the reader has to guess at. * fix(dbt): resolve argument references, clamp the show limit, flag renamed relations `handle_dbt_job` cloned `job.args` where every other executor calls `build_args_map`, so a `$var:` / `$res:` / `$encrypted:` argument reached dbt as the literal string. A placeholder holding a schema or an `enabled` flag would then build a different slice of the project than the caller asked for. `--limit` took any positive i64, and the worker buffers the whole of dbt's stdout to read the rows out of it — so a caller with only run permission could make it hold an unbounded allocation. It is clamped to a ceiling now, extracted as `show_limit` so the bound is pinned by a test rather than inline in an async function nothing can reach. And a model keeps its id when its alias or schema changes, so an old run's node showed today's relation while the run wrote another — the page asserting it had materialized a table that did not exist yet. The run's result carries the relation each node actually wrote, so the drift is detectable without a graph snapshot, and the count is stated above the graph. Rendering the run's own lineage still needs a per-job snapshot; this stops the page claiming otherwise. * fix(dbt): stop persisting resolved secrets, bound the preview by bytes Resolving `$var:` / `$res:` / `$encrypted:` for dbt — added in the previous commit — meant `save_run_state` wrote the resolved PLAINTEXT into `dbt_run_state.args` and the worker's `state.json`. The row outlives the job, so a secret stayed in the database and a later `dbt_command: retry` replayed it after the grant was revoked or the value rotated. The invocation now carries the args as submitted alongside the resolved ones, run state persists those, and the restore path resolves them again under whoever is retrying. Clamping `--limit` bounded the row COUNT, not the size: one column can hold a megabyte, so a thousand rows is a thousand megabytes, and `run_capturing` buffers all of it. The captured output has a byte ceiling now. `limit` became a built-in argument without joining `RESERVED_ARG_NAMES`, so a descriptor writing `{{ limit }}` was silently handed the preview control's default instead of being told the name is taken. Two display fixes: the relation-drift banner compared a canonicalized (lower case) asset path against the warehouse's own spelling, so it fired on every model of every finished Snowflake run; and caching a preview's failure left `Preview rows` dead for that model until reload. * feat(dbt): key the graph by script version so a run renders its own project The dbt graph was keyed by path alone, so a deploy overwrote the only copy and a run page could only ever show today's project — an older run rendered today's models, SQL and `ref()` lineage no matter what it had run. My previous attempt filtered that view to the ids the run named, which stopped it lying but could not show what was gone: the data no longer existed. `dbt_node` / `dbt_edge` now carry `script_hash` in their primary key, so each deployed version keeps its own graph, and the run page passes the version its job recorded. Per DEPLOY, not per run — ten thousand runs of one version share one graph — and a composite FK to `script (workspace_id, hash)` with ON DELETE CASCADE means a version's graph dies with it. Nothing pruned these before, because there was one copy per path; they would otherwise have accumulated with no sweep. Two deploys of one path now write disjoint rows, so the graph can no longer be lost to a race. `claim_graph_publication` remains only for what is still path-keyed — the `asset` usage rows — and an older deploy finishing late records its own graph before declining to touch those, where before it published nothing at all. A pinned request is scoped by the version's own nodes rather than by `asset`: that table describes the current deploy, so scoping through it would filter a model out of the very run that built it. Verified end to end: deployed v1 (8 models), ran it, deployed v2 with four models removed and one rewritten. The old run renders 8 models, 6 ref() edges and v1's SQL; a new run renders 4 and the v2 rewrite. * fix(dbt): scope graph cleanup to one version, bound the preview capture Archive and delete both act on a single `hash`, but the graph cleanup they called deleted every row for the path. Now that the graph is keyed per version, archiving an old version erased the live one's models, SQL and lineage, and nothing repaired it. Both callers have the path in hand, so the by-hash wrapper is gone and they use the version-scoped clear directly. `dbt show` checked its 8 MB ceiling after `wait_with_output` had already buffered everything, so the ceiling could not bound what the worker held. `run_capturing` now reads both pipes incrementally against a caller-supplied limit and kills the child on overflow. The read buffers are heap-allocated: as arrays they were baked into the future, which the job poller boxes several layers deep, and that overflowed the worker thread's stack — a `dbt show` run aborted the whole worker process. A retry's `dbt parse` ran on the arguments as submitted while the build ran on resolved ones, so a `$var:` shaping the graph parsed verbatim. The parse moves to the caller, after resolution. A run that names its own `select`/`exclude` now drops the descriptor's `selector`: dbt resolves `--selector` instead of `--select`, so passing both made a preview of one model return another's rows. Also: log instead of silently swallowing a `modules` column that fails to deserialize (pre-existing, but for dbt it means running with no project at all); keep the model SQL reachable once a preview has landed; render which node the rows came from; stringify object-valued cells; document `dbt_script_hash` in the OpenAPI spec. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * docs: record the per-worktree dev environment and the backend-run check Three mistakes this guidance would have prevented, each of which cost a cycle: A worktree has its own database and ports, but AGENTS.md stated the single-checkout defaults as facts. Pointing `DATABASE_URL` at another worktree's database makes `cargo sqlx prepare` fail on every query touching a table your migrations added — and it deletes `.sqlx/` before it fails, so the cache is gutted rather than merely stale. Starting a backend on the wrong port leaves the UI up with every call 502ing, which reads as an application bug. Both values are now discoverable with commands that work as written. `prepare` is also documented as the wrong tool for a removal-only change: the cache is already complete for CI, and the only residue is orphaned entries that can be found by text-matching against the sources without a database. Nothing told a reader that `cargo check` does not exercise a worker path. A read buffer declared as an array inside an async block is baked into the future, and once boxed by the job poller it overflows the worker thread's stack — compiling and unit-testing clean while aborting the whole worker process at runtime. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * fix(dbt): scope the remaining path-wide reads and clears to one version Four places still spoke for a whole path after the graph became per-version: The relation-root drift check read `dbt_node` by path with an unordered `LIMIT 1`, so with v1 at root A and v2 at root B it could answer with v1's row, suppress the refresh v2 needed, and leave v2's graph naming relations the run does not build. It now reads this job's version. `dbt_dep`'s no-resource branch cleared the path, so a descriptor edited to bring its own `profiles.yml` emptied every earlier version's graph and with it every finished run's page. The ownership being given up is the path-keyed `asset` usages cleared beside it; the graph clear is now this version's. The graph was inserted before the publication claim checked the version was still live. Archive and delete only soft-update `script`, so the foreign key still accepted an in-flight dependency job's rows and the failed claim committed them — and because pinned queries deliberately serve archived versions, deleted model SQL became readable again. The write is now gated on a `FOR UPDATE` liveness check. `clear_dbt_run_state_by_script_hash` resolved a hash to a path and deleted the path's saved run. `dbt_run_state` is keyed by path by design — one saved run per script — so archiving one version discarded the live version's resumable failure. It clears only once no live version of the path is left; `identity` already refuses a resume whose project, warehouse or engine moved. "Preview rows" ran `runScriptByPath` while the SQL beside it was pinned to a hash, so an old run showed its own SQL over today's rows. Verified end to end: with v3 deployed, the v2 run's preview runs v2's hash and returns v2's rows. Also: keep the TAIL of a captured stderr, since dbt prints its summary last; one `$derived` for the parsed result rather than five; collapse three near-identical argument accessors onto one generic; fold the single-use `copy_dir_command` into its caller; and give `parseDbtRun.ts` one status classifier instead of spelling dbt's failure vocabulary twice. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * refactor(dbt): one JobCtx down the executor, one table per adapter Two changes aimed at the operations this code will keep having: adding a phase, and adding a warehouse. `JobCtx` already bundled the five values every phase needs, and nine functions took it — but the top of the executor threaded the fields apart and rebuilt the struct at each call, so the same literal appeared eight times and each new phase meant five more parameters. It is now built once per entry point and reborrowed. `prepare_project` goes from 21 parameters to 17, `retry_failed_nodes` from 15 to 11, and `run_dbt` drops below the lint threshold. The two remaining constructions are the worker boundary, where the pieces genuinely arrive apart. `DbtAdapter` answered five questions with five parallel matches over the same eleven variants, plus a sixth list of adapters kept by hand in a test. The facts now live in one `AdapterSpec` per adapter, reached through one exhaustive match, so adding a warehouse states its name, driver, package, port, database key and licensing together and the compiler demands the arm. Each arm spreads from a Postgres base, which makes the inheritance visible per adapter instead of hidden in the `_ =>` defaults `default_port` and `database_key` used to carry. `DbtAdapter::ALL` replaces the list the test kept separately. Verified by dumping all seven facts for all eleven adapters before and after: byte-identical. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * fix(dbt): job-keyed run progress, and stop path-wide reads and clears Six findings from the last round, in the order they bite. The publication liveness gate refused on `archived`, but `create_script` archives the parent on every redeploy — so deploying v2 while v1's dependency job was still parsing left v1 without a graph, permanently, which is the exact case the unconditional write existed to serve. It gates on `deleted` alone now; an explicit archive is still covered by the `FOR UPDATE` ordering. A project-owned `profiles.yml` trusted `profile.type` instead of reading the file. The Rust engines carry every adapter, so a CE script could declare `postgres` over a target that is `sqlserver` and have dbt connect with the enterprise adapter. The file is read whichever way, and a descriptor that disagrees with it is refused. Renaming a dbt script, or editing one so its newest version is no longer dbt, cleared the graph for the whole path — every older version's models, SQL and lineage, which their own finished runs still render. Neither needs it: graph queries join on `(path, hash)` through a `language = 'dbt'` CTE, so an old version's rows cannot attach to whatever lives at that path next. Live progress read `materialized_partition`, whose key is the relation and whose `job_id` is only the last writer. Two runs of one project took rows from each other. Progress now has its own job-keyed table; the relation table is untouched, because one row per relation is right for the pipeline canvas and fork defer. Verified with two overlapping builds: both keep 6 rows in the new table, while the old one attributes 6 to one run and 0 to the other. `Scratch::drop` removed a half-installed virtualenv synchronously from inside the job future, blocking a runtime thread; it goes to `spawn_blocking`, with a direct call when there is no runtime to hand it to. The E2E list asked for a `# on table://` subscription the deploy now refuses. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * feat(dbt): snapshot a dynamic descriptor's graph per run A `{{ }}` placeholder in `vars` can enable a different set of models per run, so those runs re-ingest the graph. Keyed by version alone, each re-ingest overwrote the last: reopening an older run showed the newer run's project, and a model only the older run built was gone entirely — no SQL, no lineage, and nothing the saved result could colour, since it can only tint nodes that are there. `dbt_node` / `dbt_edge` gain `job_id`. A run of a dynamic descriptor writes its own snapshot under its job id and its page reads it back; a static descriptor writes the version's graph once, under a zero-UUID sentinel, and every run of it reads that. The sentinel is a value rather than NULL because `job_id` is part of the primary key and Postgres does not treat two NULLs as the same key, so each re-ingest would add a row set instead of replacing one. `/assets/graph` takes `dbt_job_id` and prefers a snapshot when one exists, falling back to the version's graph otherwise — so a run page passes it unconditionally and static descriptors are unaffected. Snapshots age out after 30 days, pruned by the runs that write them, so no background sweep has to learn about these tables. Verified end to end: one deploy, two runs of it with `extra=yes` and `extra=no` gating a model's `enabled`. The version's graph holds 6 models, run 1's snapshot 7 including `opt_extra`, run 2's 6 without it; the endpoint returns each run's own and falls back to the version's when the parameter is omitted. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * perf(dbt): only snapshot a run whose graph differs, and prune from every run Two costs the per-run snapshot carried, both found by measuring it rather than by reading it. A snapshot was written for every run of a dynamic descriptor, but marking one dynamic is conservative: `graph_is_per_run` is true whenever `vars` holds a `{{ }}` placeholder or `env` holds a `$var:`, which says the arguments reach dbt and not that they change which models exist. The usual case is a date var, whose graph is identical run after run, so the table filled with copies of an unchanging picture — around 1 KB per model per run, which is a gigabyte or so a month for a 200-model project on an hourly schedule. A row set now carries a digest of its nodes, edges and relation root, and a run whose digest matches the version's writes nothing; the read already falls back to the version's graph, so those pages are unchanged. Only a run whose model set really differs pays. The prune was hung off the progress reporter, which exists only for engines that emit node events — so a Fusion or dbt-core-2x instance accumulated snapshots and never deleted any. Retention that stops working because of an engine choice is not retention; it runs detached from every dbt run instead. Verified against a descriptor with a var-gated model: the version's graph holds 8 rows, a run that resolves to that same graph stores none at all, and a run that enables the extra model stores its own 9. Both pages still render their own project — 6 assets without the extra model, 7 with it. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * fix(dbt): scope every dbt_node join to the chosen snapshot `job_id` joined the key, but only the scoping CTE and `dbt_edge` were taught to filter on it. The outer node SELECT and the parent/child joins in the edge query were not, so each model came back once per retained snapshot plus once for the version's graph, and each edge matched every combination of the two — the model count multiplied and the edge join fanned out quadratically. Measured against one stored snapshot: 17 node rows where 8 are wanted, and 28 edge pairs where 7 are. The response dedup hid the edge blow-up from the payload, not from the plan, and the run page refetches the graph every two seconds. The progress table gained writers it was missing. `terminalize_running_relations` settled only the relation-keyed table, so a cancelled or killed run — the case that function exists for, since it leaves no `run_results.json` — showed every in-flight model still spinning on the run page for as long as the row lived. An agent worker cannot write the new table at all, having no database of its own, so the read falls back to the relation-keyed one when a job has no rows there. Also: a wrapped string literal missing its backslash put eighteen spaces in the middle of the profile-disagreement error; a comment still described concurrent runs of one dynamic version overwriting each other's graph, which is what keying by job removed; and the `materialized_partition` index justified itself by a run-page poll that has since moved to another table, though the closing sweep still earns it. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * fix(dbt): give a graph snapshot a marker row, and scope what reads it Five findings, four of which are the same mistake in different places: a snapshot's identity was inferred from its contents. Existence was inferred from a `dbt_node` row, so a dynamic run that disabled every model — a legitimately empty graph — read as "no snapshot" and its page showed the deployed models instead. The digest was a column repeated on every node and read back with a `LIMIT 1` carrying no `job_id`, so a run could compare itself against another run's digest and suppress a snapshot it needed. The relation-root drift check read the same rows unscoped, so after a drift it could find a previous run's root and conclude nothing had moved. `dbt_graph_snapshot` holds one row per stored graph — path, version, job, digest, timestamp. Existence is that row, the digest lives there once, the drift check reads the deployed row explicitly, and the retention sweep deletes markers first and then the rows no marker stands for. The digest is SHA-256 rather than `DefaultHasher`, whose output is documented as unstable across Rust releases: this value outlives the process that computed it, so a toolchain bump would have silently stopped every comparison matching and quietly reinstated the duplicate snapshots the digest exists to prevent. `/run_progress` ignored the view token, so a share-link viewer got the graph and was refused the progress that colours it. A preview sent only its own three arguments, so a descriptor with a required `{{ }}` var could not be previewed at all and an overridden one previewed a different relation than the page was showing. The run's arguments go first now, with the preview's three overriding. Verified on a project whose only model is var-gated: the deploy stores a marker with zero nodes, a run with the var set stores a marker with one, and the endpoint answers 0 and 1 respectively rather than showing the deployed models for both. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * refactor(dbt): squash the runtime's migrations into one Ten migrations reshaping the same three tables is a history no installation ever had. `dbt_node` gained `script_hash`, then `job_id`, then `ingested_at`, with its primary key rebuilt twice; `graph_digest` was added by one migration and dropped by the next after the digest moved to its own table. On a fresh database all of that replays to arrive at a shape the schema can simply state, and this feature has never shipped, so there is no upgrade path to preserve. One migration now creates `dbt_node`, `dbt_edge`, `dbt_graph_snapshot`, `dbt_run_state` and `dbt_run_progress` in their final shape, carrying forward the rationale each of the replaced migrations recorded. The enum additions stay in `add_dbt_lang`, since a value cannot be added and used in one transaction, and the `materialized_partition` index stays separate because it belongs to a table this feature did not introduce. Verified by rebuilding: dropped the five tables, replayed from the single migration, and confirmed the result is identical — same primary keys, the same two composite `script` foreign keys, the same seven indexes. Every `sqlx::query!` in the workspace then compiled against it, which checks each column's name, type and nullability, and a deploy plus run on the rebuilt schema produced 8 nodes, 7 edges, a snapshot marker and 6 progress rows. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * fix(dbt): authorize snapshot reads, bound the prune, give the marker a lifecycle `dbt_job_id` is caller-supplied and selected straight from `dbt_graph_snapshot`, which carries no RLS — so a caller who could see the script could read any run's model set and relation paths, which a dynamic alias or schema can encode. Both graph queries now require the job itself to be visible, in the authed transaction, the same gate `raw_code` already applies to the script that produced it. The drift check compared against the deployed graph alone, which misses the way back: a run at root B republishes the path-keyed `asset` usages at B, and returning the profile to A then matches the deploy and skips the refresh, leaving those usages at B while dbt builds A. It reads the most recent ingest for the version instead — the one that last wrote them — ordered rather than an arbitrary `LIMIT 1`. The prune anti-joined every non-deployed node and edge with no age predicate, so each run scanned the whole retained sidecar and concurrent runs duplicated it. All three deletes share one age bound again, with the sentinel spelled as a literal so the partial indexes apply — a bound parameter cannot be proven to match the index predicate. `dbt_graph_snapshot` was the one dbt table nothing in the script lifecycle deleted: no `script` foreign key and absent from both `clear_dbt_manifest*` sites. A marker outliving its rows is read as a snapshot with no nodes, and its digest still answers the suppression check, so an identical run would write nothing and then render an empty graph. It cascades like the rows now and both clears take it. Also: the preview cleared `exclude` rather than inheriting it, since previewing a model the run excluded reached dbt as `--select m --exclude m`; and `terminalize_running_relations` no longer claims to cover a killed worker, which never reaches it. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * fix(dbt): quote profile names, record where usages were published, stop polling the graph A profile name comes from the project's own `dbt_project.yml` and a target from the descriptor, and both were interpolated into `profiles.yml` as bare YAML — including as mapping keys. A name like `prod # hidden` truncates the mapping and a newline opens a sibling key of the author's choosing. Both are rendered as quoted scalars now, as are the BigQuery keyfile's keys, with a test that asserts the document still parses to exactly the keys we wrote. The drift check read the most recent ingest, which latches: a run that returns to the deployed root re-ingests but stores no snapshot (its digest matches the version's), so the moved run's rows stay newest and every later run pays an extra parse and ingest. The publisher now records the root it published the path-keyed usages at, which is the only thing that answers "where do the current usages point" — the deploy's own root goes stale as soon as a run republishes. The run page polled `/assets/graph` every two seconds alongside progress, so it re-sent every node's SQL for the length of a run — hundreds of KB a tick on a real project, for a graph that a dynamic descriptor re-ingests exactly once before the build. It fetches once more shortly after mount and then polls progress alone. Node results carry `outcome` beside `status`. `status` stays dbt's own word, but dbt owns that vocabulary — 1.x and 2.x differ on casing and `no-op` arrived in a minor release — so publishing only it would force a break or a lie the first time it moves. `outcome` is the stable half a downstream script branches on. Also: the worker's dbt entry points are `pub(crate)`, since nothing outside the crate calls them and they resolve secrets and launch processes; and the snapshot gate records that it is RLS-only where `/jobs/run_progress` also honours a share-link token, which is a gap in what a shared page shows rather than in what it protects. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * test(dbt): pin the graph storage invariants against a real database Every defect review found in this area was DB-shaped — which row set a read resolves to, which rows a clear takes, whether a snapshot exists at all — and none of it is reachable from a unit test on a pure function. Four rounds established these answers and nothing guarded them, which is why each round kept finding another. Six cases, on the harness the repo already uses for schema-shaped behaviour: an identical run stores no snapshot and leaves no marker; a differing run keeps its own while the version's is untouched; an empty run graph is still a snapshot rather than an absent one; clearing one version leaves the others whole; the path-wide clear takes the markers with it; and the sweep ages out run snapshots while never touching a version's own graph. `IngestedNode` gains `Default` so a test can state the two fields a case is about rather than the eighteen it is not. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * perf(dbt): bound a script's stored graphs by deploy count Run snapshots expire on a clock, but a VERSION's graph could not: its reader is every finished run of that version, and a run page is as old as its job. So nothing reclaimed them — a deploy graph went only when its `script` row was hard deleted, which Windmill does not routinely do. A CI deploying on every commit added a full model set with SQL bodies per commit, forever: roughly 200 KB a deploy for a 200-model project, which is gigabytes a year across an instance. Bounded by COUNT instead of age, since age is the thing that cannot be right here. The newest 50 deploys per path keep their graph and older ones are reclaimed, making growth `versions x models` rather than unbounded in time. Generous on purpose: reaching the bound empties that version's run pages, so it exists to stop unbounded growth rather than to be hit in normal use. Ordered by the script's own `created_at`, so a late-finishing job re-ingesting an old version cannot promote it. Pinned by a test that deploys past the bound and asserts both halves: the count holds, and the newest version is always among the survivors. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * fix(dbt): let the run page know when its snapshot has landed The one-shot graph refetch was wrong: a dynamic descriptor's ingest happens before the build but after cloning, dependency install and parse, so a fixed delay either fires too early — and the run page then shows the deployed models for the whole run, never that run's own — or keeps re-sending the whole graph for the length of it. Neither is a timing problem to tune; the page had no way to tell "the snapshot is not written yet" from "this run has none". `/assets/graph` answers that directly: `dbt_snapshot_job` is the job the dbt half resolved from, when one was asked for and found. The page polls the graph until that is its own job, and stops. A static descriptor never snapshots, so an attempt cap ends it there rather than polling for the run's duration. `dbt_node.relation_root` is gone. The drift check moved to the marker's `published_relation_root`, which left the column written on every node and read by nothing. `outcome` was published as the stable half of the result contract, but the in-tree consumer still ranked and coloured from dbt's own word — so the field existed and nothing used it. `statusRank` takes it, `DbtRunResult` passes it, and `classifyStatus` is documented as the fallback for results that predate it and for the live event stream, which carries dbt's word alone. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * docs(dbt): record what a share-link viewer actually sees The comment at the snapshot gate said the graph "falls back to the deployed set", which is only the rarer half of it. A share link is an extra grant for a logged-in user who lacks access to the job, so the usual case is no read on the script either — and then the `live` CTE matches nothing and the whole dbt half comes back empty. A blank Models panel over working progress rows, not a fallback. `docs/dbt-runtime.md` now carries the analysis a follow-up needs: that relaxing this leaks nothing, because `v2_job_completed.result` already gives that viewer every node's `unique_id` and `relation_name` — the graph's only incremental exposure is `raw_code`, which is gated separately on seeing the script. And the shape of the fix: `OptViewToken` and `validate_view_token` are self-contained enough to move into `windmill-api-auth`, which `windmill-api-assets` already depends on, after which the gate can honour a token for that job's snapshot alone while `raw_code` stays where it is. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * fix(dbt): keep model SQL behind the scripts:read scope, and unbreak CI `/assets/graph` is authorized as `assets:read`, and RLS decides whether the caller can see the script that produced a node — but RLS is not a scoped token's grants. A token deliberately narrowed to `assets:read` could therefore read model source and repository paths for scripts outside its `scripts:read` paths. The same `build_scope_path_predicate` the macro endpoint already applies now gates `raw_code` and `original_file_path`; the relation's shape is unaffected, only its body is withheld. `DbtAdapter::ALL` exists for the tests that must cover every adapter, so it is dead in a release build and `-D warnings` failed all four backend checks on it. It is `#[cfg(test)]` now. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * fix(dbt): stop the graph poll at the ingest, and type `limit` in the schema The poll's stop condition was a snapshot appearing, with a 40-attempt cap behind it — so a STATIC descriptor, which never snapshots, took the cap every time and re-fetched the whole graph forty times. That is most of what removing the poll was meant to save, and static is the common case. The ingest runs BEFORE the build, so the first model to report progress proves it has already happened: a snapshot absent by then is one this run never writes. Progress arriving is now the second exit, and the cap is only a backstop for a run that reports none at all. `limit` is declared `Typ::Int` but `dbt_arg_schema` had no integer arm, so the run form and the generated clients saw an untyped default and offered no numeric control for a value the worker clamps. Covered by the schema test. `relationOutcome` still re-derived from dbt's word while `statusRank` had moved to `outcome`; both read it now. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * fix(dbt): make a retry prove its arguments still resolve the same The saved arguments are the ones SUBMITTED, so a `$var:` in them is re-resolved on retry. The identity did not cover the resolved values, so a variable that changed between the failed run and the retry was accepted — and which graph the retry then used depended on WHERE it landed: a worker holding the local snapshot replays the saved manifest, while a database restore reparses with the new value. Placement decided whether the resumed failures described the relations being built. The identity gains a digest of the resolved arguments, and is compared in two halves because resolution happens between them. Project, warehouse, engine and env are checkable up front; the arguments are not, because a retry request carries only `dbt_command` and the ones to compare are the SAVED arguments after this caller has re-resolved them. Comparing the whole string up front would have refused every retry — which is what the obvious version of this fix does. A row written before the digest existed has no last segment, and still restores rather than being refused. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * fix(dbt): keep pre-upgrade retries working, and stop losing a late snapshot Splitting the identity on its last `|` read a pre-upgrade row's env digest as an arguments digest and left only `<run_identity>` as the prefix, so every saved failure on an upgraded instance became unretryable — a regression the previous commit's own test missed by using an identity with no `|` in it at all, which is not what an old one looks like. The digest is tagged (`|args=`) rather than positional, and the test now uses a real pre-upgrade identity. The graph poll gave up after a bounded number of tries, but provisioning and `dbt deps` precede the ingest and can outlast that on a cold worker — and the engines that emit no node events never produce the progress that ends it early. A finished run now reloads the graph unconditionally, and the poll's own exit issues one last load: progress proves the ingest happened, not that the previous tick saw it, and dbt's compile window is wider than one tick. A `dbt retry` restores the failed run's arguments inside the worker and they are never written back to the retry job, whose own args are just `{"dbt_command": "retry"}` — so previewing a row on a retry's page ran without the vars the run used. The result now carries the invocation's arguments as SUBMITTED, so a `$var:` stays a reference and no resolved value is published. The deploy-count sweep ran instance-wide on every dbt run: `FROM script WHERE language = 'dbt'` has no index to stand on, and both orphan deletes are the complement of every partial index here. It is scoped to the running script's `(workspace_id, path)` — which `index_script_on_path_created_at` serves — and the orphan deletes only run when a marker actually went. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * fix(dbt): hide dbt from module-less pickers, and stabilise the retry digests `processLangs` feeds every language picker, including flow steps and app inline scripts. Those are raw bodies with nowhere to carry a module bundle, and a dbt script IS its bundle — so choosing dbt there produced a job that could only fail once the worker looked for `dbt_project.yml`. Those two surfaces use `processInlineLangs`, which drops the languages that need modules; a flow still reaches dbt the way it reaches any script, by path to a deployed one. `graph_digest` moved to SHA-256 because it is persisted and compared by a later worker, and `DefaultHasher` is documented as unstable across Rust releases — but the retry identity's own digests were left on it, and they are persisted in `dbt_run_state.identity` for exactly the same comparison. A toolchain bump would have refused every saved failure as a different project. All three go through one `stable_digest`, length-prefixed so no split of the same bytes collides. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * fix(dbt): enforce the tag scope on snapshot reads, reset state between runs A tag scope is an orthogonal hard restriction: a token limited to some tags must not read a job outside them however else it is authorized. The snapshot lookup went through `v2_job` RLS alone, which knows nothing about tags, so such a token could still retrieve a run's model set and its dynamic relation paths. The same predicate `require_job_read_access` applies for the progress half of the page is applied here — `get_scope_tags` is already public in `windmill-api-auth`, and it is `None` for an unscoped caller, so a normal session pays nothing. SvelteKit reuses the run graph between run ids, and `graphTries`, `polled` and `raw` all describe the previous job: a spent retry count stopped the next run's snapshot poll before it began, and stale progress coloured its models with another run's statuses. All three reset when the graph key changes. Also a wrapped string literal missing its backslashes, which put two ~22-space runs in the middle of the retry-refusal message. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * docs(dbt): put each digest helper's rationale on its own function Inserting `stable_digest` above `split_identity` split that function's doc, so five lines describing where the identity divides ended up introducing the hasher. Each is back on the function it describes, stated once. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * fix(dbt): read the run-pinned graph through the job, not the asset graph Pinning the asset graph to one run is job-scoped data, but `dbt_job_id` sat on `/assets/graph`, authorized as `assets:read`. The job-read contract — `require_job_read_access` — is five parts that pull in opposite directions (tag scope restrictive, `created_by` permissive, app-embed restrictive-overriding, view token permissive, RLS underneath), so plain RLS is neither a stricter nor a looser approximation of it. Restating the parts near the graph query kept leaving one out: first the job check entirely, then the share-link asymmetry, then the tag scope, and the app-embed restriction was still missing and failing open. The helper cannot be called from `windmill-api-assets`, because `windmill-api` depends on that crate. So move the read instead of the check: the run-pinned graph is now `GET /w/{w_id}/jobs/dbt_graph/{id}` in `windmill-api`, on the same gate as the `run_progress` it colours, and `/assets/graph` has no `dbt_job_id` parameter at all. - `asset_graph_for` takes the job as an argument from an already-authorized caller; the route handler passes `None`. - Extract the graph response into a `AssetGraph` component schema, now that two paths return it. - The run page fetches the job route when it has a job id. * fix(dbt): charge assets:read on the run-graph route, trust the job gate in SQL Round 13 findings on the route moved last commit. The scope domain comes from the URL segment, so putting the read under `/jobs` asked a scoped token for `jobs:read` alone while returning asset-graph data that `/assets/graph` charges `assets:read` for. A token narrowed to polling run status could read workspace topology, and the missing-job fallback made it cheaper still — any random UUID skipped the job gate. Both scopes are now required: the job gate reaches this run, `assets:read` reaches asset data at all. The `chosen` CTE re-decided job visibility under plain RLS after the caller had already passed `require_job_read_access`. It could only disagree, and did so silently by falling back to the deployed graph — a share-link viewer entitled to the run was shown a different run's model set. Dropped; the contract is that a job reaching `asset_graph_for` is already authorized. Also: the flow editor's `+` insert menu still offered dbt (the third `processLangs` caller, missed when the other two moved to `processInlineLangs`), the docs still described the deleted `dbt_job_id` parameter, and the new handler had again been inserted between `get_run_progress`'s doc comment and its function. * fix(dbt): resolve a pinned run's version from the job row, not script RLS Local codex review of the branch. A share-link viewer is entitled to the run and usually has no grant on the project — that is what the link works around. The graph's `live` CTE resolved the version by selecting `script` inside the viewer's RLS transaction, so it answered for their access to the project rather than for the run they were given: the Models panel came back blank beneath working progress rows. A pinned run now takes its path and hash from the job row the handler already read after authorizing the job, so `live` does not consult `script` at all. `raw_code` keeps its own `EXISTS` against `script`, so the model bodies stay behind access to the project. Verified under RLS as an unprivileged role: the shape query goes 0 rows -> 1, the `raw_code` gate stays 0. Taking the version from the job also means a caller can no longer pin one project's version while naming another's run, since `dbt_script_hash` is ignored when a job is given. The run page's graph fetch is a raw `fetch`, which bypasses the interceptor that adds `X-View-Token` to generated-client calls, so a shared page was refused before any of this mattered; it goes through `appendViewToken` now. Also trims three comments to the AGENTS.md limit, dropping drafting-history rationale that belongs in docs/dbt-runtime.md. * fix(dbt): snapshot vars-overridden runs, carry the pinned version everywhere Second local codex pass. A `vars` run argument overrides the descriptor's, and vars drive `enabled`, alias, schema, database and materialization — so such a run builds relations the deployed graph does not describe. It now snapshots under its own job id, which per-job keying makes safe: the version's graph stays for runs that did not override. The old comment claimed gating on it would strand the override's graph for the next default run, which was true only when the write went to the deployed slot. Two sites still read the caller's `dbt_script_hash` instead of the version resolved from the job, so `/jobs/dbt_graph/{id}` without that redundant parameter dropped models the run's version had and a later deploy removed. The `dbt_snapshot_job` marker re-checked `v2_job` under RLS — the recheck the graph query itself drops. A share-link viewer got the right graph and a null marker, so the run page refetched it 40 times before giving up. `wmill script preview` read the bundle with the generic `__mod` suffix and script-module parsing, so previewing a `.dbt.yaml` omitted the project and failed on the missing `dbt_project.yml`. It uses the same suffix and verbatim read as deploy. * fix(dbt): key retry state by principal, not by script path alone `dbt_run_state` held one row per (workspace, script path), and a retry replaces the caller's arguments with the saved ones. Anyone able to run the script could therefore retry whoever ran it last, replaying that run's literal `select` and `vars` against the warehouse and publishing them as their own job's `invocation_args`. Running the script was already theirs to do; seeing another principal's arguments was not. `permissioned_as` joins the key, so a retry resumes only state written under the same authority. Two runs sharing an authority can already act for each other, so this is the boundary that matches the rest of the job model. * test(dbt): pin what a caller without access to the project sees of its run The share-link case had no regression guard, and every fix in this area touched one of its two halves: the graph's SHAPE has to survive a caller who cannot read the script, and the model SQL must not. Two cases against a real database, calling `asset_graph_for` as a member with no grant on the project's folder: pinned to a run, the models render and `raw_code` is withheld; unpinned, the same caller sees nothing of it, so making the first work did not relax the second. Both assertions were checked by mutation — reverting the `live` bypass empties the graph, and dropping the `raw_code` script gate leaks `select 1` — so neither passes on the code it is meant to catch. * fix(dbt): key the worker-local retry cache by principal too Keying `dbt_run_state` by `permissioned_as` left its worker-local twin keyed by workspace and script path alone, so the boundary held only where the database row was consulted. An agent worker never reads that table — `Connection::Http` leaves `latest_job` as `None` — so there the local cache was the whole boundary and it had none: the next principal to retry the script on that worker restored the previous one's `select` and `vars`. Also records the sqlx `--all-targets` trap in the update-sqlx skill: it is needed for queries inside tests, and in a CE checkout it aborts on `tests/otel.rs` (EE-only `otel_ee`) after having already emptied the cache. * fix(dbt): log a dropped retry-state save, correct the run-progress contract Saving retry state is best-effort — losing it costs a retry, not the run that just finished — but `.ok()` dropped the reason too. The only symptom was `dbt retry` reporting nothing to resume, which reads as a bug in retry rather than a failed write. Found by running a real failing build against a worker whose binary predated the `permissioned_as` column: the insert violated NOT NULL and said nothing. The run-progress endpoint's OpenAPI description promised an empty list for a caller who cannot see the job. It is refused instead; an empty list means the job recorded nothing yet or is unknown here. * fix(dbt): return the retry-state write failure the warning was added to report `save_run_state` discarded the insert result, so the caller's warning could never fire and a lost retry row stayed silent — the symptom being `dbt retry` finding nothing on another worker. The error is held rather than returned at once: the worker-local copy is what an agent worker resumes from, so a failed insert must not cost that too. Every exit after it surfaces it, including the ones that give up on the local save. * fix(dbt): decide a retry's graph from its restored args, keep local state in step Three from the seventh local review. A retry submits only `dbt_command`, so the vars-override check ran against an empty argument set and left `graph_is_per_run` false. The failed run's arguments are restored afterwards, and those are what the retry builds with — an overridden one wrote no snapshot for its own job and its page fell back to the deployed graph, showing the wrong enabled models, aliases and schemas. The decision is re-asked once the restore has happened. A failed durable write no longer publishes the worker-local generation either. `restore` accepts a local generation only when the database row names it, so publishing one the database never recorded made this worker reject its own newest state and resume the previous run's — its selection and vars, or "nothing to retry" if that one had succeeded. An agent worker attempts no durable write, so it keeps its local copy as before. A rename that also converts away from dbt moved the old path's retry state onto the new one, reinstating what the conversion had just cleared and leaving one user's arguments and results under a path no dbt script occupies. It moves only while the destination stays dbt, and clears the source otherwise. * fix(dbt): drop retry state when a run produced none, let module pushes fail loudly Three from the eighth local review. A run that never wrote `run_results.json` — cancelled, timed out, or dead before dbt got there — left the PREVIOUS run's state authoritative in both the database and the local pointer, so a later `dbt retry` resumed that older invocation's failed nodes. Producing nothing resumable now clears both copies, so neither can answer for the other. `wmill sync push` wrapped the descriptor lookup and its deployment in one try/catch meant for a missing parent. Any API failure or invalid descriptor was reported as "no parent found" and swallowed, so a module-only push exited zero with the remote project unchanged. Only the lookup is tolerated now. Also condenses a comment that narrated how earlier status comparisons behaved. * fix(dbt): forget retry state on pre-build exits too, drop cascade claims A dynamic run whose pre-build `dbt parse` or graph ingest fails returns before the save that clears stale state, so the previous run stayed authoritative in both the database and the local pointer and `dbt retry` resumed ITS failed nodes — writing relations the run that just failed never touched. Both exits now invalidate, through one helper shared with the no-artifact case. Two frontend comments described dbt producer rows as driving cascade dispatch. The executor returns before dispatch for every dbt job and deployment rejects `table://` subscriptions, so they promised behaviour that cannot occur; they describe the lineage and ownership that is actually retained. * fix(dbt): let the database decide retry state where it is reachable A SQL worker treated "no `dbt_run_state` row" as no opinion and accepted any worker-local generation. But no row is the authoritative answer that the last invocation left nothing resumable, so a local pointer that outlived it — an unlink that failed, a process killed between the delete and the removal, a stale cache — resurrected a replaced run and let `dbt retry` write relations it never touched. An agent worker keeps accepting its local copy: it has no authority to consult. Invalidation failures are logged rather than dropped, since a silent one is exactly what leaves the pointer behind. * docs(dbt): record how to run an agent worker locally, keep archived graphs Every step of standing one up fails as something else: a normal build cannot start one at all, the server's routes need a separate feature, and all three token mistakes surface as a bare 401 on the agent with the reason only in the server log. Written down with the error each produces. Also keeps a dbt script's graph when it is ARCHIVED rather than deleted. The pinned read resolves versions through a CTE that already skips archived rows, so clearing bought nothing and emptied the Models panel of every completed run of the project. Deletion still clears it. * feat(dbt): let an agent worker publish its graph, through one endpoint An agent worker was refused any dbt script whose profile comes from a Windmill resource — the common case — because it could neither read the stored relation root to check for drift nor re-ingest a corrected one. Those look like two needs but collapse into one: verification exists only to decide whether the stored graph still describes reality, so a worker that can PUBLISH never has to ask. It stores what it just parsed. `POST /api/agent_workers/dbt_graph/{workspace_id}` is the whole addition. It wraps the same `replace_dbt_manifest` the SQL path calls, so digest suppression, the marker write and retention cannot drift between the two transports, and it refuses a job the token's tags do not cover. `IngestedManifest`/`IngestedNode` gain Deserialize to cross the wire. Two guards go, both now false: the pre-build refusal, and the `Connection::Sql` gate added earlier to stop a `vars` override grounding an agent run. Live progress stays SQL-only — that is a per-model event stream, and routing it through the API would mean a round trip per node. * docs(dbt): warn that a differing cargo feature set swaps the shared binary * docs(dbt): record the verified agent-worker behaviour and the tmpfs quota trap An agent worker now runs a dbt job end to end, retries, and publishes its graph — confirmed with a dynamic descriptor whose per-run snapshot came back through the new endpoint. The doc said it was refused; that was true before the endpoint existed. Also `WINDMILL_DIR`: on a dev box the job dies with `Disk quota exceeded (os error 122)` writing the project's files while `df` shows free space AND free inodes, because /tmp is a tmpfs carrying a per-USER quota. Point the worker at a real disk rather than trying to clean up beneath it. * chore(dbt): pin the EE revision carrying the agent graph endpoint * fix(dbt): bind the published graph to the job, break the completed-page poll loop Five from the thirteenth local review. The EE endpoint took `script_path` and `script_hash` from the payload and checked only that the supplied job carried one of the agent's tags, so an agent holding any matching-tag job could name another script and replace its graph. Both are read from the verified queue row now and the request carries only the job id. A raw preview has no version, so it no-ops rather than 422ing before dbt runs. `IngestedManifest`/`IngestedNode` take `#[serde(default)]`: they were serialize-only, and a field the serializer skips made the whole manifest unparseable on the receiving side. A completed run page fetched the graph forever — `load()` assigns `raw`, which recomputes `settled`, which re-entered the same effect. The final fetch is keyed to the job by a plain (non-reactive) variable, and `settled` is read untracked. The pin now names a revision that compiles: the previous one still called `authed.tags()`, a method that does not exist, because both that fix and the JSON response landed after it was committed. * fix(dbt): keep a run snapshot out of the script's deployed ownership Everything `persist_ingest` writes after the manifest is keyed by PATH — one row set per script, describing what is deployed there. A run snapshot was still reaching it, so a one-off `vars` override republished that invocation's relations as the script's ownership and the workspace graph stayed on the override's schemas and aliases: an ordinary run of a static descriptor never ingests again to correct it, so only a redeploy would. A snapshot now stops after recording its own rows. The row preview also selected a bare model name, which dbt resolves across every installed package while `show` takes a single node — a project model sharing its name with a package's was previewed wrongly or refused. It selects the package-qualified FQN. * fix(dbt): forget stale retry state when preparation itself fails `prepare_project` runs before every path that could clear it, and it fails for reasons unrelated to the saved run — a profile that stopped resolving, a provision cancelled, packages that will not install. The invocation still left nothing resumable, so the previous one must not stay authoritative: a repaired project would otherwise let `dbt retry` rebuild an older run's selection and write relations the latest invocation never reached. A retry is exempt, since it is trying to use that state and failing to prepare says nothing about it. Also corrects the runtime doc, which still described agent workers as unable to run dynamic descriptors or Windmill-resolved profiles. They publish their graph through the API now; what they do not get is live progress and a durable retry row, and the doc says so. * fix(dbt): spell the whole FQN for preview, bound retained retry generations The FQN selector added last commit was `<package>.<name>`, but a dbt FQN is the resource's path within its package and the matcher must consume the selector and end on equal lengths — so it matched nothing for a model under `models/marts/`, which is the layout most projects use and the one this repo's own complex fixture has. The middle segments come from `original_file_path`, whose first element is the resource root the FQN excludes. Without a path it falls back to the bare name: ambiguous across packages, but a selector dbt resolves rather than rejects. Tested on a nested model, which is the input that separates the three spellings. Superseded retry generations were removed only when a later run published one, and never inside the hour-long grace period — so a burst left a manifest and a results copy per run with nothing afterwards to collect them. At most four now sit in the grace window, oldest evicted first. * fix(dbt): scope preview state to the run, seed the project on a language switch Previews are keyed by `unique_id`, which is the same string for the same model in every run, and the run-change effect reset only the graph and progress. Opening a second run of one project therefore showed the previous run's rows immediately, and `runPreview` treated them as cached and refused to fetch. A generation counter also drops a preview that resolves after navigation, which the reset alone cannot catch. The dbt project was seeded only by the empty-script bootstrap, but dbt is in the ordinary language picker: reaching it by switching a draft produced a script with no `dbt_project.yml`, which the runtime refuses to deploy or run. Both entry points seed now, and neither touches modules that already exist. * chore(dbt): cache the agent graph endpoint's query for the EE offline build * fix(dbt): publish the graph a moved profile built A run snapshot stopped before everything `persist_ingest` keys by PATH, which is right for a one-off `vars` override and wrong for the other two reasons a run re-ingests. `graph_is_per_run` was one bool for all of them, and the profile drift check both sets it and reads back what the publisher recorded: a profile moved A->B was detected by every run forever, each paying a `dbt parse` for a snapshot nobody reads while the asset rows went on naming schema A. The reason is carried now (`GraphRefresh`), and it decides both writes. Drift is the version's own move, so it rewrites the VERSION's graph and republishes the ownership that ends the drift; a dynamic descriptor snapshots under its job id and still publishes; anything the CALLER scoped — an overridden `vars`, a narrowed `select` — snapshots and publishes nothing, so one invocation's subset can neither stand as what the script owns nor drop the models it left out from the version's graph. Where they meet the caller wins, and the next ordinary run settles the drift. A restore also rebuilt `run_results.json` by copying the generation directory a second time, so a burst of saves pruning it mid-restore left `dbt retry` with nothing to resume and a job that reported success. It is written from the bytes the restore already read; a manifest that went the same way falls back to the parse a database restore pays anyway, and a generation that vanished before either read falls back to the database's row for that same run instead of reporting there is nothing to retry. * fix(dbt): select a row preview by package, not by file path The preview built dbt's FQN by dropping one segment of `original_file_path`, which assumes the model root is `models/`. A project setting `model-paths: ["src/models"]` turned `src/models/marts/orders.sql` into `pkg.models.marts.orders`, and dbt's matcher — equal lengths, compared from the front — resolves that to nothing: the preview came back empty for every model in the project. It selects `<name>,package:<pkg>` instead. The comma is dbt's intersection operator, so this names the node by its own name and the package it belongs to, which is what the FQN was reaching for and needs no knowledge of the resource root. Verified on dbt-core 1.12, dbt-core 2.0.0-alpha.5 and fusion 2.0.0-preview.202, including a package shipping a model whose name the root project also uses. * fix(dbt): refuse a lockfile version that is not one, keep a named selector Two things a preview reaches that a deploy does not vouch for. A raw preview submits its own `lock`, so `engine_version` arrives from the caller and was interpolated straight into the engine cache path — `../..` in it made the download, extraction and rename land anywhere the worker can write, and provisioning runs on the host rather than inside the dbt jail. Both it and `adapter_version` (a pip requirement) are now accepted only as a plain version token. `effective_selector` also read any submitted `select`/`exclude` as an override of the descriptor's named selector. The generated run form posts a default back for every field the caller left untouched, and a selector descriptor's `select` default is `[]` — so pressing Test, saving a schedule or firing a webhook built the WHOLE project instead of `--selector nightly`. An override is now one that DIFFERS from the descriptor's own value; a run that wants the whole project despite the selector asks with `["*"]`. * fix(dbt): let a moved profile settle, from the runs that actually happen Two ways the drift check could never come to rest, both verified against a real run of a real project on a normal worker. `add_caller_args` read any submitted `select`/`exclude` as a caller's narrowing. The generated run form posts a default back for every field left untouched, so every run from the UI, a schedule, a webhook or a flow step carried them and was marked caller-scoped: with the profile moved A->B, each one stored its models under its own job id and left the workspace graph — and the root the check reads back — at A. Since no UI run omits the field, the "an ordinary run settles it" escape hatch was unreachable. Both this and `effective_selector` now ask one question, `selection_is_overridden`: DIFFERENT from the descriptor's, not merely submitted. The root was also recorded beside the path-keyed publication rather than beside the graph it describes, so a version that cannot claim the path — an older one run by hash, a deploy overtaken by a newer one — rewrote its graph at the moved root and recorded nothing. Its next run then compared against a root that was absent or two moves stale and skipped the refresh its own run page needed. It is written wherever the deployed row's graph is. Verified end to end: same UI-shaped arguments before and after, the moved profile now republishes (asset rows and version graph both move to the new schema), a second run detects nothing and re-parses nothing, and moving the profile back settles it again. `prune_dbt_run_graphs` also ran from runs alone, while a deploy writes a whole node set of its own, `raw_code` per model included: a project redeployed on every push by CI and run nightly kept one full graph per push until the next run, and one deployed but never run kept them for good. * fix(dbt): drop a self-dependent effect in the run graph `previewGen` was `$state` written by the effect that also reads it, three lines under a `finalLoadFor` that is a plain `let` for exactly that reason. Nothing reactive reads it — the only reads are inside `runPreview`, a plain async function — so it becomes a plain `let` too. * fix(dbt): pin a retry to the engine versions it resolved `run_identity` carried the engine KIND but not the version it resolved, nor the dbt-core 1.x adapter's. Redeploy an unchanged project after a release and it locks a newer dbt or adapter while the saved `run_results.json` still passes the check, so `dbt retry` feeds one version's artifacts to another — the exact reproducibility the lockfile exists to hold. Both resolved versions are in the identity now; a real failure and retry still resumes. Also drops three comments that outlived what they describe: two said `[]` clears a descriptor's selector, which `selection_is_overridden` reversed, and one pointed at an agent-worker guard that no longer exists — the agent path reaches the ingest deliberately and publishes through the API. * fix(dbt): discard a run graph the page has already navigated away from The component is reused across runs, so a slow `/jobs/dbt_graph` or progress response could land after the reset and put the previous run's models, statuses and failure state on the current run's page, where nothing would fetch again to correct it. Every response is now checked against the generation it was requested under — the counter the preview path already used, renamed for what it means. The graph poll also backs off. Neither of its stops is reachable for a whole class of runs — `dbt_snapshot_job` never matches a static descriptor, and `polled` stays empty for the engines that emit no node events — so an ordinary run walked to the cap, re-sending every model's SQL 40 times in two minutes. * fix(dbt): forget the previous run when the durable save fails, keep quoting `save_run_state` returns the database error when its upsert fails, which leaves run N-1's row and local generation in place: same project, same arguments, so a `dbt retry` matches them and resumes an older attempt's failed nodes against this checkout — the outcome the no-results branch twelve lines above calls `invalidate_run_state` to prevent, reached by another door. It now goes through the same call. Best effort, since the delete goes to the database that just refused a write, but the local pointer is what a retry landing back here reads. The run page also rejoined a relation's parts after `splitRelation` stripped their quotes, so the one name the button exists to paste — `"wh"."analytics.v2"."Order Items"` — was copied as something no client resolves. It copies `relation_name` verbatim. And a source on a finished run was called another project's: the check that guards against two projects claiming one relation asks whether this run executed the node, and a run executes no sources — they appear in no `run_results.json`. Nothing materializes a source, so that warning could never be true of one. Docs: the `vars`-override paragraph still said such a run does not refresh the graph, which the table above it contradicts — it refreshes under its job id and publishes nothing. * fix(dbt): keep the asset rows and the version's models describing one graph The workspace graph takes an asset's relations from the path-keyed `asset` rows and its models, SQL, tests and lineage from the version's `dbt_node`/`dbt_edge`. A dynamic descriptor published the former while storing the latter under its own job id, so a placeholder that moved an alias or a schema left the current graph with assets no model stands behind — nothing dbt contributes to them survives. Ownership is published exactly when the VERSION's graph was written now, which is the only state in which the two agree. Two cases are settled elsewhere by design: an override's relations are a one-off, and a dynamic descriptor at a moved profile keeps the deploy's ownership until a redeploy — its runs each show their own models and it re-parses regardless, so the undetected drift costs it nothing it was not already paying. An agent worker has no durable row, so its local `current` pointer is the whole of what a retry reads — and every local publication failure returned success with the PREVIOUS run's pointer still in place. Where a row exists that is harmless (`restore` takes a local generation only when the row names it), so the abandonment is scoped to the agent case. A failed `dbt show` also cleared the retry state: the preparation-failure exempts `retry` but not a read-only command, and the run page's row preview is exactly that, run as the principal the state is keyed by — so a preview that could not provision took the retry away from the run being looked at. Frontend: the run-change reset left `loading` and `failed` behind, so the gap before the next run's answer rendered "no models in the asset graph" — a claim about the descriptor — over a project that is fine. And three derivations argued from "the graph is the current deploy", which the pinned endpoint made untrue; each is still needed, for the version-graph rewrite and retention reasons now written down. * fix(cli): let --skip-scripts cover a script's module files The module shortcut in `elementsToMap` maps the file and `continue`s before every skip filter, and a module is deployed as part of its parent script — so `wmill sync push --skip-scripts` still pushed the script whenever one of its modules changed, and pull still overwrote them locally. Harmless while a module was a rare helper file; every file of a dbt project is one of these now. * docs(dbt): a dynamic descriptor's ownership stays the deploy's * fix(dbt): read the run out of a failure whose message has braces of its own `parseDbtRun` anchored on the FIRST `{` in the error message and parsed everything after it. The worker appends the structured result after the error text, and dbt's errors carry braces — a Jinja template, the compiled SQL, an adapter's own JSON — so the failures most worth reading were the ones whose summary and per-node outcomes the run page dropped. Every brace is tried now, bounded, and the first that parses as a run wins. Pins the EE revision that gives the agent publish endpoint the deleted-version guard the SQL path takes: deletion is soft, the foreign key still accepts graph rows, and the pinned graph query serves non-live versions, so an agent finishing during a delete put a deleted project's model SQL back on screen. The query is byte-identical to `persist_ingest`'s, so the offline cache already covers it — verified with a full-EE `SQLX_OFFLINE=true` check. * fix(dbt): seed a project when a modular draft switches to dbt `seedDbtProject` returned whenever the draft carried any module at all, so a modular script holding a `helper.ts` reached dbt with none of what dbt needs: the project view is read-only, and the worker refuses a bundle without `dbt_project.yml`, so that draft could neither run nor deploy. Keyed on the project file now, and the seed goes in under whatever is already there — the previous language's helpers are inert to dbt and the user's to remove. Also records this runtime's schema in `backend/summarized_schema.txt`: the `table` asset kind, the `dbt` script language, the five dbt tables and the `materialization_status` enum the progress table uses. * docs(dbt): move the pipeline-membership rationale out of the deploy path * fix(dbt): gate a pinned run's model SQL on the version it belongs to The `EXISTS` against `script` is the only thing standing between a share-link viewer and the project's source, and it matched the workspace and path alone. `extra_perms` is a grant on a ROW: archive a version that granted someone access, recreate the path with narrower permissions, and that stale grant satisfied the probe while the query returned the NEW version's `raw_code`. Both probes name the hash now. The regression test drives exactly that shape and fails without it, returning `select 2` to a caller granted only on the archived version. * fix(dbt): keep a delimiter an identifier escaped by doubling Every dialect these relations come from escapes its own delimiter by doubling it, and both split functions closed the quoted section on the first half and reopened on the second: `"schema"."a""b"` came out as `a.b`. The manifest keeps the real spelling, so the run wrote its per-model status and row counts under an asset path no graph node has — the node simply never moves, which is the failure mode this splitter exists to prevent. Fixed in the worker and in its frontend mirror, which have to agree, with a case per delimiter on both sides. * fix(dbt): key retry state by the caller, not only by the principal it runs as An `on_behalf_of` script executes every caller's job as its owner, so `permissioned_as` names one principal for all of them and the retry state — the durable row and the worker-local generation both — collapsed onto a single entry. After one caller's run failed, the next could submit `dbt_command: retry` and resume it: their arguments replayed against the warehouse, and handed back through `invocation_args`. Nothing else separated them, and on an agent worker the local directory is the whole boundary. `created_by` joins the key in both places. For an ordinary script it changes nothing — `permissioned_as` is already that caller — and a run that was itself superseded was never resumable anyway. Includes the offline cache for the four changed queries and the three the pinned-graph regression test added last commit, which had none: `prepare` without `--all-targets` does not compile test targets, so CI's `SQLX_OFFLINE=true ... --all-targets` would have failed on them. * fix(dbt): compare the schema too when reporting a relation that moved `relationDrift` compared the leaf name alone, and the move it exists to report — a profile repointed at another schema, which a later run then writes into the version's graph — leaves every model's name exactly where it was. So the one case that reliably produces a graph naming relations this run did not write was the one case the notice stayed silent for. The schema segment joins the comparison, qualified against qualified: an unqualified one means the target's own database, which the relation names anyway, so comparing that would report a move on every node. * fix(dbt): bound the retry state now that it is keyed per caller Keying by `created_by` fixed one caller resuming another's run and created a growth problem doing it: a shared `on_behalf_of` script kept one row and one worker directory for everyone who had ever run it, and the generation prune only bounds files INSIDE a directory. Three bounds, none of them new machinery. A run with nothing failed or skipped saves nothing — `dbt retry` builds from those nodes alone, so that state could only ever be refused — while still clearing what the previous run left, since its failures are no longer what last happened here. The rows expire on the same 30-day clock as a run snapshot, swept per path by the prune every dbt job already spawns. And the worker-local directories are swept there too, by the age of the pointer a save rewrites, because their digest names neither the script nor the caller. * docs(dbt): the retry state is worker-affine only on an agent worker * fix(auth): only the server may set a token label that names a user `create_token_internal` wrote `NewToken.label` verbatim, and the auth layer reads some labels as an IDENTITY: `username_override_from_label` maps `ephemeral-script-end-user-<name>` to exactly `<name>`, which then becomes `created_by` on every job that token pushes. The label is free-form request input, so any member could mint a token that speaks as somebody else — the shape `require_job_read_access` already works around when it refuses to trust `username_override` and falls back to an RLS probe, and the one that made dbt's retry-state key (`created_by`) forgeable for an `on_behalf_of` script. The labels are refused where request input enters: the member-facing `tokens/create`, and `impersonate`, which names its subject in `impersonate_email` and has no business renaming the caller too. The legitimate producers are unaffected — a job's own token comes from `create_token_for_owner` in the worker, and native triggers and app-embed tokens build their labels themselves rather than accepting one. `Ephemeral lsp token` stays allowed: its override is the fixed sentinel `lsp`, not a name the caller chose, and the editor mints exactly that label through this endpoint for its language server. The test pins the two lists together, so an arm added to `username_override_from_label` that lets a label choose a name fails until it is reserved too. * Revert "fix(auth): only the server may set a token label that names a user" This reverts commit efaa498d82. * fix(auth): only the server may set a token label that becomes a bare username * revert(dbt): key retry state by the execution principal again Reverts the per-caller key and the retention it needed. `created_by` cannot carry an isolation boundary: it is `display_username()`, which a token LABEL supplies, so two callers can share one value and — before the guard two commits back — one could name a third person. GHSA-8x8x-88qc-qp4r settled that class by refusing to trust the name for authorization, and keying on it here was the same mistake in a new place. It also cost a migration, a PK column, two sqlx cycles and a retention sweep to defend. Back to `(workspace, script_path, permissioned_as)`, which is derived from the authenticated username and cannot be chosen by a request. What that boundary is, and the one case it does not cover, is now written down where the retry is specified rather than left to be re-derived: anyone entitled to run the script as that principal may resume its last failure — the same capability as re-running that job, since running it requires the read access that already shows them the run and its arguments — except for a run pushed `invisible_to_owner`, whose arguments a retry still returns. Closing that means having a retry NAME the job it resumes and authorizing it as a job read, which is a change to the run argument, not to the key. * fix(dbt): give a caller's own selection a graph, and load a finished run once A run that overrides `select`/`exclude` was marked caller-scoped but not per-run, so it ingested nothing and its page fell back to the deployed graph. That is only right when the override NARROWS the descriptor's selection — `["*"]`, or any model outside it, builds relations the deployed graph never had, and those are exactly the ones whose progress, SQL and lineage had nothing to draw. It ingests its own graph now, still caller-scoped, so the subset stays out of what the script owns. `relationDrift` also read a schema whose own name contains a period as a move: a segment holding one is ambiguous — an overridden database, or a schema really called `a.b` — so either spelling now counts as unmoved. And a finished run fetched the whole graph twice on mount, every model's SQL included: the reset effect and the finished-run effect both fired in the same tick. The reset only loads while the run is in flight; the other owns the finished case, because a snapshot can land after the run ends. * fix(dbt): classify a restored retry by its resolved arguments `add_caller_args` was handed `raw_args` on the retry path — the arguments as SUBMITTED, where a `select` spelled `$res:` is still a string. `arg_list` then refuses it as "must be a list of strings" and the retry dies before parsing, for a reference that resolves to the very list the failed run built with. The resolved map decides it now, matching the selection resolver and the build; `raw_args` stays what is persisted and published, so no resolved secret outlives the job. Verified against the shape it breaks on: a `select` from a resource, failed, then resumed. Two docs that outlived their code: `dbt_script_hash` is described as a fallback for a job naming no deployed script rather than as the pinning mechanism, since a script job's version comes from the job row and the query value is ignored (spec plus the generated client, which carries the same sentence); and `GraphRefresh`'s fields no longer claim a caller's selection only ever narrows, which is the premise the previous commit disproved. * fix(dbt): a hidden run keeps no retry state The state is keyed by the execution principal, which every caller of an `on_behalf_of` script shares, and a retry publishes the arguments it restored — so for a run the other callers cannot read, that retry was the one way to see them. The equivalence the key rests on ("resuming it is the same capability as re-running it") holds only while the run IS readable, and exactly one run is not: one pushed `invisible_to_owner`. Those now save nothing, and clear whatever the previous run left, since this invocation happened. A hidden run therefore cannot be resumed by anyone, its author included — the cheaper half of the trade, and the reason having a retry NAME its source job is the design that would give it back. Verified: a visible failure still saves and resumes; a hidden one leaves no row, and the retry after it refuses without returning the hidden `vars`. Also collapses `GraphRefresh::caller_scoped`, which stopped distinguishing anything once a selection override became per-run: both writers set both flags, so `snapshot_job` and `publishes_ownership` are now one field and its negation — which is what lets the agent payload carry a single `per_run` bit, recorded there. `is_reserved_token_label` says why `ephemeral-` is reserved (a client-chosen system label is a token its owner cannot list or revoke) rather than restating a username rule its second prefix does not follow. And the module-cache import comment states the invariant once, without the shape of the cache that preceded it. * fix(dbt): a pre-build failure leaves the saved run alone Three exits cleared the retry state before `dbt build` ever ran: a preparation failure, a failed `dbt parse`, a failed ingest. None of them touches a relation, so the warehouse is exactly what the previous run left and its failures are still the accurate description of it — clearing there just costs a resumable failure, and a resume that no longer fits is refused by `run_identity` and the arguments digest regardless. The reachable one is a cancellation during provisioning. Only an interrupted BUILD invalidates, which the save at the end of the job already decides. Verified: a failure saves state, a run whose selection matches no node fails in the ingest and leaves it, and the retry after that still resumes the original failure. Agent-published graphs also had no sweep — `Connection::Http` snapshots every run and spawns none — so the EE endpoint prunes too (pinned at f58bf06), and `prune_dbt_run_graphs` now names every writer rather than only the runs. Drops `ephemeral-webhook-` from the reserved token labels: it yields the label verbatim, not a bare username, so the check now matches the rule its doc and its test state, and the reason given for it was wrong about what `is_user_token` costs an owner. And the result contract lists `invocation_args` — the field most needing it, being another invocation's arguments on a retry. * fix(assets): decode a doubled delimiter when canonicalizing a table key `split_relation` decodes it on the worker side, so the canonicalizer had to as well: a relation whose identifier escapes its own delimiter — `"sales""east"` — was filed by the ingest under `sales"east` and by a hand-written `table://` annotation under `sales""east`. Two nodes for one table, no edge, which is the exact split this key exists to prevent (decision 11). Fixed for both the name and each half of a database-qualified schema segment, with a case per dialect. `invalidate_run_state`'s contract also still promised the pre-build behaviour the previous commit removed; it now names the cases that do invalidate — an interrupted build, and a run hidden from the script's owners. * fix(dbt): reclaim package trees no project asks for any more The cache key covers the whole project digest — a `local:` dependency's content is in no manifest, so nothing narrower is safe — which means every edit of a project that declares packages publishes another full dependency tree, and nothing ever removed one. A worker that lives through a hundred deploys held a hundred trees until an operator cleared the entire cache by hand, and the disk that fills fails every job on that worker, not only dbt's. Swept by last USE, not by publication: a hit dates the tree, so a project unchanged for months is not evicted from under the jobs still running it. The marker is a sibling rather than a file inside the tree, which the restore copies into the project. Staging directories go after a day — one belongs to a single job and is removed when it ends, so an older one is from a worker that died mid-publish. Verified by planting a 30-day-old tree and a stale staging dir: one run reclaimed both and left a fresh tree alone. Narrowing the key to the declared `local:` paths would fix the churn as well, and is deliberately not done here: getting it wrong runs a project against another revision's packages, which is worse than the disk it saves. * docs(api): declare the dbt half of the asset-graph response `AssetGraph` is the response of both `/assets/graph` and the new `/jobs/dbt_graph/{id}`, and the spec described neither `assets[].dbt`, `runnables[].dbt`, `dbt_edges` nor `dbt_snapshot_job` — so every client generated from it saw a graph with no dbt metadata, no `ref()` lineage and no snapshot marker, which is why the run page reaches the endpoint with a raw `fetch` and a hand-written cast. `frontend/src/lib/gen` is gitignored, so the spec is the only tracked description of this surface. Declared, and the client regenerated from it now carries all four (verified: the regen is otherwise a byte-for-byte no-op). * fix(dbt): stop the package sweep from racing the restore it protects Three faults in the sweep the previous commit added. The `.last_used` marker was written AFTER the copy, and the tree it matters for is the one at the retention edge — where the first use in a fortnight is also when the sweep fires. So the source looked stale for the length of the restore and a concurrent job's sweep could remove it mid-copy, failing a run that should merely have refetched. Marked before the copy, and a restore that loses the race anyway is treated as a cache MISS: `dbt deps` resolves the tree again, costing a fetch rather than the run. The sweep also sat inside `if let Connection::Sql`, though it is a walk of the worker's own disk that needs no database — so an agent worker, which has none, published a tree per project edit and reclaimed nothing. That is the state the doc claimed was fixed, and the same argument the graph sweep makes two lines above: retention that depends on the connection is not retention. And the canonicalizer's two halves disagreed after `5890436`. `unquote_identifier` requires the quote at both ends, so a lone `"` in an already-decoded name survives; the schema walk treated one as opening a quote and dropped it, filing the ingest's `sa"les` under `sales` while an annotation's `"sa""les"` decoded to `sa"les` — the split this canonicalization exists to prevent, in the function I had just touched. The halves share one rule now, and the test drives the decoded spelling against the quoted one rather than asserting only the annotation's. * fix(cli): track a dbt project's parent descriptor for every authored file `buildTracker` decides whose top hash `wmill-lock.yaml` refreshes, and it reached the module branch only for files matching a Windmill script extension. A dbt project is mostly files that are not: `dbt_project.yml`, `packages.yml`, schema YAML, seed CSVs. Editing any of them left the descriptor untracked and its module-inclusive hash stale, so the lock disagreed with the bundle that was pushed. Module paths are now handled ahead of that gate. The parent was also derived by searching the RAW path for `__dbt/`, which finds nothing on Windows, where the folder is spelled `__dbt\` — so even a model edit was skipped there. It goes through `getScriptBasePathFromModulePath`, which normalizes separators, and which the sibling helpers already used for this exact reason. The regression test drives all three shapes. With the old gate reinstated it fails on the non-script files and on the backslash path, and passes only for `.sql`. * fix(dbt): re-resolve unlocked dependencies, and reclaim retry state nobody writes A project with no checked-in `package-lock.yml` asked dbt to RESOLVE its ranges and mutable git revisions, and dbt does that on every run. A cache hit skipped `dbt deps` entirely and every hit refreshed the marker, so the first resolution was pinned for as long as the project stayed in use — a version range that moved was never picked up. An unlocked tree is now a miss once it is a day old; a locked project keeps its tree, since the lock is in the key and pinning is what it asked for. Retry state: `invalidate_run_state` removed the pointer and left the generations, and the whole state root was swept by nothing after `a236972` — so a script that stops running, or a principal who stops running it, kept its last generations (a `manifest.json` each) for good. Forgetting the state now prunes its generations, and a 30-day sweep of the state root runs beside the package sweep, for the reason that one already gives: retention that runs only for the script being run is not retention for the ones that are not. This restores one half of what `a236972` reverted — the local sweep, not the per-caller key or the row retention — because it never depended on that key. The graph orphan sweep also committed its marker delete separately from the two deletes it gates, so an error or a restart in the gap left graph rows whose marker was gone — and since the sweep only runs when a marker went, every later call skipped it and those rows were unreachable for good. One transaction now. `parseDbtRun` scans from the LAST brace: the payload is appended, so backwards finds it immediately where forwards parsed the whole message once per brace in the error text. And `reject_reserved_label` moved into `create_token_internal`, the one path every caller-supplied label reaches, so the invariant sits where a new route would break it. * fix: undo two of my own regressions, and agree with the backend everywhere `2e34dd48` hoisted the module check above the extension gate, which is right, but did not bring the entry-point rule with it: a folder-layout script's METADATA is `<base>__mod/script.yaml`, an entry-point path, and pushing it as a content file makes the metadata pass ask for the language of `.yaml` and abort the whole command. Reached by editing the summary of ANY modular script — pre-existing, nothing to do with dbt. Metadata resolves to its content file now, with a test. `d3716a09`'s unlocked-dependency refresh is reverted. It measured staleness against the last-USE marker, which every hit rewrites, so an actively used project never reached the threshold — and when it did fire, the re-publish `rename` cannot land on a populated directory, so the freshly resolved tree was discarded and the stale one re-stamped: a network resolve thrown away per idle day. Doing it properly needs a publication timestamp and a publish that can replace a tree; the limitation is recorded where the cache is keyed instead of half-implemented. `parseDbtRun` no longer counts braces in either direction. The payload is appended pretty-printed, so its `{` is the only one at column zero — forwards blew the cap on an error full of braces, backwards on one `{` per node, which a few hundred nodes reaches. Test drives a 400-node failure. And `parsePipelineAnnotations` was the third copy of the quote rule, still splitting a doubled delimiter: the live canvas keyed `"sales""east"` as `saleseast` where the deploy keys `sales"east`, so an annotation pointed at a node the deployed graph does not have. All three agree now, including on a lone delimiter in an already-decoded name. * chore(dbt): leave cache retention and the migration's comment out of this PR Two deliberate subtractions, so what ships is the set that has been verified rather than the set that was written. The package-tree and state-directory sweeps are gone (recoverable on `dbt-cache-retention-followup`). They delete directories on a worker, they were the newest code here, and one of them already needed a second pass for racing an active restore — while what they buy is disk hygiene, not correctness: today an operator's `cache_clear` reclaims these caches, exactly as it does for every other language. Both limitations they addressed are now recorded where the cache is keyed. `invalidate_run_state` still prunes the generations it orphans, since that reuses the existing pruner and its grace window rather than sweeping a root by age. And the migration is back to its pre-PR bytes. The only change left in it was a corrected comment, which every database that already applied this migration would have paid for with a checksum mismatch at startup — the explanation lives in `prepare_project` and in the design doc, which is where it is read. * fix(dbt): pin the resolved package tree to the deployed version A cache hit skipped `dbt deps`, so an unlocked range or mutable git revision stayed on whatever the first worker resolved, and a retry could feed one resolution's run_results.json to another. The deploy now records the digest of the generated package-lock.yml, the package cache is keyed by it, and it joins the run identity; a worker that resolves anything else is refused rather than run. Also carries the agent-wire round trip test for an ingested manifest, and the EE pin for the agent publish endpoint. * docs(dbt): state the dependency contract the deploy pins The design doc still described the package cache as keyed by the project digest alone, and carried an open question about refreshing unlocked dependencies that the deploy-time pin answers: resolution happens once, at deploy, and every run of that version installs it or is refused. Records what that costs and buys — committing package-lock.yml makes deploys cache-hit, deploying again is what picks up a newer range — and that these worker-local caches are reclaimed by cache_clear, as every other language's are. * fix(dbt): four findings from the review round - An interrupted retry republished the results it had only restored. A retry starts with the previous attempt's run_results.json in place; cancelled before dbt rewrites it, the save dated those failures to this job, so the next retry rebuilt nodes this one had already redone. Verified by A/B: without the check the row survives, carrying the cancelled retry's id. - The deploy refused a committed package-lock.yml that dbt itself updates, which it does whenever the sha1_hash it recorded for packages.yml no longer matches. Only a run has a resolution to be held to; the deploy establishes one. - A second graph load for the SAME run could land out of order, leaving a finished run showing the deployed fallback for good. - Directory nodes in the project tree carried no path, so two of one name at the same depth shared a collapse key and folded together. * fix(dbt): name every adapter, and stop guessing which project owns a model The invalid-adapter message listed four of the eleven accepted spellings. And a relation may have several script producers, none of which the provenance record identifies — prefixing the first one names a __dbt folder that does not exist, so an ambiguous relation now shows the path inside the project alone. * refactor(dbt): name the graph-snapshot column for when it is written `published_relation_root` described a design persist_ingest does not use: the root is recorded by every ingest that is not a run's own snapshot, including one that publishes no ownership — deliberately, since a version that cannot claim the path would otherwise record nothing and compare against a stale root forever. The column comment said the opposite and the name followed it. Safe to edit the migration in place because this PR introduces it: no database outside a checkout of this branch has ever applied it. One that has needs UPDATE _sqlx_migrations SET checksum = decode('<sha384 of the file>','hex') WHERE version = 20260725084314; alongside the column rename, or a fresh database. * style(dbt): bring comment blocks within the four-line guidance AGENTS.md asks for each invariant at the place someone would break it, in at most four lines. The new dbt files carried 60 inline blocks past that, several of them three rationales deep at one site. Nothing durable is dropped: what covered several constraints at once is split to the lines it constrains, and eight comments that had drifted above the wrong test — six stacked over one profile test, two over the wrong executor test — are reattached to the tests they describe. * docs(dbt): carry the snapshot-column rename into the summarized schema The rename reached the migration, both queries, the sqlx cache and the design doc, but not the compact schema summary — which is the file agents read instead of the migrations, so it was the one copy that could mislead silently. * docs(dbt): bound the retry-state and package-pin residuals precisely Two claims in the design doc were true but not precise enough to act on. Retry state: the equivalence between resuming a failure and re-running the job holds because folder read grants job read, which is also what grants execution. It does NOT hold for a u/<owner> script shared through extra_perms, where no folder policy applies and a grantee cannot read even their own on-behalf run. The package pin: the refusal is per worker. A ranged dependency with no committed lock reproduces its resolution only from a cache hit, so it keeps running where the tree is warm and fails on the first cold worker. * fix(dbt): refuse a retry of a run its caller cannot read Retry state is keyed by execution principal, so every caller of an on_behalf_of script shares one saved run. Under a folder that matches job visibility exactly, but a u/<owner> script shared through extra_perms has no folder policy: a grantee can read neither the run nor their own, while a retry published its arguments. The restore now applies require_job_read_access's rule — you can always read a job you launched, otherwise the row must be visible under your own RLS. Deliberately NOT job_perms, which carries the identity the job runs AS: for an on_behalf_of script that is the owner, and probing with it authorizes everyone. Verified with two members on one shared principal: bob is refused alice's run on the u/ path and her marker never reaches his result, alice resumes her own, and bob still resumes alice's run of the same script under a folder. * fix(dbt): close the retry check's own gaps The check landed with three holes and four rough edges, all found by review: - A pruned source run was allowed. dbt_run_state outlives job retention, so that handed the last failure's arguments to whoever asked next. It now fails closed, and says so rather than claiming the run was someone else's. - An agent worker skipped the check entirely: it reaches no database. The saved generation now records who launched it, and an agent authorizes the half it can prove — the launcher may resume. A generation written before this field is not resumable there. - The row was authorized on one read and restored on another, so a run completing in between was restored unauthorized. restore_from_db now takes the job it was authorized against. Also: restore_run_state's doc comment had drifted onto the new helper, the not-a-member comment described the saved run rather than the caller, and the refusal message carried a run of spaces from its line continuation. * fix(dbt): keep the descriptor in --json sync, and fail closed on a stateless row A dbt script's content is `<name>.dbt.yaml`, and `elementsToMap` drops every .yaml when metadata is JSON — so a --json workspace tracked the metadata, the lock and the whole project bundle but not the descriptor: a descriptor-only edit pushed nothing and a fresh pull wrote a script with no source. Pinned by a test that fails without the exemption. And the retry check read job_id flattened, which merged "no row" with "a row naming no job" and skipped authorization for the second. Unflattened, the row that cannot be authorized is refused. Also graph_digest: serde rather than {:?} for edges, and delimited parts. * docs(dbt): state that retry authorization is by identity, not token scope The section claimed the read-access equivalence was enforced. It is, for identity — but the worker never sees the submitting token, and neither v2_job nor job_perms records a scope, so a token denied jobs:read can still resume its own principal's last failure. What it gets is the arguments as submitted, so a reference is re-resolved under whoever retries rather than disclosed. Also names the cases the check refuses outright, which were spread across three commit messages and nowhere a reader would look. * chore(dbt): repin EE after rebasing the agent-graph branch onto EE main * fix(dbt): say why a test-only run has no models, instead of blaming the profile A selection of tests alone builds nothing, and a test is an assertion rather than a relation, so the ingest keeps nodes with no asset to hang on and the graph comes back empty. The run is correct and its results render — but the empty state named the one cause it is not, a project with no warehouse identity, and sent the reader to their profile. The graph itself is unchanged: retaining a selected test's attached model would have a test-only script claim a read on every model it asserts against, which is an asset-graph semantics change and wants its own review. * fix(dbt): authorize a retry by RLS alone, and eight review findings Retry authorization drops the created_by grant that mirrored require_job_read_access's "you launched it". That name is a display name derived from a token label, so a worker cannot tell a launcher from a collision — the objection Codex raised on four heads. Visibility under the caller's own RLS is the whole rule now. An on_behalf_of script under u/<owner> shared by extra_perms is resumable by the owner alone; under a folder every caller keeps the resume. Also, and each verified against the code first: - The local generation is revalidated after the file work, so a newer run publishing state mid-restore no longer lets the superseded one resume. - Both dbt zip lookups in sync normalize the OS separator, as the resource lookup beside them already did: on Windows a pulled dbt project was laid out as __mod, the one layout dbt cannot run from. - A resource storing port as a string no longer silently connects to the adapter default; a value that is not a port is refused. - An oversized but readable project file fails the deploy instead of shipping an incomplete project that compiles and fails at run time. - cache_clear does NOT reach the dbt caches: it clears cache/ while these live under cache_nomount/, as bun's do. The doc said the opposite, and that claim was the justification for dropping the retention sweeps. - A dbt unit test counts as a test for the empty-state message, as it already does for the worker's test-phase detection. - Per-node results keep the catalog segment when rows disagree on it. - The project panel copies through the app's clipboard helper, .dbt.yaml resolves back to dbt in EXTENSION_TO_LANGUAGE, and two test blocks clean up their temp directories. A comment describing a race against a sweep this PR does not ship is gone. * fix(cli): keep the oversized-file check to a bounded read The refusal statted the file and then read it whole to tell text from binary, which is the cost the two comments above it and isBundledModuleFile exist to avoid: a multi-gigabyte seed would be loaded just to be refused. Same 8 KB head read as that predicate. * revert(dbt): accept the retry residual instead of gating it Resuming grants no capability a caller lacks: they may already run the script as that principal, and a plain run builds a superset of what a retry rebuilds. The one thing a retry adds is information -- the result echoes the resumed run's submitted arguments, for the row preview -- and in almost every shape the caller could already read that run: a folder grants job read alongside execution, and an ordinary script's runs are keyed under the caller's own principal. It takes a user-path script AND extra_perms sharing AND on_behalf_of for the two to diverge. Gating it needed an identity the worker does not have. created_by is display_username(), which a token label supplies: trusting it authorizes a collision, and resolving it as a username denied every labelled token -- a CI token is `label-<name>`, no workspace member, so its own retry was refused. That regression cost more than the exposure. Kept, because none of it is about the caller: - a cancelled retry no longer republishes the results it only restored, - the restore is pinned to the row it chose rather than re-reading it, - a newer run publishing mid-restore no longer lets the superseded generation resume. The three sqlx entries the gate needed are gone with it. * fix(dbt): select unit tests, and say why the state key is the boundary `dbt ls` enumerated five resource types and omitted `unit_test`, which dbt treats as its own type rather than a flavour of `test`. A descriptor selecting `test_type:unit` or a unit test by name therefore resolved to the empty set, which the deploy refuses outright with "matched no dbt nodes". Verified both ways on a fresh path: the deploy fails without this and succeeds with it. All three engines list `unit_test` among the accepted values. The restore's state lookup now carries the reason it keys on the principal and not the caller. That decision was in the design doc, the commit messages and a PR thread — everywhere except the line someone would change to "fix" it, which is where AGENTS.md asks for it and where I twice went wrong myself. Also: the `GraphRefresh` doc claimed a dynamic descriptor publishes its graph as the script's ownership, while `per_run_models` snapshots it under the job id and `publishes_ownership()` returns false — only a moved profile republishes. And `fn n` in dbt_profiles.rs had no callers left after `port_of` replaced it. * feat(dbt): offer the resume on a failed run, where the choice is relevant "Run again" prefills the arguments the run just used, so the obvious action after a failure rebuilds the whole project while the cheap resume stays invisible unless the reader knows dbt_command has a retry value. A finished run with failed or skipped nodes now says how many, and links to its own run form with the command already switched. A link rather than a submission: the caller still presses Run, so nothing about permissions or arguments changes. Deliberately not automatic. A resume is not a re-run — models are usually rebuilt because upstream data moved, not because they failed — so a schedule that resumed after a failure would leave every model that succeeded carrying yesterday's data while reporting success. * feat(dbt): put the command first, and offer the resume where the rebuild is Three things made rebuilding the whole project the path of least resistance after a failure, when resuming its failed and skipped nodes is what dbt offers: - `dbt_command` was LAST in the generated signature, under `limit`. The schema's `order` drives the run form, so the argument that decides what the run does sat below the ones that only narrow it. It leads now, and the signature test asserts the order rather than the set, with the reason it is pinned. - "Run again" prefills the arguments of the run it came from, so for a failed run it reproduces the rebuild. It now carries `dbt_retry_hint`, and the form shows what resuming would do instead -- without choosing it, so the caller decides. - Its dropdown offers "dbt retry with same args" above the plain re-run, for a failed dbt job only: a run that succeeded has no saved failure to resume. Verified in the browser: both dropdown labels render unclipped, the alert appears only on that path and disappears once the command is switched, and the schema of a redeployed script lists dbt_command first. * feat(dbt): describe the run arguments, and hide the ones a command ignores The form offered six fields at once with no indication that most of them apply to one command each. `retry` reuses the arguments of the run it resumes, so every override is ignored; `limit` belongs to `show`; `full_refresh` to `build`. Each argument now carries what it is for, and a `showExpr` for when it applies, which SchemaForm honours by hiding the field AND dropping its value from the payload. Selecting `retry` leaves the command alone on the form. Not modelled as a `oneOf`, which is the more precise shape: Windmill renders that as a variant selector whose value is a nested object keyed by a discriminator, so the arguments would stop being flat -- and the worker reads them by name, as do schedules, webhooks, the CLI, saved inputs and the retry state's own copy of them. * fix(dbt): a retry may name the run it means to resume The retry actions I added an hour ago appear on any failed run, but the saved failure is keyed by script and principal rather than by job — so from an older run's page they resumed whatever failed most recently, under a label promising "same args". Both now pass `dbt_retry_job`, and a retry naming a run the state no longer holds is refused with the id it does hold. Verified: with failures A then B saved, a retry naming A is refused and names B in the message, while a retry naming B resumes. Also from the same review: the `select` help advertised `state:modified`, which needs a comparison manifest this runtime supplies no `--state` for, and the openflow note still described a dbt script as a descriptor naming a git repo rather than the module bundle this ships. * fix(dbt): make both retry actions actually name their run `dbt_retry_job` is not a form field, so the banner's link to the run form dropped it and the retry resumed whatever failed last — the bug the argument exists to prevent, reintroduced by the affordance meant to use it. The banner submits the run itself now, as the "Run again" dropdown item already did. And an agent worker never applied the check at all: its `latest_job` is always None, so the local generation was accepted unconditionally. Both reviewers found this independently. The decision moved into `chosen_generation`, which the agent path shares, with the refusal naming the run the worker does hold. Pinned by a test that fails without the check, on the connection kind where it was missing rather than on the one that already worked. * fix(dbt): let an unchanged push be a no-op, and stop advertising a retry the form cannot aim Two of a dbt script's fields are DERIVED by the server after the no-op check runs: the lock, which only a dependency job can produce, and the schema, which comes from the descriptor and which no client can derive (windmill-parser-wasm has no dbt arm). The check compared what arrived instead of what would be stored, so every unchanged `wmill sync push` created another version and another dependency job -- the sync churn skip_if_noop exists to prevent. Both are now compared as stored, with the parent required to hold a lock so a deploy whose dependency job failed can still be retried by pushing again. Verified by A/B on a faithful payload: without this the same unchanged push creates a version, with it the existing hash comes back and the count holds. The run form's retry hint is gone. `dbt_retry_job` is not a form field, so a retry started there resumes the last failure of the script rather than the run the message pointed at -- the two buttons name their run, that path could not. Better to not offer it than to offer it wrong. Also reunites a comment with the derivation it describes, 40 lines below where an earlier edit of mine left it. * docs(dbt): an unchanged push no longer re-resolves dependencies The section told the reader that a byte-identical deploy re-pins a ranged dependency, which was true only because no-op detection was broken for dbt. Now that an unchanged push is skipped, moving a pinned resolution takes an actual change. * feat(dbt): offer the resume only on the run it would actually reach * fix(dbt): a refusal names both the run asked for and the one held * docs(dbt): say that show previews one node when several are selected * feat(dbt): a retry names the run it resumes, and the form can fill it in * refactor(dbt): make the run's command a oneOf carrying its own overrides * fix(dbt): refuse a command block that names no command, and keep state off retired paths * fix(assets): let copilot and the assets filter see warehouse tables * fix(dbt): drop the unused db handle from the retry-principal lookup * fix(dbt): stop a preview poll when the run page moves on * fix(dbt): refuse an escaping packages-install-path, re-arm the graph poll on navigation * docs(dbt): concurrent runs of one script are the script's to serialize * fix(dbt): bound what the log tailer holds in the worker process * fix(dbt): a stale prefill answer must not aim another script's retry * fix(dbt): a rename leaves the old path archived, so state must not be rewritten there * fix(dbt): keep a retry's routing inputs, and stop an app-embed token probing resumability * docs(assets): state the authorization the graph helper expects of its callers * fix(dbt): merge the retry into the fetched arguments, not the too-big placeholder * fix(dbt): the command block's label decides the command, not map order * fix(cli): an oversized dbt project file must not read as generated output * fix(dbt): the prefill names a run only when the caller may read it * fix(cli): refuse an oversized dbt file before anything reads its body * fix(dbt): fold table paths as the deploy does, and preview with fetched args * fix(dbt): stream an engine archive to disk instead of holding it in the worker * refactor(assets): the dbt asset kind is dbt://, keyed on the relation * chore: regenerate the auto-generated prompts for the dbt asset kind * fix(assets): rename the kind at the callers that request it, and refuse an unknown one * feat(dbt): show the project's models under the run form on the script page * fix(dbt): the rename missed the dbt-edge membership keys, so the DAG lost every edge * fix(dbt): a preview must run the dbt writer, with the form's arguments * feat(dbt): the models graph takes the bottom of the script page, as a flow's does * fix(dbt): an unknown outcome is not a pass, and a preview keys on its arguments * feat(dbt): a warehouse is configured on the workspace, named like the lake * feat(dbt): asset identity keys on the warehouse name, resolved for agents too * feat(dbt): warehouses are configured in workspace settings * feat(dbt): the descriptor lives in the project and is optional * docs(dbt): the warehouse is a workspace setting and the descriptor is optional * fix(dbt): the warehouse route names its capture, and the editor stores the map * test(dbt): pin the warehouse setting round-trip and the job-scoped route * fix(dbt): a project-owned profile must authorize the warehouse it names * fix(dbt): the settings tab imports TextInput from where it lives * fix(dbt): a warehouse name is validated wherever one enters, and an absent descriptor is not a diff * fix(dbt): the settings inputs pass their placeholder the way TextInput takes it * fix(dbt): pushing a project that has no descriptor is not an error * chore(dbt): keep the module suffix private to its module * fix(dbt): warehouses survive a fork or rename, and an empty descriptor is never a file * fix(dbt): a workspace rename carries its dbt graph with it * fix(dbt): a rename takes the run state with it, not only the graph * chore(dbt): drop the query-cache entries the settings edits orphaned * docs(dbt): state the three rules that keep an absent descriptor absent * fix(dbt): an unreadable descriptor is an error, and an emptied one is a deletion * feat(dbt): the warehouse is unpermissioned, like the workspace bucket * feat(dbt): an agent worker's run reports its per-model state too * docs(dbt): how an agent worker reports what it ran * feat(dbt): a project-owned profile reports its database, so its models share nodes * style(dbt): rustfmt the files this branch touched * fix(dbt): drop the job id write_profiles no longer reads a resource with * fix(dbt): a preview reads the graph's edges and sends the model, not the cache key * fix(dbt): a deleted producer explains itself instead of a bare 404 * fix(dbt): one warehouse resolution path, interpolated against the job * fix(dbt): a project with no descriptor pulls and opens in dev mode * feat(dbt): one file tree, editable, and Test builds the model you have open * fix(dbt): a preview asks the writer's own graph, and lfs keeps its omission * fix(dbt): metadata stays beside the project, and only a model narrows a build * fix(dbt): a descriptor-less project previews, and dev reloads the whole bundle * fix(dbt): forks carry the graph, and a vanished project stops the push * fix(dbt): no-auth resolves warehouses, and a preview runs the version it read * fix(dbt): a retry recognizes its own run when the profile carries a job token * fix(dbt): the project marker is not deletable and the descriptor path is reserved * fix(dbt): a templated adapter type is refused, and a removed project archives * perf(dbt): the manifest lands in batches, and .py models are creatable * fix(dbt): only a 404 excuses a failed archive, and edges are covered * fix(dbt): a python model narrows the build, and a failed removal is not silence * fix(dbt): a settings name may be any word, and the reserved path is stated once * fix(dbt): an absent descriptor needs its project, and the form matches the run * fix(dbt): a bundle is dbt by language, and both job routes agree who may call * docs(dbt): every identity doc names the warehouse, as the code does * perf(dbt): an agent posts its run's progress once, and show cannot touch a seed * fix(dbt): bound what a warehouse may be named, and say who may ask * fix(dbt): a run's graph withholds what the project's author wrote, and dbt_project.yml is rendered before it is read * feat(dbt): withhold dbt from the language picker until it has its own editor * fix(dbt): refuse a path claimed by both a dbt project and an ordinary script * test(dbt): the bundle assertion is a set, not an order * fix(dbt): a project's generated dirs are re-read when its config changes * test(dbt): assert the synthesized descriptor without assuming a separator * fix(dbt): both push paths refuse a shared path, and every writer sweeps its progress rows * fix(dbt): preview only what dbt show can select, and a retry keeps its command block * fix(dbt): an identifier carrying a path separator gets no asset key * fix(dbt): no engine ships in an image, and show may name only one node --------- Co-authored-by: Claude Opus 5 <noreply@anthropic.com>	2026-08-01 11:58:05 +00:00
Ruben Fiszel	bfc3f5242a	feat: sync data table migrations to git, gated by a new object type (#10436) ... * fix: auto-sync data table migrations to the linked git repo * fix: deploy data table migrations when a data table is renamed or deleted * fix: hold new data table names to the git-sync-safe charset * fix: reject leading-dot data table names and warn on unsyncable legacy names * chore: update ee-repo-ref to 15c9eef2a4f867eb90d841aee1ce762f4b725589 This commit updates the EE repository reference after PR #702 was merged in windmill-ee-private. Previous ee-repo-ref: e2fb073a3d0057683666424e463b2ce423664caa New ee-repo-ref: 15c9eef2a4f867eb90d841aee1ce762f4b725589 Automated by sync-ee-ref workflow. * feat: make data table migrations a git-sync object type with its own toggle * fix: never let an untracked checkout delete data table migrations on push * fix: confirm ambiguous data table migration deletions instead of dropping them * fix: settle ambiguous migration deletions before the dry-run preview prints * fix: restore the split shared-UI comment and count migration records in prompts * chore: keep the deletion-safety doc block attached to its function * fix: trust git history, not the working tree, for migration deletions * fix: scope migration history to HEAD, detect shallow clones and subdir roots * chore: give the unattested-history case a remedy that applies to it * chore: pair each unattested-history cause with its own remedy * fix: treat a sparse checkout as unattested history for migration deletions * fix: normalize the sparse-checkout boolean and give it a remedy that works * chore: describe both shapes of unattested migration history * chore: update ee-repo-ref to a786cd42b5aaf0aa6789fbb723d956560f93b1b3 This commit updates the EE repository reference after PR #703 was merged in windmill-ee-private. Previous ee-repo-ref: 4f312642b5d8fd37ab5e20473a011d6f1d299cf6 New ee-repo-ref: a786cd42b5aaf0aa6789fbb723d956560f93b1b3 Automated by sync-ee-ref workflow. --------- Co-authored-by: windmill-internal-app[bot] <windmill-internal-app[bot]@users.noreply.github.com>	2026-07-31 23:33:47 +02:00
Ruben Fiszel	81b23a2ba0	feat: make the fork lineage the only deploy relationship (#10410) ... * feat: make the fork lineage the only deploy relationship `workspace_settings.deploy_to` (2023) and `workspace.parent_workspace_id` (2025) both expressed "which workspace does this one deploy into". Fork creation and dev-workspace attach seeded both, but nothing kept them in agreement, so every reader picked one and they disagreed. Drop `deploy_to`. A migration folds surviving pairs into the lineage: a sole claimant on a target with no dev workspace becomes that target's dev workspace and keeps its own job tags, while many-to-one pairs become plain forks. Pairs that the lineage cannot express -- dangling target, self-reference, chain, mutual -- are reported and left unlinked. Job tags were never lineage-aware: `per_workspace_tag` mapped any parented workspace to its parent while `$workspace` interpolated the raw id, so a fork running a script tagged `<tag>-$workspace` produced a tag no worker serves and the job queued forever. Both paths now resolve to the nearest ancestor whose id an admin would provision workers for. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix: preserve unconvertible deploy links and sweep tag caches on reparent Review findings on the deploy_to unification: - convert chains instead of discarding them, and keep whatever the lineage cannot express in workspace_deploy_to_unmigrated so the down migration can restore it - ignore soft-deleted workspaces when choosing between a dev workspace and a plain fork; an archived claimant was demoting live pairs - mirror attach_dev_workspace's git-sync strip, which the migration skipped - sweep the tag cache over whole subtrees on rename and delete: tag resolution now walks ancestors, so a nested fork kept a tag nothing serves - call a dev workspace a dev workspace in the settings copy - redirect a root away from ?tab=deploy_to instead of rendering an empty target Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix: detect lineage cycles and record archived links in the deploy_to migration Second review round on the unification: - detect cycles over the lineage as it would exist after conversion, not over the deploy_to graph alone: a root whose target was one of its own forks closed a loop that no deploy_to edge revealed - record an archived source's link instead of filtering it out entirely, which dropped it with the column - treat a fork whose deploy_to merely repeats its parent as redundant rather than reporting every pre-existing fork as unmigrated - read the row count from the lineage update rather than the git-sync one - sweep the tag cache when archiving a dev workspace, the last site that mutates is_dev_workspace without one Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix: resolve $workspace on preprocessed flow tags regardless of $args Third review round on the unification: - a flow tag containing only `$workspace` skipped interpolation entirely on the preprocessed path, because the branch that ran it keys on `$args`. The raw tag was written back and named a queue no worker serves. Resolve `$workspace` before the branch and leave `$args` to it. - record the new table's foreign key in the schema summary - describe what the archive tag sweep actually does: the dev flag is cleared for any archived workspace, which is why it is unconditional Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix: keep the deploy_to leftovers table only when it holds something * fix: sweep tag caches on archive only where the dev flag actually changes * feat: broadcast lineage changes and walk ws_specific ancestors only - propagate tag-cache invalidation across processes over notify_events: the cache is per-process, so replicas kept resolving stale lineage for the TTL. The listener clears the whole cache rather than tracking ids, since a single mutation invalidates an unbounded set of descendants and lineage changes are rare admin actions. - narrow list_ws_specific_versions to ancestors: walking down as well made a root fan out over its entire live fork subtree, and each member costs an identity lookup plus an RLS switch and probe. Ancestors are bounded by the fork depth limit. - probe the leftovers table unqualified so rollback restores on a PG_SCHEMA install, where search_path is not public - drop the nativets client method for the removed edit_deploy_to endpoint Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix: let a prod see its dev workspace in ws_specific, and stop the walk oscillating Descending into plain forks made a root fan out over its whole live fork subtree, but a dev workspace is the paired editable environment rather than a throwaway copy, so a prod should still see it. There is at most one per parent and attach rejects nested dev chains, so that edge stays bounded. The edges run both ways, so the recursion never converged: it bounced parent<->dev until the depth cap on every call, 33 rows for a two-member set. A visited-path guard ends the walk when nothing new is reachable. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix: keep dev pairings unnested, gate the delete broadcast, cover the ws_specific walk Fifth review round: - a root that already owns a dev workspace no longer converts: linking it under its deploy target would leave that dev nested beneath a fork, the shape attach_dev_workspace refuses to create. The link is preserved instead. - broadcast a lineage change on delete only when descendants are orphaned. Deleting a leaf, which ephemeral fork churn does constantly, changes nobody else's resolution and was making every replica drop its whole tag cache. - call list_ws_specific_versions in a test. plpgsql defers everything past a raw parse to the first call, so replaying the migration only proved it parses. - use unwrap_or_default for the descendant sweeps, which run after the transaction has committed; a transient failure must not fail the request - trim the traversal comment to the four-line limit Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix: cache the renamed tally query and clear instance alerts on conversion The integration test's query was never cached: `cargo sqlx prepare` without --all-targets skips test targets entirely, and renaming its fixture workspace changed the query text. Regenerated with --all-targets --features all_sqlx_features,private, which is what lets the EE-gated otel test compile. Also from review: - clear error_handler_fallback_to_instance_alerts on converted workspaces. Dispatch ignores it once a parent exists, but the settings page keeps submitting the stored true, which the API rejects on a fork. - restore the schema summary row to the file's name: columns format and put it back in alphabetical order Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix: never cache an unresolvable tag workspace, and unadvertise the removed endpoint - lookup_tag_workspace cached a "no row" result as self-resolution. A rename resolves the new id before its row lands, so a fork could be pinned to its own wm-fork-* id -- which nothing serves -- for the whole TTL, and its schedules kept re-pushing onto that dead tag. Fall back for the call without caching, matching how the error path already behaved. - change_workspace_id swept its children but never itself. Sweep the new and old ids and broadcast unconditionally, since a rename always changes lineage. - openapi-deref.{json,yaml} are served to clients via include_str!, so they were advertising edit_deploy_to after it started 404ing. The audit-action enum keeps the entry: historical rows still carry it. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix: align the served YAML spec with the JSON one and correct two comments - the YAML deref lost the removed path but kept deploy_to on get_settings, so the two served specs disagreed. Both are now identical. - the rename-sweep comment blamed cached-unresolvable lookups, which the same commit stopped caching. The real reason is that workspace ids are reclaimable, so a new id can carry a previous occupant's resolution. - the instance-alert comment claimed the settings page submits the stored true and gets a 400. It hides the option on a fork and sends false; the hazard is the value outliving the pairing and re-enabling alerts after a detach. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * chore: update ee-repo-ref to 82da6cb2bafeda18acd6b70c599013a12117ecb0 This commit updates the EE repository reference after PR #694 was merged in windmill-ee-private. Previous ee-repo-ref: f9ddf6a75aa13d1c13a3d7216a361a96f75ca435 New ee-repo-ref: 82da6cb2bafeda18acd6b70c599013a12117ecb0 Automated by sync-ee-ref workflow. * fix: grant the deploy_to preservation table to the windmill roles * test: drop the one-shot migration tests, keep the ws_specific execution guard The two conversion tests replayed the migration against the fully-migrated schema, which is not how it runs -- in production it runs mid-sequence against the schema as of that point. A later migration touching workspace or workspace_settings would break them without breaking anything real, and sqlx checksums already freeze a released migration. They earned their keep finding the archived-claimant and nested-dev cases during development; there is nothing left for them to guard. list_ws_specific_versions is different: it is live, no caller exercises it, and plpgsql only parses a function body until first call. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix: invalidate a reclaimed fork id cluster-wide without flushing every entry Gating the delete broadcast on orphaned descendants stopped leaf churn flushing every replica, but fork ids are reclaimable: the deleting process invalidated locally while every other replica kept the old parent for the TTL, so a job pushed in a recreated fork routed to the previous parent's tag. The broadcast payload now carries meaning. A workspace id drops that one entry, used for leaf deletion where exactly one id changed what it denotes. The `*` sentinel drops everything, used for attach, detach, archive, rename and deletions that orphan descendants -- reshaping a subtree no single id names. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * docs: name the right broadcast for each invalidation case * docs: attach does invalidate the tag cache; the resolver walks the whole chain --------- Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com> Co-authored-by: windmill-internal-app[bot] <windmill-internal-app[bot]@users.noreply.github.com>	2026-07-30 14:20:27 +00:00
Ruben Fiszel	68debab877	feat(triggers): add AMQP (RabbitMQ) trigger via lapin (#10230) ... * feat(triggers): add AMQP (RabbitMQ) trigger using the lapin library Fixes WIN-2214 Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * chore(triggers): defer AMQP cross-workspace deploy pending utils-internal publish Revert the amqp_trigger additions to the shared windmill-utils-internal TriggerDeployKind and the frontend cross-workspace deploy adapter: the frontend installs the published npm package, which lacks the new kind until a release is cut. AMQP create/edit/delete/list/sync/capture are unaffected (they use local types); only cross-workspace deploy/merge of AMQP triggers waits on the package bump. Also document the at-most-once ack in the consumer loop. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(triggers): address AMQP review — at-least-once ack, workspace cascade, contracts - ack AMQP deliveries only after successful dispatch; nack+requeue on failure - add ON DELETE CASCADE workspace FK so amqp_trigger rows are cleaned on workspace deletion (and the listener stops) - fix the /amqp_triggers/test OpenAPI body and add amqp_trigger to WorkspaceDiffRow.kind - register AMQP in the generated workspace trigger tool (create_trigger) - drop banned $bindable defaults on optional props in the config section - add build_uri unit tests (encoding, ports, vhost) Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(triggers): stop AMQP poison-message loop and reconnect on transient drops Chaos testing against a live RabbitMQ broker showed the previous nack(requeue) + immediate re-poll spun a tight redelivery loop (~1000 critical-error reports/sec) on a poison message, and any connection blip permanently disabled the trigger (lapin has no built-in reconnect). - on dispatch failure: nack+requeue then stop consuming; the listener framework re-lists the trigger after its ping goes stale (~15s), backing redelivery off to that cadence instead of a tight loop (verified: rate dropped from ~1000/s to ~1 per ~26s, message preserved) - on connection/stream error: stop and let the framework reconnect instead of disabling; persistent failures are still disabled via get_consumer (verified: a forced connection close now auto-reconnects and resumes) - finish the AI create-trigger action wiring for AMQP: add amqp to CreatedResourceTriggerKind, the action-card registry, and the drawer registry so the result card renders and its "Open" action works Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(triggers): complete AMQP frontend registries and defer merge rows - add amqp to capturableTriggerTypes (so AmqpCapture mounts), the Runs jobTriggerKinds filter, and CLOUD_DISABLED_TRIGGER_TYPES - wire AMQP into global AI chat mode: TRIGGER_KINDS, the request union, writeTriggerSchema, triggerServices, and the draft adapter - stop emitting actionable AMQP fork-comparison rows (revert amqp_trigger from TRIGGER_OR_SCHEDULE_TABLES) since cross-workspace deploy is deferred until windmill-utils-internal is published — avoids a deploy that fails with "Unknown kind: amqp_trigger" - use design-system TextInput instead of raw <input> in the config section Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(triggers): complete AMQP session/draft registries and constrain prefetch - add amqp to the session-deploy, draft-compare, preview-router, and copilot workspace-item registries so AMQP drafts/deploys/nav/path resolution work - include amqp_count in the MoveDrawer attached-trigger rename warning - replace the raw prefetch <input> with a design-system TextInput bounded to an integer 1-65535 (backend u16) and block save on invalid values Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(triggers): make AMQP disconnect/reconnect consistent with the Kafka trigger lapin, like rdkafka, has no transparent reconnect, so the AMQP listener now mirrors the Kafka trigger's explicit reconnect loop instead of relying on the framework re-list (which disabled the trigger once get_consumer failed on a sustained outage): - get_consumer returns cheaply; consume owns a (re)connect loop that retries with a 30s backoff, reports a critical error every 10 failed attempts, and reports a recovered critical error once it reconnects — never disabling the trigger on a connectivity failure - a consumer/stream error breaks out to reconnect rather than disabling - dispatch failure still nacks+requeues (at-least-once) with a short backoff to avoid a tight poison-message loop, keeping the connection alive Verified against a live RabbitMQ broker: killing the broker keeps the trigger enabled and retrying (attempt N), and restarting it auto-reconnects (logs "reconnected after N attempts") and resumes dispatch. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(triggers): complete AMQP capture registries and constrain prefetch contract - add the 'amqp' case to triggerKindToTriggerType so opening the AMQP editor from a capture button no longer throws "Unknown TriggerKind: amqp" - register AmqpIcon in CaptureTable's icon map and add an AMQP entry to the script/flow CaptureButton menu - bound the OpenAPI prefetch_count to an integer 1-65535 (matches the Rust u16) and regenerate clients/prompts - require a non-empty exchange name when the exchange binding is enabled - build_uri: fall back to "/" on a blank vhost and bracket IPv6 hosts (+ tests) Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(triggers): wire AMQP into pipeline graph, git-sync, and preprocessor types - asset_graph: discover attached amqp_trigger rows and emit an AMQP TriggerEdge so AMQP triggers render (and can be opened/deleted) on the data-pipeline canvas - frontend pipeline graph: add amqp to NativeTriggerKind, the add-trigger menu, node presentation, event-trigger set, annotation keywords, and the editor/service registrations - git-sync: add the amqp_trigger include pattern (+ test) so an AMQP git-sync deployment stages only its .amqp_trigger.* file, not an unrelated same-path object - preprocessor starters: add the AMQP event to the generated TS/Python/PHP trigger event types (kind/payload/exchange/routing_key/queue_name/redelivered/ delivery_tag) Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(triggers): finish AMQP pipeline/parser wiring, prefetch validation, source lists - fix a stray edit that corrupted the pre-existing MqttTriggerEditor import ($lib/... path) in PipelineTriggerEditors.svelte - reject prefetch_count = 0 server-side in validate_config (RabbitMQ treats 0 as unlimited) and defensively skip basic_qos(0) in build_consumer (covers the capture path that bypasses CRUD validation) - recognize `// on amqp` in the canonical parser (TriggerSpec::Amqp) and add amqp to the CLI non-autorun/event-trigger sets so a pipeline cascade never runs an AMQP-only node as a manual root without an event - add amqp to the preprocessor intro lists and both pipeline AI instructions Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(triggers): reject zero AMQP prefetch in all paths and finish guidance lists - extract a shared validate_amqp_options used by both CRUD validate_config and build_consumer, so capture configs (which bypass CRUD validation) also reject prefetch 0 instead of silently connecting with an unlimited buffer (+ unit tests for 0/1/65535/None) - add AMQP to the main script-writing preprocessor-sources prompt and the CLI triggers-skill guidance list Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * docs(triggers): de-duplicate AMQP prefetch comment and fix GET response text - keep the zero-prefetch rationale only on the shared validate_amqp_options doc; drop the redundant call-site comments - correct the getAmqpTrigger OpenAPI 200 description ("deleted" -> "retrieved") Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * chore: update ee-repo-ref to aaa6cb89b05b76139252c64f057e53b94d12ac60 This commit updates the EE repository reference after PR #680 was merged in windmill-ee-private. Previous ee-repo-ref: 5da5fd65aca9594b2611837a52e4677b544b0380 New ee-repo-ref: aaa6cb89b05b76139252c64f057e53b94d12ac60 Automated by sync-ee-ref workflow. * chore(migrations): consolidate the four AMQP migrations into one The table and the three enum ADD VALUE statements (trigger_kind, job_trigger_kind, draft_kind) are one atomic feature. ALTER TYPE ... ADD VALUE runs inside the migration transaction on PG >= 14 (Windmill's minimum) since the amqp_trigger table doesn't reference those enum types, so they can share a single migration instead of four. Verified applying cleanly in a single transaction on a fresh DB. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Co-authored-by: windmill-internal-app[bot] <windmill-internal-app[bot]@users.noreply.github.com>	2026-07-21 15:10:29 +00:00
hugocasa	51d8db6602	feat: automatic git-to-windmill sync (polling, webhooks, in-app PRs + checks) (#9552) ... * docs: add design doc for automatic git-to-windmill pull sync * docs: add migration plan and implementation phases to git-sync pull design * feat(git-sync): add auto_pull settings schema and pull enqueue primitive Adds AutoPullSettings/AutoPullMode/AutoPullStatus on GitRepositorySettings (workspace_settings.git_sync JSONB), the GIT_SYNC_PULL_SCRIPT_PATH constant, and should_pull/effective_poll_interval_s helpers with unit tests. Exports the EE enqueue_git_pull_job primitive. Foundation for repo→Windmill auto-pull. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(git-sync): poll repos and auto-pull new commits into the workspace Phase 1 of automatic repo → Windmill sync. A monitor task (EE-licensed, single-replica via advisory lock) git ls-remotes each auto-pull-enabled repository ~every minute and enqueues a pull when the tracked branch moves, reusing the {workspace_id}:git_sync concurrency key so pulls serialize with in-flight push commits. - windmill-store: background (no-authed) resolver get_git_repo_head_for_autopull that resolves the repo resource (incl. $var: refs) and ls-remotes; GitHub-App repos are skipped here and will sync via webhooks (phase 2). - monitor.rs: poll/reconcile/persist with optimistic sha advance and failure status; targeted jsonb update so concurrent settings edits aren't clobbered. - edit_git_sync_repository: preserve server-owned auto_pull state on UI save. - openapi: AutoPullSettings/AutoPullMode/AutoPullStatus + auto_pull field. - frontend: per-repo "Automatically deploy changes from Git" toggle with last sync status; demote the GitHub Actions link to an advanced CI option. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(git-sync): wire webhook lifecycle + receiver; share reconcile logic OSS side of phase 2 auto-pull webhooks: - edit_git_sync_repository creates/removes the repo webhook on save (EE-gated, best-effort → falls back to polling). - monitor poller now delegates to the shared windmill_git_sync reconcile/persist helpers (also used by the webhook receiver), removing duplicated logic. - export the shared reconcile/persist/failure helpers; bump EE ref. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * chore(git-sync): bump EE ref for phase 3 in-app PR creation Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(git-sync): show webhook vs polling status on the auto-pull toggle When a repo has an active webhook (auto_pull.webhook_id set), the status line reads "instant via webhook"; otherwise it reads the ~1-minute polling cadence. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(git-sync): post PR diff check on dry-run completion (phase 4) Worker completion hook in process_completed_job: when a DeploymentCallback job carrying the __git_sync_pr_check marker finishes, parse the dry-run SyncResponse and patch the GitHub check run with the diff summary (success/neutral/failure). Export enqueue_git_pull_dry_run; bump EE ref. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * chore(git-sync): bump EE ref (drop unused GHES webhook_secret) Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * revert(git-sync): defer phase 4 PR diff checks (OSS side) Remove the worker completion hook that posted the PR check run, drop the enqueue_git_pull_dry_run re-export and the orphaned sqlx cache, bump EE ref. Phases 1-3 (polling, webhooks, in-app PR creation) are unaffected. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * Revert "revert(git-sync): defer phase 4 PR diff checks (OSS side)" This reverts commit 0137d3ca48. * chore(git-sync): point EE ref at restored phase 4 commit Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * chore(git-sync): bump EE ref for clone_ref dry-run Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * chore(git-sync): bump init-repository hub script to v28784 Picks up the clone_ref param (windmill-integrations#158) so the phase 4 PR-check dry-run can clone the PR head. Backward compatible; manual pull/push and the automated pull/poller/webhook all move to the same published version. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * chore(git-sync): bump EE ref for auto-pull admin-permissioning fix Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * chore(git-sync): bump EE ref for superadmin pull fallback Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(git-sync): refresh auto-pull tooltip; bump EE ref for webhook secret encryption The auto-pull toggle tooltip claimed GitHub App repos would sync via webhooks "in a future update"; webhook delivery now works, so describe the webhook-vs-polling behavior accurately. Bump the EE ref to pick up encrypting the webhook HMAC secret at rest. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * fix(git-sync): poll app-backed repos in auto/polling mode The auto-pull poller skipped app-backed repos (the ls-remote head check can't authenticate a tokenless URL), so auto- and polling-mode app repos never synced when their webhook wasn't live. Wire the poller to fetch the head via the GitHub API for app repos and reconcile. Bump the EE ref. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * feat(git-sync): auto-pull UI — direction split, delivery mode, fallback notice Reorganize the repository card into two clearly labeled directions: "Push to Git on deploy (Windmill → Git)" and "Pull from Git (Git → Windmill)". In the pull section: - new connections default to auto-pull enabled (webhook with polling fallback); existing repos load with auto-pull off and are unchanged - a Delivery selector chooses "Webhook with polling fallback" or "Polling only (air-gapped)" - a notice surfaces webhook_error when delivery falls back to polling - a reminder to remove any pre-existing GitHub Action that pushed into Windmill, to avoid conflicting double-syncs Adds the webhook_error field to AutoPullSettings (+ openapi) and bumps the EE ref. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * feat(git-sync): clearer push indicator + gate webhook delivery to app repos - Push-on-deploy is shown with a check icon + concise line (via the shared GitSyncModeDisplay, restyled from the oversized "Sync:" text); the setup wizard reuses it without the check (pre-save preview). - The delivery-mode selector only shows for GitHub App-backed repos; token-based repos show a "webhooks require the GitHub App (managed or GHES)" note with a docs link and poll instead. Bumps the EE ref. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * feat(git-sync): fork auto-sync (phase 5) + live deploy check (phase 6) Phase 5 — fork auto-sync configured at the parent (replaces the *-to-forks GitHub Actions): - Add fork_open_prs + fork_pull_sync to GitRepositorySettings (openapi + UI). - UI: two "Forks of this workspace" toggles in the repo card, gated on app-backed and not-a-fork; serialize the flags on save. - On fork creation, strip the inherited auto_pull block (and fork_* flags) from the copied git_sync repo: a fork must not carry the parent's webhook id (it would delete the parent's hook on disable) or self-poll on top of the parent's fan-out. Push-direction config + installation are still inherited unchanged. Phase 6 — live deploy status check on the commit (Cloudflare-style): an in-progress "Windmill" check on the head commit that flips to "Deployed N changes"; completion handled by the generalized git-sync check hook. Bump EE ref for the phase 5-6 EE implementation. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01PP5gBSPfo1YtkL1sWVAjJm * chore(git-sync): bump EE ref for PAT auto-pull mode normalization Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01PP5gBSPfo1YtkL1sWVAjJm * fix(git-sync): address PR review findings - webhook_secret: redact from the settings API response and Debug output (still persisted encrypted); it's a server-only HMAC key the UI never needs. - poller: honor each repo's effective poll interval (relaxed ~10 min when a webhook is live) instead of probing every ~60s tick. - settings save: roll back a just-created webhook if the settings transaction doesn't commit, so a failed save can't orphan a hook. - auto-pull head check: fail SSH remotes with an actionable message (background polling has no SSH identity) instead of a confusing ls-remote error. - deploy/PR check summary: a pull result carrying neither changes nor a settings diff now falls back to the unsummarized path instead of a false "in sync". - UI: reset isGithubApp on resource change / failed fetch so webhook + fork controls can't show for the wrong repo. - tests: cover parse_git_sync_changes and format_change_list edge cases. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01PP5gBSPfo1YtkL1sWVAjJm * fix(git-sync): correct feature gating for OSS builds - monitor.rs: keep the AUTO_PULL_LAST_POLL static, slack const, and poll_git_auto_pull_inner all behind #[cfg(feature = "private")] (an inserted static had split the cfg off the function, ungating it in OSS builds). - edit_git_sync_repository: the webhook create/rollback block references windmill_common::git_sync_ee (private module), so gate it on all(enterprise, private) instead of enterprise only. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01PP5gBSPfo1YtkL1sWVAjJm * chore(sqlx): cache workspace_diff query pulled in from origin/main Re-merged origin/main (advanced past the earlier merge); regenerate the offline sqlx entry for the new workspace_comparison test query so SQLX_OFFLINE builds (cargo_test) pass. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01PP5gBSPfo1YtkL1sWVAjJm * fix(git-sync): address Codex review findings (webhook cleanup on delete) - Deleting a git-sync repository now tears down its managed GitHub webhook (deletion bypassed the sync_repo_webhook lifecycle, orphaning the hook so GitHub kept delivering to the instance). - Worker completion hook rolls back the optimistic auto-pull sha on job failure (OSS side of the EE change) + caches the new marker query. Bump EE ref. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01PP5gBSPfo1YtkL1sWVAjJm * fix(git-sync): delete repo webhook after the removal commits Codex re-review nits: - delete_git_sync_repository deleted the webhook before the settings transaction committed; a failed save would then leave the repo pointing at a hook that no longer exists (sync_repo_webhook treats a set webhook_id as live and won't recreate it). Capture the hook id, commit the DB removal, then delete the hook. - Reword a fork-copy comment to drop drafting-history wording per AGENTS.md. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01PP5gBSPfo1YtkL1sWVAjJm * fix(git-sync): reconcile the edit-path webhook after the settings commit Codex nit: edit_git_sync_repository ran sync_repo_webhook before the transaction committed. The rollback only covered created hooks, but sync_repo_webhook also deletes a hook on disable/switch-to-polling — a commit failure then left the DB with a webhook_id whose hook was already gone (and it wouldn't be recreated). Save + commit first, then reconcile the webhook against the durable config and persist any hook id/secret change (best-effort). Bump EE ref. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01PP5gBSPfo1YtkL1sWVAjJm * fix(git-sync): preserve webhook secret on whole-config save + default on visible add Codex nits: - edit_git_sync_config saved the client config verbatim, so the webhook_secret redacted from the GET response would be dropped (breaking delivery). Preserve server-owned auto-pull state (webhook id/secret, synced sha, last status) per repo from the existing settings, matching edit_git_sync_repository. - addSyncRepository (the visible add path) didn't set the auto_pull default, so new sync repos added from the UI came up with auto-deploy off. Match addRepository's default (webhook + polling fallback). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01PP5gBSPfo1YtkL1sWVAjJm * refactor(git-sync): drop fork_pull_sync (parent-level keep-forks-in-sync) Removes the "Keep forks in sync with the tracked branch" toggle and its fan-out. Pulling the tracked branch straight into every fork was the inconsistent piece; the consistent model is per-fork branch sync (each fork tracks its own wm-fork/** branch), which is a separate follow-up. fork_open_prs is kept. Also tightens the fork toggle-section spacing. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01PP5gBSPfo1YtkL1sWVAjJm * fix(git-sync): detect dev workspaces in CLI fork branch derivation isForkWorkspace / computeGitSyncDeployBranch keyed off the wm-fork- id prefix. Dev workspaces are forks with a custom, prefix-less id, so their wm-fork/** branch was never derived or created. Detect them via parent_workspace_id too (which the backend already passes), mirroring the backend's `parent.is_some() || wm-fork- prefix` rule. Pairs with the hub-script clone-flag fix (windmill-integrations#163); both take effect once the CLI is released and the pinned version is bumped. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01PP5gBSPfo1YtkL1sWVAjJm * fix(git-sync): reconcile webhooks on full-config save edit_git_sync_config preserved server-owned webhook fields but never created or deleted the managed GitHub webhook, so enabling auto-pull through the whole-config endpoint only polled, and disabling or removing a repo left an orphan hook still delivering. Mirror the per-repository endpoint: after the commit is durable, reconcile every saved repo's webhook (sync_repo_webhook) and delete the hooks of repos the save removed, including the clear-whole-config case. Addresses the Codex nit. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01PP5gBSPfo1YtkL1sWVAjJm * fix(git-sync): address Codex nits (webhook orphan on cleared auto_pull, fork detection) - edit_git_sync_config: also delete a repo's old webhook when the save drops the repo OR clears its auto_pull. Webhook fields are only preserved onto a Some auto_pull, so a save that present-but-clears a repo would otherwise orphan its hook. - GitSyncRepositoryCard: isFork now uses parent_workspace_id OR the wm-fork- prefix (was AND), matching the backend/CLI rule, so prefix-less dev workspaces are detected as forks and don't show the parent fork-PR toggle. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01PP5gBSPfo1YtkL1sWVAjJm * docs(git-sync): update design doc for the dropped fork_pull_sync Phase 5 documented "Keep forks in sync with the tracked branch" (fork_pull_sync) and its fan-out as implemented; that feature was removed. Rewrite the section to reflect what ships (fork_open_prs), note the drop + the per-fork-branch follow-up, and remove the stale fan-out mentions elsewhere. Addresses the Codex nit. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01PP5gBSPfo1YtkL1sWVAjJm * fix(git-sync): redact webhook secrets from workspace export; fix doc endpoints - Export (P1): strip the server-owned auto_pull state (webhook secret/id/error + synced sha + last pull status) from git_sync before it is written into an export's settings.json for both settings formats. The HMAC webhook secret must never leave the server (matching the GET-settings redaction), and a re-imported workspace must not inherit another install's hook/sync state. - Docs: the webhook receiver is a single per-workspace endpoint /api/w/{workspace}/github_app/webhook (host-aware for managed + self-managed); update the stale push_webhook/{id} and instance-global /api/github_app/webhook references. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01PP5gBSPfo1YtkL1sWVAjJm * fix(git-sync): skip deleted/archived workspaces in the auto-pull poller The poller scanned workspace_settings directly, so an archived (soft-deleted) or renamed-away workspace — whose settings row persists — kept polling and could enqueue a pull into a dead workspace. Join workspace and require NOT deleted. The EE webhook receiver gets the same filter (ee ref bumped). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01PP5gBSPfo1YtkL1sWVAjJm * fix(git-sync): never trust client-supplied server-owned auto-pull fields Both write endpoints (edit_git_sync_repository, edit_git_sync_config) persisted caller-supplied auto_pull.webhook_id / webhook_secret / webhook_error / last_synced_sha / last_pull_status when adding a repo or newly enabling auto-pull, letting a client inject a webhook id/secret or fake sync state. Strip those server-owned fields from the request up front; existing repos re-derive them from the DB (carried over), new ones start clean and the server (re)creates the webhook. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01PP5gBSPfo1YtkL1sWVAjJm * fix(git-sync): merge webhook fields post-commit instead of clobbering the row The post-commit webhook reconcile in edit_git_sync_repository and edit_git_sync_config wrote the whole pre-reconcile git_sync snapshot back after the main save committed. A concurrent git-sync edit or poller status write that landed in the gap could then be dropped by the stale snapshot. Re-read the current row and merge only the reconciled webhook id/secret/error for the repos the reconcile actually changed. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01PP5gBSPfo1YtkL1sWVAjJm * feat(git-sync): parent-managed fork sync + PR-on-deploy toggles Fork sync (push-on-merge-to-forks parity): a parent-level auto_pull.sync_forks toggle routes changes on each fork's wm-fork/** branch into that fork workspace, via the parent's existing webhook and one extra fork-heads listing per poll tick (git ls-remote pattern for token repos, git/matching-refs for app-backed). Fork state is a server-written status-only auto_pull blob on the fork's own repo entry; the fork's card shows a read-only "managed in the parent workspace" line with its branch and last pull status. Dev workspaces (prefix-less ids) use the same branch parsing (unit-tested in windmill-common). PR-on-deploy: opening PRs for Windmill-pushed branches moves into the deploy pipeline, per repo toggle (promotion_open_prs on the promotion repo; parent-level fork_open_prs for fork deploys). The push job carries a marker and the job-completion hook derives the pushed branch (helper unit-tested against the CLI formula) and opens the PR outbound, so it works without inbound webhooks; the webhook-side wm_deploy PR arm is removed. The documented open-pr-* GitHub Actions remain valid alternatives (PR creation is idempotent). Fork guards: promotion mode, enabled auto-pull, and fork_open_prs are rejected on fork workspaces (they are parent-managed; a fork's deploys always target its wm-fork/** branch) and the promotion card is hidden in a fork's settings. Enabling auto-pull now also requires EE, and the post-commit webhook reconcile persists the normalized delivery mode. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01PP5gBSPfo1YtkL1sWVAjJm * feat(git-sync): dev workspaces sync with their environment-label branch A dev workspace's git branch is its environment label verbatim (dev/ staging, default dev) — a first-class env branch like the documented push-on-merge-staging layout — instead of the wm-fork/** form. The label rides the deploy job args (backend → hub script → CLI --dev-workspace-label), the PR completion hook derives the same head, the webhook/poller route label branches into the matching dev-workspace child (poller lists them alongside wm-fork/* via extra ls-remote refs / per-label API lookups), and manual pulls from the UI pass clone_ref accordingly. The CLI refuses to deploy when the label branch equals the checked-out tracked branch, which would otherwise commit fork content straight to it. Because the branch is keyed on the label, the label is now immutable after creation: set at create/attach only, the set_dev_workspace_label endpoint is removed and the settings tab shows it read-only. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01PP5gBSPfo1YtkL1sWVAjJm * feat(git-sync): nested fork routing + fork-of-dev branch rooting A fork of a dev workspace now roots its wm-fork/** branch on the dev's environment-label branch (the content it diverged from) and its PR merges back into that branch: the backend passes parent_dev_workspace_label with the deploy (parent row joined in both enqueue paths), the CLI gains --parent-dev-workspace-label and checks it before the wm-fork- prefix fallback when rooting a fork-of-a-fork branch, and the PR completion hook uses it as the PR base. Fork sync routing covers the whole live descendant chain of the webhook/poller workspace (recursive, depth-capped) instead of direct children only, and fork_open_prs is resolved at the root ancestor — only the root can hold auto-pull config, so grandchild forks sync through it. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01PP5gBSPfo1YtkL1sWVAjJm * feat(git-sync): PR deploy-preview comment, clearer check copy, app-only hints - The PR diff completion hook maintains one managed comment on the PR (Cloudflare deploy-preview style: workspace, status, commit, collapsible change list), upserted per synchronize via a hidden marker. The check run stays for required-check gating. - A settings difference in the diff summary is worded by cause: the PR changes wmill.yaml, vs pre-existing drift between the repo's wmill.yaml and the workspace, vs undetermined (neutral wording). - Deploy-status check titles name the target workspace ("Deployed 2 change(s) to staging"), since GitHub shows a head commit's checks on any PR containing it and a bare "Deployed" read as if the PR had deployed. - Token-based repos see a hint pointing at the open-pr-on-commit / open-pr-on-fork-commit workflows where the app-only PR toggles would be; an API-set toggle on a non-app repo now logs a warning naming the fallback; the design doc lists app-only features and their degradation. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01PP5gBSPfo1YtkL1sWVAjJm * fix(git-sync): EE-gate auto-pull UI, fork pull clone_ref, no-op push PR gate - CE: the auto-pull and fork-PR toggles are disabled with an EE badge, and new sync repos only default them on when licensed (basic git sync is available on CE since #8493, but auto-pull is EE and the backend rejects it) - The pull modal passes clone_ref for wm-fork- forks (wm-fork/<tracked>/<id>) so a manual pull fetches the fork branch instead of the tracked branch head - PR-on-deploy skips no-op pushes: when the push script reports pushed=false (e.g. the deploy was caused by an auto-pull), the completion hook no longer ensures a PR, so closed PRs aren't recreated by the sync loop Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01PP5gBSPfo1YtkL1sWVAjJm * chore: refresh package-lock after main merge (windmill-utils-internal 1.8.2) Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01PP5gBSPfo1YtkL1sWVAjJm * test: auto-pull e2e integration tests; fix PR comment table formatting Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01PP5gBSPfo1YtkL1sWVAjJm * fix(git-sync): runtime license gate for auto-pull saves; user/group promotion-branch parity Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01PP5gBSPfo1YtkL1sWVAjJm * fix(git-sync): explain in-sync PR verdicts with the repo's sync filter scope A PR that only touches files outside the repository's include paths gets "In sync", which reads as a wrong verdict; the check summary (and managed comment) now name the filters, e.g. "Only files matching this repository's sync filters deploy on merge: `f/**` (excluding `f/pat/**`)." Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01PP5gBSPfo1YtkL1sWVAjJm * feat(git-sync): clearer card copy/structure; surface PR-creation failures - Fork sync toggle renamed and kept in the pull section; the fork PR toggle moves to the push section with a note that push settings apply to forks - Fork/dev workspaces' push section names their actual branch instead of the tracked-branch line; promotion repos hide the pull direction (promotion pushes deploy branches on top of a sync-mode setup) - Promotion mode line describes the wm_deploy/** branch + merge-to-promote flow; workflow-fallback hints lead with the how-to and link to the docs; test connection button demoted from accent per brand guidelines - New server-owned open_pr_error on repo settings: the deploy completion hook records why a PR couldn't be opened (e.g. app permission not yet approved) and clears it on the next success; shown as a warning under the PR toggles Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01PP5gBSPfo1YtkL1sWVAjJm * fix: cfg-gate scope-note helper (dead code on OSS builds) Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01PP5gBSPfo1YtkL1sWVAjJm * fix(git-sync): license-gate preserved auto-pull; attach strips parent-only settings - edit_git_sync_repository re-checks the runtime Enterprise gate against the EFFECTIVE repo state after preservation: the older-client arm copies the existing auto_pull back, which the request-side check never saw - attach_dev_workspace now mirrors the fork-creation copy on the attached workspace's own git sync: promotion repos dropped, auto_pull/fork PRs/PR error stripped, and any managed webhook deleted after commit (the attached workspace is parent-managed and must not keep pulling its old tracked branch) - integration test: attaching an auto-pull-enabled workspace strips it Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01PP5gBSPfo1YtkL1sWVAjJm * fix(git-sync): detach clears standalone parent; reject label == tracked branch - detach_dev_workspace clears parent_workspace_id for prefix-less (attached standalone) workspaces so they stop classifying as forks and deploying to wm-fork/** branches; wm-fork- re-designated forks keep their parent; cache invalidations mirror attach - dev-workspace create/attach reject an environment label that equals a git-sync repository's tracked branch (prod's or the candidate's): deploys would target the very branch the repo syncs from, and the CLI guard would fail every push job after the fact - CLI unit tests: prefix-less fork beats wm_deploy derivation; isForkWorkspace parent-id argument Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01PP5gBSPfo1YtkL1sWVAjJm * chore(git-sync): bump hub script pins (push 28786, pull 28785) Published from windmill-integrations #163 with windmill-cli@1.753.1-gitsync.0: dev-workspace label deploys, fork-of-dev rooting, fork checkout on the existing remote branch, and the pushed-flag result. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01PP5gBSPfo1YtkL1sWVAjJm * fix(git-sync): no parent-only defaults on fork repos; rename strips webhook state - addSyncRepository skips the auto_pull/fork_open_prs defaults on fork/dev workspaces where the backend rejects them (saving a new sync repo from an EE fork 400'd deterministically) - change_workspace_id strips webhook id/secret/error from the copied git_sync and deletes the stale GitHub hooks post-commit: they deliver to the old (archived) workspace URL, so the new workspace would report a live webhook while polling at the relaxed interval; next save re-registers cleanly - EE: PR diff checks for contributor-fork PRs clone the synthetic pull/<n>/head ref (head.ref doesn't exist in the base repo) Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01PP5gBSPfo1YtkL1sWVAjJm * chore(git-sync): bump pull script pin to hub/28787 (synthetic PR ref support) Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01PP5gBSPfo1YtkL1sWVAjJm * fix(git-sync): targeted jsonb update for open_pr_error (no full-blob clobber) The full read-modify-write raced the poller's concurrent last_synced_sha / last_pull_status writes on the same column; mirror the EE status writer and update only the matching repository element's open_pr_error key. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01PP5gBSPfo1YtkL1sWVAjJm * style(git-sync): inline EE badge on gated toggles (matches settings nav) Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01PP5gBSPfo1YtkL1sWVAjJm * copy(git-sync): both directions in page/card descriptions; clearer promotion flow - Page header and sync-card description mention the pull direction, not only push-on-deploy - Promotion description walks the actual flow (wm_deploy/** branch, merge to promote, sync the target workspace) and points at the PR toggle / workflow; the Git Promotion docs link now also shows on configured cards, not only in the empty state Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01PP5gBSPfo1YtkL1sWVAjJm * fix(git-sync): resolve branch-less resources' default branch for fork sync A git resource without an explicit branch polled as the bare "HEAD" ref, which the fork/dev-label fan-out cannot scope (wm-fork/<branch>/*), so fork sync silently never ran on polling-only repos. Resolve the remote's default branch name with `ls-remote --symref HEAD` (one call for name + head sha); "HEAD" only remains when resolution fails. The polling e2e test now uses a branch-less resource to cover this shape. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01PP5gBSPfo1YtkL1sWVAjJm * fix(git-sync): runtime license gate for in-app PR creation promotion_open_prs/fork_open_prs are rejected on save without an Enterprise plan (like auto_pull), and the deploy completion hook re-checks the plan before opening PRs so flags stored while licensed stop driving GitHub calls after a lapse. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01PP5gBSPfo1YtkL1sWVAjJm * feat(git-sync): app-aware pull defaults, always webhook delivery, token-repo guidance - Pull-from-Git defaults on only for app-backed repos (applied when the selected resource resolves); polling is opt-in for token repositories, with a warning alert recommending the GitHub App (instant pull + in-app PRs) or the sync GitHub workflow - App repos always use webhook delivery with polling fallback: the delivery selector is gone and a stored polling mode is normalized back to auto - Post-save modal reflects the auto-pull state instead of telling the user to turn on a toggle that is already on - Non-app PR hints recommend the GitHub App explicitly Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01PP5gBSPfo1YtkL1sWVAjJm * copy(git-sync): single info box for token-repo pull guidance Merges the instant-pull recommendation with the GitHub Action conflict note, shown only for non-app repos; app repos need neither, and the redundant 'instant webhook sync requires' line is gone. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01PP5gBSPfo1YtkL1sWVAjJm * copy(git-sync): keep the GitHub Action conflict note on app repos Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01PP5gBSPfo1YtkL1sWVAjJm * copy(git-sync): anchor docs links to their exact sections GitHub App references point at integrations/git_repository#github-app, the workflow hints at deploy_gh_gl#github-actions-setup, and the sync workflow at git_sync#github-actions (all anchors verified against the live docs). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01PP5gBSPfo1YtkL1sWVAjJm * copy(git-sync): fork workflow hint links to git_sync#github-actions open-pr-on-fork-commit is documented on the git_sync page, not deploy_gh_gl. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01PP5gBSPfo1YtkL1sWVAjJm * fix(git-sync): fork PRs are opt-in on new connections too Only auto-pull and fork sync default on for new app-backed connections; opening pull requests stays a deliberate per-repo decision. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01PP5gBSPfo1YtkL1sWVAjJm * feat(git-sync): default the managed PR on for new app-backed promotion repos A promotion deploy's wm_deploy/** branch exists to be merged; without a PR it's an orphaned branch. Fork PRs stay opt-in. Also scope the sync-repo auto-pull default to sync mode so promotion repos can't pick it up. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01PP5gBSPfo1YtkL1sWVAjJm * docs(git-sync): GHES self-managed app permission setup Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01PP5gBSPfo1YtkL1sWVAjJm * docs(git-sync): frame permission update against GitHub Actions, not polling Existing installations don't have polling; their git-to-Windmill direction runs on GitHub Actions today, so the approval text describes the update as replacing those workflows and notes every feature is opt-in. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01PP5gBSPfo1YtkL1sWVAjJm * copy(workspaces): drop 'cosmetic' qualifier from dev-workspace label UI * chore: update ee-repo-ref to 9b2a6375f838436cf68cff449cc9bc621cca5281 This commit updates the EE repository reference after PR #632 was merged in windmill-ee-private. Previous ee-repo-ref: 99eef24e2f0402b9a997cde5f67be52ee5d54b0e New ee-repo-ref: 9b2a6375f838436cf68cff449cc9bc621cca5281 Automated by sync-ee-ref workflow. * fix(git-sync): reject '/' in fork and dev workspace ids * fix(git-sync): bound auto-pull git probes with a per-command timeout * fix(git-sync): persist webhook reconcile via targeted jsonb updates --------- Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Co-authored-by: windmill-internal-app[bot] <windmill-internal-app[bot]@users.noreply.github.com>	2026-07-16 15:55:44 +02:00
Diego Imbert	e47aedac0a	feat: add SQL migrations for data tables (#9693) ... * feat: add datatable_migrations table * feat: add route to run datatable migrations * feat: sync datatable migrations as .up.sql/.down.sql files * feat: add datatable migrate up/down commands and post-push run prompt * feat: add datatable migrate new command to scaffold migrations * feat: add datatable migrations management UI * feat: prompt to create migration on DDL in datatable SQL editors * feat: support running a single specific datatable migration * feat: view migration content, run single migration, fix stacked modal * feat: per-row revert button with out-of-order warning * fix: avoid migrations list flicker on refresh after an action * feat: generate initial datatable migration via pg_dump * fix: surface datatable migration API error details in toasts * fix: revert created migration if create-and-run fails to run * fix: include postgres error detail in migration run/rollback failures * feat: sync datatable migrations as files via the workspace export * refactor: move datatable migrations to migrations/datatable/ path * fix: drop redundant datatable_migration label in sync output * fix: exclude datatable migration sql files from script metadata generation * feat: run datatable migrations as user-permissioned labeled jobs * feat: reject invalid datatable migrations on sync push * feat: datatable migrate up/down default to all datatables, --datatable to target one * fix: surface postgres error detail when datatable migrations fail to run * chore: regenerate CLI docs for datatable migrate commands * feat: default new datatable migration to a BEGIN/END transaction template * fix: validate datatable migration name and datatable at the API boundary * fix: ensure detected DDL ends with semicolon when wrapped in transaction * fix: re-prompt instead of stripping DDL when new-migration modal is cancelled * feat: refresh datatable schema after running a migration from the SQL REPL * feat: record db manager DDL on data tables as migrations * feat: make datatable migrations opt-in per data table * fix: make migration view editor read-only so its code can scroll * fix: don't re-prompt DDL guard when creating a migration without running * feat: generate down migrations for db manager DDL (postgres) * fix: correct down migration for db manager alters (no double-wrap, serial) * feat: explain migrations purpose with a tooltip in the migrations modal * compare paeg * feat: add datatable_migration kind to workspace diff pipeline * chore: point ee-repo-ref at datatable_migration git-sync companion * fix: harden datatable migration version allocation and initial-migration bookkeeping, add tests * feat: deploy and run datatable migrations on workspace merge Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * Refactor + handle datatable setting delete/rename * refactor: move datatable migration rename/delete cascade into module Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * chore(windmill-utils-internal): bump to 1.7.1 for datatable migration deploy provider methods Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(db-manager): add Migrations button to top bar, make Refresh icon-only Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * BEGIN/END placeholder in down migration * feat: autofocus migration name input and flag it red when empty Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(datatable-migrations): allow non-admins to create/run/revert migrations, gate only opt in/out Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * border nits * refresh db manager schema on migrations * BEGIN/END scaffold in CLI * feat(cli): push local datatable migrations before running on migrate up Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat: flag invalid migration name with red border, not just empty Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * refactor: drop random slug from auto-generated migration names Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat: offer revert-and-delete when deleting an installed migration Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat: record fork merge as a migration when target datatable opts in * nit * clone migrations on fork * windmill-utils-internal * fix(datatable-migrations): serialize run/rollback with a per-db advisory lock Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(db-manager): fail closed when migrations-status check errors on DDL apply Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * docs: fix generate_initial migration ordering comment to match code * chore(datatable-migrations): remove unused update_datatable_migrations endpoint Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix: run DDL migration guard on the script editor Test button Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * split * ee-repo-ref * chore(frontend): sync package-lock with package.json (@emnapi deps) Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(datatable-migrations): never resolve instance credentials into migration job args datatable_database_arg eagerly resolved instance data-table credentials (including the shared instance-wide Postgres password) and passed them as the migration job's plaintext `database` arg, landing in v2_job.args. Since the run route has no admin gate, a non-admin could run a migration and read args.database to recover the password, granting cross-workspace psql access to all instance data-table DBs. Pass a `datatable://<name>` reference for both resource-backed and instance data tables instead; the pg executor already resolves it to real credentials server-side at run time, so nothing sensitive is ever stored in the job args. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * nit * fix: handle dollar-quoting and comments when splitting SQL statements * feat: deploy datatable migrations on merge with explicit opt-in error * fix(frontend): sync package-lock with npm 11 peer-dep resolution npm ci failed with 'Missing: @emnapi/core@1.11.2 / @emnapi/runtime@1.11.2 from lock file'. @napi-rs/wasm-runtime declares @emnapi/core|runtime ^1.7.1 as peerDependencies while @rolldown/binding-wasm32-wasi pins them to exactly 1.10.0. Newer npm (bundled with node 24 in CI) installs the peer deps at the highest match (1.11.2) alongside rolldown's nested 1.10.0, so the ideal tree needs both versions; the committed lock only had 1.10.0. Regenerate the lock with npm 11.18 so it carries both 1.11.2 (top-level, for the peer deps) and 1.10.0 (nested, for rolldown's pin). Verified npm ci passes. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * nit npm publish * fix: fail closed on migrations-status error in fork schema merge * nit CI emnapi/core version * prevent initial_datatable_migration if migrations already exist * fix(datatable-migrations): validate persisted data table names as path segments edit_datatable_config only validated rename segments, not the actual settings.datatables keys, so a data table could be saved directly under a name like '..' or one containing '/'. Since new tables default to migrations_enabled = true, generate_initial_datatable_migration would then insert a migration row and the sync export would build migrations/datatable/<name>/... paths from that name, producing malformed or directory-escaping export paths. Validate every persisted data table name in edit_datatable_config (alongside the existing rename checks) and add validate_datatable_path_segment to generate_initial_datatable_migration for defense in depth. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix: scope datatable _wm_migrations by data table and cascade renames/deletes Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(system_prompts): resolve nested local command groups in CLI docs generator The CLI docs generator anchored on the first `new Command()` in a file and never resolved locally-defined command groups passed as `.command("name", localCmd)`. For datatable this flattened the nested `migrate` group: it emitted `datatable new/up/down` plus a bare `datatable migrate`, and mislabeled the datatable command with the migrate group's description. jobs was broken the same way (its description was pull's, and pull/push rendered empty). Anchor block extraction on the `export default`ed command, recurse into locally-defined `const x = new Command()` groups mounted as subcommands, and render nested sub-subcommands. Regenerated docs now show `datatable migrate new/up/down` and `jobs pull/push` with their real options. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * refactor: drop unreleased _wm_migrations legacy-upgrade handling Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix: return datatable migration SQL from getItemValue for the diff drawer Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * chore(frontend): use windmill-utils-internal 1.8.2 for migration diff drawer Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * nit * nit * fix: handle datatable migration renames on push and dedupe timestamps * fix: reject rewriting an already-applied datatable migration on upsert Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(frontend): add missing @emnapi/core and @emnapi/runtime lockfile entries Resolves npm ci EUSAGE failure: the optional cpu:wasm32 @rolldown/binding-wasm32-wasi declares deps on @emnapi/core@1.11.2 and @emnapi/runtime@1.11.2 that had no resolved lockfile entries. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(cli): datatable migrate up/down default to main datatable, not all Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix: fail closed when applied status unreadable on datatable migration rewrite Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix: surface full error detail in Database Manager DDL/query errors * "See migration" button in the toast * feat: add Enter shortcut to Create-a-migration in the DDL guard * fix(frontend): warn before running a newly-created datatable migration out of order The row-level Run action warns when earlier migrations are still pending, but the create-and-run paths ran a just-created migration with `only` directly, applying it ahead of older pending migrations without that confirmation. Reuse the same "Run migration out of order" confirmation across all create-and-run paths via a shared helper (datatableMigrationUtils): - NewDataTableMigrationModal "Create and run" (and the DDL guard path) - DatatableSchemaDiff fork→parent merge - dbOps schema ops (DB manager create/alter/drop) — the pure factory throws a MigrationRunCancelled sentinel on decline, which DBTableEditor treats as a silent cancel Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix: keep renamed datatable migrations visible in compare view * fix: record per-migration deployment on datatable migrations disable * fix(cli): run deployed datatable migrations after workspace merge The merge command upserted datatable_migration definitions into the target workspace and reported the item as successfully deployed, but never ran the migrations. For forked datatables backed by separate databases, this left the target schema unchanged until someone manually ran `wmill datatable migrate up`, while the CLI reported a successful merge. Collect the datatable migrations deployed (not deleted) into the target and, after the deploy loop, offer to run them via the existing offerToRunNewMigrations helper — the same post-deploy run prompt the push/sync path uses (interactive only; `--yes`/non-TTY skip the mutating run, matching push behavior). Export parseDatatableMigrationDeployPath so the merge path can parse the deployed items. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(backend): serialize datatable migration edits/deletes with the run lock A migration run snapshots a migration's code_up from datatable_migrations and only records its version in the data table's _wm_migrations after the job succeeds. upsert_datatable_migration checked _wm_migrations before allowing an edit but took no lock, so a concurrent edit could read "not applied yet", rewrite code_up/code_down, and then the in-flight run would record the version for the old SQL — leaving _wm_migrations pointing at SQL that was never applied (migrate up then skips it; rollback runs a down that doesn't match). Serialize definition rewrites and deletes with the same per-database advisory lock the run/rollback paths use: - Factor the connect+advisory-lock into lock_datatable_migration_runs and the applied-versions read into read_applied_versions_on_client. - run_datatable_migrations now snapshots the definitions AFTER taking the lock, so code_up can't change between snapshot and version-record. - upsert (when changing an existing def) and delete take the lock across the applied-check and the write; delete now rejects deleting an already-applied migration (would orphan its _wm_migrations record), symmetric with upsert. Both fail closed if the data table database is unreachable. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(frontend): stack the out-of-order migration confirm above the DB editor preview Creating a table on a migrations-enabled data table opened the DB table editor's "Confirm running the following" preview modal, whose confirm triggers applyDdl, which then asks for out-of-order confirmation. Both are ConfirmationModals with a hardcoded z-[9999]; the out-of-order one lives in DBManagerContent (mounted before the editor), so it rendered behind the still-open preview modal. Add an optional zIndexClass prop to ConfirmationModal (default z-[9999], backward-compatible) and give the DB-manager out-of-order confirm z-[10000] so it stacks on top. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * chore: update ee-repo-ref to 27672e37df5d9dfde94f19963d5ffcdf8dd5448c This commit updates the EE repository reference after PR #623 was merged in windmill-ee-private. Previous ee-repo-ref: 6c287041cd7edd4a77a4bc07ad0e156cec32cce4 New ee-repo-ref: 27672e37df5d9dfde94f19963d5ffcdf8dd5448c Automated by sync-ee-ref workflow. --------- Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Co-authored-by: windmill-internal-app[bot] <windmill-internal-app[bot]@users.noreply.github.com>	2026-07-07 08:25:16 +00:00
Ruben Fiszel	33521505db	feat(ducklake): scheduled lake maintenance (expiry, compaction, orphan cleanup) (#9916) ... * feat(ducklake): scheduled lake maintenance (snapshot expiry, compaction, orphan cleanup) Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(ducklake): review fixes — starts_with not LIKE, CE license-lapse escape Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * docs(ducklake): auth-contract docs + _unchecked rename per codex review Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * refactor(ducklake): move maintenance payload construction into EE module Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(ducklake): fall through to script resolution for non-managed reserved-prefix schedules Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * docs(ducklake): document accepted pre-existing-schedule limitation on the reserved prefix Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(ducklake): CE save-off clears the managed schedule row and queued occurrence Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * chore: update ee-repo-ref to 2fab310d4f50ed7c34857d69c9b854f4491bf217 This commit updates the EE repository reference after PR #645 was merged in windmill-ee-private. Previous ee-repo-ref: fff1fd830a36beba732486f05941ec243cf6b640 New ee-repo-ref: 2fab310d4f50ed7c34857d69c9b854f4491bf217 Automated by sync-ee-ref workflow. --------- Co-authored-by: Claude Fable 5 <noreply@anthropic.com> Co-authored-by: windmill-internal-app[bot] <windmill-internal-app[bot]@users.noreply.github.com>	2026-07-04 17:23:23 +02:00
Ruben Fiszel	0dbd9c1231	perf: eliminate dual-connection DB pool contention across worker, queue, and api (#9798) ... * perf: eliminate dual-connection DB pool contention across worker, queue, and api Reuse the held transaction (or move pool reads before begin()) instead of checking out a second pool connection while a tx is open, extending the fix from #9789/#7861. Targets the per-worker pool (max 5) hot paths plus several server-pool API handlers. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix: pass owned pool to get_email_from_permissioned_as in http trigger handler The generified signature takes impl PgExecutor; the http trigger handler passed &db where db is already &DB, yielding &&Pool which does not impl PgExecutor (only surfaced under the full feature set in CI). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix: keep RLS-exposed reads on the non-RLS pool and isolate flow-eval reads in a savepoint Addresses review of the dual-connection sweep: - worker_flow: wrap the stop_after_all_iters_if reads in a SAVEPOINT. The caller swallows the error and keeps using tx, so a DB read failure must not leave the outer transaction aborted (it would fail the later commit). Matches the previous pool-read semantics. - Revert reads that were moved onto an RLS (user_db) transaction back to the non-RLS pool, since RLS row-visibility/role context can change results: push_scheduled_job (email/tag/settings lookups; reachable with a user_db tx from api-schedule/api-flows), push_inner native-retry dedicated_worker routing (RLS isolation variants), resources.rs app-namespace folder auto-create (non-admins must not be blocked), and the script archive/delete UPDATEs. Non-RLS db.begin() reuse and move-before-begin are kept. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * test: failpoint proving the stop_after_all_iters_if savepoint isolates an aborted read Adds a worker-crate failpoints feature and a data-driven hook: when the stop_after_all_iters_if expr is the magic sentinel, the in-evaluation read runs SELECT 1/0 to abort its (savepoint) transaction. The test asserts the flow still completes (iteration marked failed) — which only holds if the savepoint keeps the outer status-update transaction committable. Without the savepoint the abort would poison the outer tx and the job would never complete. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>	2026-06-25 21:33:17 +00:00
hugocasa	e19594df2a	fix: re-enforce scoped API token boundaries across handlers (#9712) ... * fix: re-enforce per-path token scope on store rename, delete and interpolation Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix: enforce token scope on workspace export and resume-url minting Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix: enforce per-item and runnable scope on trigger create paths Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix: enforce app write scope before persistence and on rename Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix: enforce scope containment on mcp oauth approval Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix: scope mcp endpoint-proxy jwt to the proxied route Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix: treat resource-linked variables and resources as covered by resource scope Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix: only require variables:read for plaintext-secret workspace export Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix: handle singlestepflow resume, reject empty mcp grant, scope var-skipped tarball Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>	2026-06-22 23:55:19 +02:00
Diego Imbert	1fc355709c	feat: Db-backed user drafts (#9351) ... * Db draft removal * refactor: drop unsaved-changes confirmation modal from editors * fix: remove nodraft from flow row edit link * fix: remove nodraft from app and raw app edit buttons * fix: remove nodraft from all edit links * fix: merge backend defaults into legacy autosaves to avoid spurious restore toast on raw apps * feat: add username column to draft table for user-scoped drafts * feat: add sync_drafts and list_users_with_draft_on_path endpoints * feat: add UserDraftDbSyncer service for bi-directional draft sync * feat: wire UserDraft.save through DbSyncer + conflict modal * refactor: gate useLocalStorageValue nested-update effect behind opt-in flag * refactor: move sync force flag from request-level to per-entry * feat: sync all userdraft kinds, switch draft owner to email FK, add id PK, scope draft list to readable paths * refactor: route draft permission check through authed.folders + RLS, drop client-supplied email * feat: support draft deletion via sync (value: null) with same conflict semantics * feat: surface other users' drafts in editors with diff+fork action * refactor: unify draft schema migrations and type kinds via DRAFT_KIND enum * perf: add (workspace_id, email, created_at) partial index for sync hot path * chore: update ee-repo-ref to a30079e75dc5b7d7413aa8ee20e40e80bfea9cbd This commit updates the EE repository reference after PR #597 was merged in windmill-ee-private. Previous ee-repo-ref: 55c19293232be379a3044eb78f677b545882ffd6 New ee-repo-ref: a30079e75dc5b7d7413aa8ee20e40e80bfea9cbd Automated by sync-ee-ref workflow. * fix(userdraft): trigger sync on deep mutations via readFieldsRecursively * Rollback UserDraft * remove queuing logic * pushDrafts * refactor: remove draft sync layer and conflict modal * feat: add save_draft, list_drafts, get_draft routes * feat: add get_draft overlay to getScriptByPath * feat: extend get_draft overlay to flow, app, resource, variable, schedule, triggers * feat: support null value in save_draft for deletes * readLastSyncMap * feat: redirect /add pages to /edit/draft_uuid with new_draft flag * fix: inline get_draft query field instead of flattening * fix: drop dangling nobackenddraft assignment in flows edit * feat: include user drafts in list endpoints with is_draft flag * fix: prefix draft paths with u/{user} and seed editor state on new_draft * fix: route draft-only deletes through UserDraftDbSyncer on home page * feat: delete user drafts when their underlying item is deleted * fix: empty path seed on new_draft so friendly auto-name fires * feat: re-add Draft and Draft only badges on home page rows * fix: synthesize value wrapper on draft-only raw_app response * fix: tolerate missing latest-version on draft-only flow reload * fix: skip first observable change in DB sync effect to match LS persist * fix: remove URL-hash sync from script editor (already marked TEMP) * refactor: drop localStorage layer from UserDraft * refactor: drop vestigial LS-era code from UserDraft * feat: migrate localStorage drafts to DB on layout mount * fix: migrate session runtime + script view to per-user draft API * feat: add 'Reset to deployed' action on draft-loaded toast * feat: hide 'Reset to deployed' action when no deployed version exists * createCoalescingKeyedRunner * example ts doc * createDebouncerByKey * refactor: drop await on draft-delete in reset flows, refetch deployed directly * fix: bridge saved-draft shape to wire shape in apps/resources/variables loaders * feat: route UserDraftDbSyncer.save through debouncer + coalescing runner * feat: add immediate-save bypass that cancels pending debouncer + runner tasks * fix: seed UserDraft cell from spec defaultValue on acquire * fix: redirect /add routes at load phase to eliminate white flash * fix: drop +page.js files in /add routes that conflicted with +page.ts * refactor: send draft as separate .draft field instead of deep-merging onto deployed * feat: surface draft path in home list when user typed one different from URL * feat: add UserDraft.stopSync/restartSync, wire on script + low-code app /add init * fix: thread URL path into ScriptBuilder.stopSync (was using empty initialPath) * fix: also stopSync in route's new_draft branch + queue pre-acquire suspensions * feat: add AutosaveIndicator backed by reactive UserDraftDbSyncer.getState * refactor: drop draft-loaded toast in non-route editors, banner now compares draft vs deployed * fix: gate per-user draft-only rows in listings on include_draft_only flag * feat: flush pending draft saves via keepalive fetch on tab hide / pagehide * autosave indicator nits * fix: route create-vs-update on /add deploys; seed policy.execution_mode; sync script template * chore: add [draft-sync] console logs to trace script bootstrap autosave * fix: seed auto-generated path in script new-draft route to suppress Path widget's autosave-triggering mutation * fix: defer script restartSync until script.path lands (Path widget gated on $userStore + $workspaceStore) * fix: poll script.path via tick() until Path widget settles before restartSync * chore: log inferArgs underlying error on deploy to diagnose 'Could not parse code' toast * fix: wait for script.path to stabilize across two ticks before restartSync * revert: drop unsuccessful path-stabilization heuristics + leftover [draft-sync] logs * fix: seed new-draft script schema as emptySchema() so inferArgs doesn't trip on undefined properties * fix: heal legacy drafts with schema={} (no .properties) on deploy * autosave indicator * refactor(editors): drop UnsavedConfirmationModal mount + Show diff button * feat(drafts): collaboration banner, cross-tab conflict detection, raw app template picker - Other-users-drafts banner (Modal2): the deployed-overlay response now carries `other_drafts_users` (workspace usernames only, never emails); each row offers View JSON + Fork. Drops the standalone `listUsersWithDraftOnPath` endpoint; `getDraftForUser` now takes a workspace `username` query param (resolved to email server-side). - Cross-tab/browser save conflict detection: the syncer attaches `last_sync` to every save (defaults to non-force); on a `conflict` response it parks a snapshot in a reactive map. Each route mounts a `DraftSyncConflictModal` and seeds the per-tab `last_sync` via `recordRemoteSync(query, draft_saved_at)` on every `get_draft` load. Keepalive flush also respects optimistic concurrency. - Raw app template picker re-added after the /add ⇒ /edit refactor: framework (React 19 / 18 / Svelte 5), data table + schema config, and optional AI prompt — extracted into `RawAppTemplatePicker.svelte` and driven by `new_draft=true` on the edit route. * fix(drafts): suppress autosave during /add template seeding on script + raw app editors - ScriptBuilder: delay `restartSync` 500ms past `initContent` + stores- ready so the Path widget's `$workspaceStore && $userStore`-gated `initPath → reset → onMetaChange → bind:path` cascade lands inside the suspension window. Two `tick()` waits weren't enough — the bind:path mutation fired ~100ms after the prior `restartSync` and posted as a "user edit". - apps_raw route: suspend autosave on `new_draft=true` and resume only after the framework picker closes (via `onStart` or X dismissal), with a two-tick settle so the picker's seeded `files/runnables/data/policy` mirror to `draftHandle.draft` observably advances `lastSerialized` before sync re-arms. * fix(drafts): land /add redirects on the real workspace username, not "me" The `/add` → `/edit/u/{username}/draft_{uuid}` redirects ran during SvelteKit's load phase, BEFORE the (logged) layout's async `getUserExt` populated `userStore`. `get(userStore)?.username` returned undefined and fell back to the `'me'` placeholder on every fresh nav, producing `u/me/draft_{uuid}` paths instead of the user's real namespace — broke ownership checks against `authed.username` and silently scoped autosaves under the wrong path. Layout now persists `username` to localStorage on every successful `getUserExt`, and `getUsernameForNamespace` (new shared helper, used by all four `/add/+page.ts` files) reads the live store first, falls back to the cached value, and only then to `'me'` for true first-ever loads. * fix(drafts): key low-code app autosave on the URL path, not the empty string `AppEditor` keyed its `UserDraft.use` handle on `newApp ? '' : path` — a legacy leftover from when `/apps/add` was its own URL (no path). With the `/add` ⇒ `/edit/u/{user}/draft_{uuid}` redirect, `newApp=true` made autosaves land on the `('app', '')` row instead of the URL path: - The `apps/list?include_draft_only=true` query joins drafts onto `app.path`, surfacing drafts at the URL path. The empty-path row didn't match the user's URL so the draft never appeared in the home list. - Refreshing `/apps/edit/u/{user}/draft_{uuid}` re-fetches at the URL path with `?get_draft=true`, finds nothing, and 404s. Drop the ternary so the handle always uses `path` — the same as scripts/flows/raw_apps. The route's `?new_draft=true` branch already seeds the empty-template baseline, so there's no longer a "the draft sits under '' until first save" race to worry about. * fix(raw_app): propagate template picker X / Esc dismissal so autosave resumes The picker mounted `<Modal kind="X" open ...>` (one-way prop, not `bind:open`). When the user dismissed via X / Esc / click-outside, the inner Modal flipped its own local `open` to false (hiding the UI) but never wrote back to the picker's `open` $bindable. The route's `templatePicker → false` watcher — the one that calls `restartSync` two ticks after the picker closes — never fired, so autosave stayed suspended and the user's edits after dismissal were silently dropped. Switch the inner Modal to `bind:open` so the dismissal bubbles all the way up to the route's state. "Start without AI" already worked because its `onStart` handler explicitly sets the picker's `open = false`. * nit unused * fix(drafts): make the home-page View/Edit JSON action work on draft-only apps The "View/Edit JSON" entry on the home page called `AppService.getAppByPath` without `get_draft=true`, so for draft-only items at `u/{user}/draft_{uuid}` the backend 404'd with "App not found at path …". Pass `get_draft=true` and render the synthesized stand-in's editable shape: - App drafts come back as `{summary, value, path, policy, ...}` — `value` is the App definition the editor was working on; show that. - Raw-app drafts come back as the flattened `{files, runnables, data, summary, policy, ...}` with no nested `value`; show the whole shape. On save, draft-only items can't go through `updateApp` (no deployed row). Route the edit through `UserDraftDbSyncer.save` (with `immediate: true` so `await` resolves after the POST lands) and relabel the button "Save draft" + Save icon. Deployed items keep the existing "Deploy" flow unchanged. * fix(drafts): render the right shape in View/Edit JSON for draft-only items The previous fix landed `fapp.value` into the editor, but the deployed-overlay flattens the bare editable shape into `inner`/the top-level response — drafts have no nested `.value`. So: - App drafts (`{grid, breakpoints, hiddenInlineScripts, …}`) rendered as empty (`fapp.value` was undefined). - Raw-app drafts 404'd outright: `get_draft=true` with no `rawApp` flag can't tell which draft kind to look up, defaults to `app`, doesn't find one. Thread the row's `raw_app` flag from AppRow → `appExport.open(path, rawApp)` → `getAppByPath({..., rawApp})` so raw-app drafts resolve to the right `UserDraftItemKind`. Read `fapp.draft` (the bare editable shape from `fetch_draft_only`) into the JSON editor for draft-only items — clean payload, no `is_draft` / `no_deployed` / overlay noise. Save the same bare shape back through the syncer so the regular editor reads it unchanged on the next mount. * fix(drafts): skip public-secret-URL fetch in the Deploy drawer for draft-only apps Opening the Deploy drawer on a `/edit/u/{user}/draft_{uuid}` app fired `AppService.getPublicSecretOfApp` immediately because the gating effect only checked `appPath != ''` + `savedApp`. The `/secret_of/{path}` route plain-SELECTs `app.id`, so a draft-only path 404'd with "App not found at name …" and the public-URL ClipboardPanel spun forever waiting on `secretUrl`. Thread the existing `newApp` signal (already on `AppEditorHeader` / `RawAppEditorHeader`) into `AppEditorHeaderDeploy`, gate the fetch behind `!newApp`, and render the existing "Deploy this app once to get the public secret URL" placeholder instead of the spinner for draft-only items. * fix(drafts): disable Diff button on draft-only items across the 4 editors Diff has no baseline to compare against on draft-only items — the button used to be gated by the pre-PR `/add` route's own state, but the `/add → /edit` redirect landed everything under the regular `/edit` page where the gate was missing. - ScriptBuilder: gate the topbar Diff on `savedScript.no_deployed`; seed `no_deployed: true` on the route's `new_draft` empty NewScript so the gate fires before the first deploy. - FlowBuilder: gate the topbar Diff on `newFlow` (route already sets it from `backendFlow.no_deployed` and the new-draft branch). - AppEditorHeader: gate both the "Diff" dropdown action and the Deploy-drawer's "Diff" button on `newApp`. - RawAppEditorHeader: gate the topbar Diff + the Deploy-drawer's "Diff" button on `newApp`. Each gate also rewrites the tooltip ("Deploy this … once to compare against the deployed version") so the hover state explains why. * fix(drafts): disable the "No login required" toggle on draft-only apps Flipping the toggle called `setPublishState`, which POSTs the new `policy` through `AppService.updateApp` — that handler's `UPDATE app ... RETURNING path` finds nothing on a draft-only path and `not_found_if_none` 404s with "App not found at name …" (apps.rs:1975). Gate the Toggle on `!newApp` too so the user has to deploy once before configuring the publish state. * refactor(drafts): drop dead draft_path field from list responses The draft-only listing branches in scripts/flows/apps computed a `draft_path` from the draft JSON (when the user-typed path differed from the URL's autogenerated `u/{user}/draft_{uuid}`), and `{Script,Flow,App} Row.svelte` preferred it over `path` for the row title. In practice that path is never written: the app, raw-app and flow editors all warn "Deploy the X to make the path change effective" — the rename only lands on deploy, never in the draft. So the field is always None and the home rows always show the autogenerated slot anyway. Drop the field from the three `Listable*` structs, the three draft-only push sites, the three OpenAPI response schemas, and the three frontend row components. Client regenerated. * fix(drafts): seed a friendly name on /flows/add The flow route passed `initialPath={page.params.path ?? ''}` to FlowBuilder, so on the `/flows/add → /flows/edit/u/{user}/draft_{uuid}` redirect the Path widget's `initPath` saw a non-empty `initialPath` and skipped the `reset()` branch that auto-generates the friendly `<random_adj>_flow` name. The other three editors all clear `initialPath` in their `new_draft` branch for exactly this reason. Track `initialPath` as route-owned state (defaults to the URL path) and clear it to '' inside the `new_draft` branch, then bind it through to FlowBuilder so any post-deploy update from the editor still propagates. * feat(drafts): render friendly user-typed path on home list for all 4 kinds Reinstate `draft_path` on `Listable{Script,Flow,App}` so the home rows prefer the user-typed name over the autogenerated `u/{user}/draft_{uuid}` URL slot, with two source rules — one per how each editor wires the Path widget: - Scripts already work: `ScriptBuilder` binds the Path widget directly to `script.path`, so the typed path round-trips through the draft JSON's own `path` field. Backend extracts `v["path"]` when it differs from `row.path`. - Flows / apps / raw apps don't write the typed path into the autosaved value (`Flow.path` is one-way-bound to `$pathStore`; the bare `App` / raw-app value has no `path` field at all). Introduce an explicit `draft_path` field on the draft JSON, written by the editor ONLY when the typed path differs from the deployed/seeded `savedX.path`: - FlowBuilder: $effect on `$pathStore` mutates `flow.draft_path`. - AppEditorHeader: $effect on `newEditedPath` mutates `$app.draft_path`. - RawAppEditorHeader: $effect surfaces `pendingDraftPath` up via the bind chain (RawAppEditor → route); the route's draftHandle.draft spread includes `draft_path` when set. Backend extracts `v["draft_path"]` and `None` when unchanged or after deploy (deploy clears the whole draft, so the field naturally disappears post-deploy without bookkeeping). Flow route's `new_draft` branch now stops sync around the Path widget cascade, with a 700ms scheduled `restartSync` (mirrors the existing scripts/apps/raw_apps stoppers) — the new draft_path mutation lands inside that window so `/flows/add` no longer fires an autosave before the user's first edit. openapi/sqlx regenerated. * fix(drafts): preserve the user-typed draft_path on reload of draft-only items The flow / app / raw-app editors all dropped the saved `draft_path` back to the URL's `u/{user}/draft_{uuid}` slot the moment the user reloaded a draft-only edit page: the route sourced the Path widget's initial path from `page.params.path` instead of the previously-saved `draft_path`, and the first user edit then mirrored that URL path back into the autosaved draft — silently overwriting the friendly name in both the row and the editor. - Flow route: after computing `effectiveFlow`, override `flowInitialPath` with `effectiveFlow.draft_path` when set. - App route: pass `newPath={(app.value as any)?.draft_path ?? app.path}` through to `AppEditor`; AppEditorHeader's `newEditedPath` default now prefers a non-empty `newPath` over the random `<adj>_app` seed (the `newApp && !newPath` branch keeps the `/apps/add` friendly auto-name). - Raw-app route: surface `savedRawAppDraft.draft_path` onto `backendApp` so the `extractRawApp` path seeds `newPath` with the friendly name. Reload + a subsequent edit now leaves `draft_path` intact for all three kinds; verified end-to-end via the `/drafts/get_draft/...` endpoint. * fix(ui): default Modal2 target to 'body' so omitting the prop doesn't throw Modal2 defaulted `target = ''` and forwarded it to `Portal`, which calls `document.querySelector(target)` — an empty selector throws "Failed to execute 'querySelector' on 'Document': The provided selector is empty" and the modal silently fails to mount. That's why `OtherUsersDraftsModal` (and `DraftSyncConflictModal`) never appeared on editors where another user had a draft — both omit the `target` prop. Other Modal2 callers (StorageSettings, CriticalAlert, CustomInstanceDbWizardModal, …) pass an explicit `target="#content"` and were unaffected. Match Portal's own default of `'body'` so omitting the prop is now a no-op rather than a runtime throw. * fix(drafts): Reset to deployed no longer resurrects the draft The toast's "Reset to deployed" callback POSTed `value: null` to the syncer, then handed control to the route's `onResetToDeployed` (which wipes the in-memory handle and reloads the deployed payload via `getDraft: false`). Both writes flowed through the reactive sync effect: the wipe scheduled a delete, the reload scheduled a re-save of the deployed value as the new draft. Coalescing collapsed them and the draft came back — making the "discard" action effectively a no-op. Wrap the whole callback in `UserDraft.stopSync` / `restartSync`. The explicit `value: null` POST still goes through (it's a direct `UserDraftDbSyncer.save` that doesn't depend on the reactive effect), the route's wipe-then-reload mutations advance `lastSerialized` silently under suspension, and the next user edit (after two ticks past the deployed-seed write) is the first real save again. * ui nit * feat(drafts): autosave-indicator popover with Reset-to-deployed action Click the cloud icon → popover with "All changes are saved as a draft on the server. The draft is per-user — your teammates' editors keep their own." When the editor isn't on a draft-only path AND the user has a draft (UserDraft.has returns true), a "Reset to deployed" button mirrors the load-time toast action — stops sync, POSTs `value: null`, runs the route's reload-without-draft callback, restarts sync past two ticks so the deployed-seed write doesn't resurrect the draft. Threaded `onResetToDeployed` from each route down to its builder (ScriptBuilder / FlowBuilder / AppEditorHeader / RawAppEditorHeader) and into the indicator. `draftOnly` is wired from `savedScript.no_deployed` / `newFlow` / `newApp` so the action hides where there's nothing to fall back to. The indicator's trigger now has a hover affordance + matches Portal's default target ('body') via Modal2's earlier fix. * fix(drafts): wait for the fork POST to land before navigating OtherUsersDraftsModal's Fork action called UserDraft.save, which routes through the autosave debouncer (1500ms). The subsequent goto fired within the same tick, so the destination editor's get_draft=true read ran before the POST landed and 404'd — refreshing worked because by then the debounced save had fired. Call UserDraftDbSyncer.save with immediate: true and await it. The syncer cancels any queued debouncer task for the key and resolves the promise only after the POST completes, so the route load can find the forked draft on the first try. * fix(drafts): conflict detection — keep last_sync map tab-local instead of in localStorage Two tabs editing the same draft both load with last_sync = T0. Tab-1 saves; the server accepts, returns T1, and the syncer wrote T1 into localStorage. Tab-2 then tries to save: it reads the SHARED localStorage map, sees T1 instead of its own baseline T0, sends last_sync = T1, and the backend's WHERE clause (`created_at <= last_sync`) is true → tab-2 clobbers tab-1's edit without ever seeing a conflict. Move the map to tab-local memory (`new Map<string, …>`). Reload of the tab now starts with an empty map; that's fine because the editor's load path calls `recordRemoteSync(query, draft_saved_at)` right after `get_draft=true` returns, reseeding from the authoritative server timestamp before any user edit could fire a save. * fix(drafts): OtherUsersDraftsModal — close on Fork, don't leak clicks through nested JSON Two bugs in the per-editor "another user has a draft" banner: - Fork landed the immediate save but didn't close the banner before navigating. Svelte hadn't torn down the previous route's components by the time goto returned, so the banner lingered on top of the destination editor. Comment the explicit isOpen=false on the happy path so it's clear it MUST run before goto. - Clicking anywhere on the screen while the View JSON drilldown was open closed the underlying banner too. Modal2's clickOutside action fired on every Modal2 instance — both the JSON modal and the underlying banner — because both attach their own listener at the document level. Add `closeOnOutsideClick` opt-out on Modal2 and pass `closeOnOutsideClick={!jsonOpen}` to the outer modal so clicks outside the JSON drilldown only close the drilldown. Drive-by: Modal2's keydown handler now ignores Escape when its own isOpen is false (was a no-op closer that would still preventDefault on every key press, swallowing key events for any siblings). * fix(drafts): conflict modal wording — drafts are user-scoped, not teammate-scoped * fix(drafts): defer reset-to-deployed restart until first user interaction Two-tick `restartSync` was too aggressive: editor remounts emit a tail of cascading writes (Monaco setValue acks, schema re-infer, UI Builder iframe handshakes, schedule-config recomputes, …) that land well after two ticks and would clobber the just-deleted draft with an upsert of the deployed value — making "Reset to deployed" a no-op in practice, the user kept seeing the draft come back. Centralise the suspension lifecycle in a new `runResetToDeployed` helper. It stopSyncs around the reset, POSTs the explicit delete, runs the route's wipe-and-reload, and then arms a one-shot listener on document keydown / input / pointerdown that restartSyncs on the user's next real interaction. A 5-second fallback re-arms sync if the user walks away without touching the editor, so suspensions don't leak. Use it from both the load-time toast (`notifyDraftLoaded`) and the autosave-indicator popover so the two stay in sync — fixes both entry points. * indicator ui nits * fix(drafts): split tab-switch and unload flushes — kill self-conflict on visibility change The single keepalive flush bound to both `visibilitychange → hidden` and `pagehide` self-conflicted on tab switch: visibilitychange fires on every tab/app switch with the page still alive, the keepalive POST advanced the server's `created_at` to a fresh `now()`, the client discarded the response (no listener), the local `lastSync` stayed at the old value, and the next foreground autosave sent that stale timestamp → server saw `created_at > last_sync` → conflict modal for the user's own background-tab write. A still-pending debouncer task made it worse: it fired a second runner POST after the keepalive with the same stale `last_sync`, the second self-conflicted too. Split into two paths: - `visibilitychange → hidden` → `flushOnVisibilityHidden`: route through the normal runner pipeline. The page is alive, so the response can land and `setLastSync` keeps the baseline current. Call `debouncer.cancel(key)` first so a queued keystroke can't double-fire with the same stale `last_sync`. - `pagehide` → `flushOnPageHide`: keep the `keepalive: true` raw fetch for the genuinely-going-away case (the JS context is torn down, the response is necessarily discarded). Same `debouncer.cancel(key)` guard. On the next mount, the route's `recordRemoteSync(query, draft_saved_at)` reseeds `lastSync` from authoritative server state before any user edit can fire a save. * fix(drafts): drop the visibilitychange flush — debouncer keeps running on hidden tabs Tab switching just hides the page; the JS context survives and the debouncer's `setTimeout` keeps counting down. When it fires, the runner POSTs normally and the server's response updates `lastSync`. There's nothing left for a visibilitychange-driven flush to do that the ordinary pipeline doesn't already handle, and adding one only creates extra POSTs to reason about. `pagehide` remains the single trigger for the keepalive flush — that's the case where the JS context is actually being torn down and the runner's pending fetch would otherwise be killed mid-flight. * nit * refactor(drafts): drop LS-era pipeline; backend is canonical on load The PR's iteration left behind a meta/staleness pipeline carried over from the localStorage era — per-rev tracking, a LocalDraftStaleModal, a 'Restored from local storage' toast, and a localDraft-vs-backend comparison branch in every editor loader. With drafts now living in the DB and the optimistic-concurrency lastSync check handling divergence, that whole stack is dead weight. Worse, the comparison branch caused 'Load from server' in the conflict modal to do nothing: the loader preferred the in-memory cell over the backend, so the user-clicked 'load from server' just re-displayed the local edits AND fired two confusing toasts (Restored from local storage + Loaded your saved draft). The rip: * userDraft.svelte.ts: drop UserDraftMeta, StoredDraft.meta, checkStaleness, UserDraftStalenessCause, normalizeForCompare, localDraftDiffers, saveMeta, getMeta, setDraftAndMeta, setMeta, handle.meta/setDraftAndMeta/setMeta, force option. Handle is now just { draft }. * userDraftToast.ts: drop notifyRestoredFromLocal + RestoreFromLocalActions. Update copy. * LocalDraftStaleModal.svelte: deleted. * AppEditor.svelte: drop initialRevs prop and the firstMirror wipe-then-restore dance (it existed only to consume the meta-mismatch skip slot). * All 4 editor routes: backend is canonical on load — the in-memory cell is overwritten with the deployed+draft overlay, the syncer's seed guard swallows the first write so we don't POST it back. * VariableEditor / ResourceEditor: drop the staleness pipeline + rev bookkeeping; backend wins on open. * useTriggerDraftSync.svelte.ts: inline the JSON-normalize + deepEqual utility as a private cfgDiffers helper (kept for the form-vs-deployed dirty check, which is a genuine semantic compare, not LS legacy). * copilot core.ts / userDraftAdapter.ts: drop meta argument from saveAppDraft, loadAppDraftValue, write*Draft. Test assertions on getMeta dropped. Net: -22 typecheck errors, fewer moving parts, conflict modal works. EOF ) * refactor(drafts): remove dead endpoints + UserDraftDbSyncer.getLastSync The list_drafts and get_draft (own) routes were added during PR iteration and never wired up to any frontend caller — the editor overlay path uses the per-kind get-by-path getDraft query parameter, and the home page lists drafts via the per-kind list endpoints, not via /drafts. Drop both routes (+ sqlx caches + OpenAPI entries). UserDraftDbSyncer.getLastSync was a peep-hole for callers that never materialised — the per-tab lastSync map is only ever read by postSave internally, where the bookkeeping already lives inline. * refactor(drafts): extract DraftEditorModals trailer block The four editor routes (scripts/flows/apps/apps_raw) mounted an identical pair of trailer modals — DraftSyncConflictModal + OtherUsersDraftsModal — wrapped in the same guard chain and {#key path} remount. Lift the markup into one component; routes thread their itemKind, path, editPathFor, and loader callback. Pure markup extraction, no state ownership change. Drops the unused userStore import where the trailer was the only consumer. * refactor(drafts): UserDraft.useReactive — kill array-of-one boilerplate The script + flow routes both wanted a handle that re-keys when the URL path changes. UserDraft.use() can't do that (its opts getter is untracked), so each route hand-rolled the same useMany-array-of-one + proxy idiom: const handles = useMany(() => [{ kind, path: reactive }]) const handle = { get draft() { return handles[0]?.draft }, ... } Add UserDraft.useReactive(getSpec) that internally wraps useMany with a single spec and returns the stable proxy. Callers collapse to one line. * refactor(drafts): unify bootstrap suspension via armRestartOnFirstInteraction The flow and raw-app routes each rolled their own end-of-bootstrap resume: a 700ms setTimeout for flows and a templatePicker watcher with double-tick gating for raw-apps. Both are timing-fragile (the comments admit it) and drift from each other. armRestartOnFirstInteraction already existed in userDraftToast.ts for reset-to-deployed: keydown/input/pointerdown listeners (capture phase) that fire restartSync on the first real user touch, with a 5s belt-and-braces fallback. Export it and use it everywhere we'd previously have picked a magic number. For raw-apps this is a tiny behavioural change: the user's template choice now POSTs immediately (the pointerdown that picks the template also resumes sync, so the picker's onStart write rides the wake-up). Previously the choice only persisted on the user's NEXT edit. That's strictly better — navigating away preserves the choice now. * refactor(drafts): type App.draft_path; drop the as-any cast The audit asked for the three editors to converge on one draft_path injection pattern. For App and Flow, the in-builder $effect-mutates- the-store idiom is wedged into a shape that doesn't natively own the field — App's editor type genuinely has no draft_path so the writer had to cast through `as any`, and consumers downstream did the same. The minimum viable fix: declare draft_path on the local App type (it's already a field on the autosaved JSON). Lifting the writes upward into a route-side merger would mean restructuring the AppEditor mirror $effect and the FlowBuilder pathStore plumbing — larger change for the same shape, deferred to a follow-up. Flow already has the typed cast localised at one site. Will get the OpenAPI-level draft_path field as part of task 47 (drop as-any casts on backend overlay reads). * refactor(drafts): extract makeDraftAddLoad helper Four identical /add/+page.ts files differing only by the edit-route prefix. Lift the redirect into a factory, slim each entry point to two lines. * refactor(drafts): type UserDraftOverlay.other_drafts_users in the OpenAPI The backend response carried other_drafts_users on every get-by-path that supports the draft overlay, but the OpenAPI schema didn't declare the field. Each route had to cast the typed response to `any` to read it (and the sibling draft_saved_at), which obscured the real shape from the type system and rotted the discoverability of the draft surface. Add it to UserDraftOverlay. Frontend casts collapse to plain property reads in the three editor routes. * feat(drafts): list & open draft-only items for variables, resources, schedules, triggers For scripts/flows/apps the list and get-by-path endpoints already surface per-user drafts that have no deployed counterpart — that's what gates the home page from 404'ing on an AI-agent-created draft. Extend the same support to the other UserDraftItemKinds: Backend (list endpoints): - Add include_draft_only to ListVariableQuery, ListResourceQuery, ListScheduleQuery, StandardTriggerQuery (the latter covers the 11 trigger kinds via the generic TriggerCrud). - Append per-user draft rows whose path has no deployed row. Same gate as scripts/flows/apps: non-operators, page 0, no narrowing filters. Synthesis is per-kind: ListableVariable/Resource get field-for-field synthesis; ScheduleLight reads NewSchedule shape; Trigger<T> uses a best-effort JSON merge + serde_json::from_value (rows skipped on deserialize failure rather than failing the list). - Add draft_only: Option<bool> with sqlx(default) to each row type so it serializes as the column is opt-in. Backend (get-by-path endpoints): - get_variable, get_resource, get_schedule, get_trigger<T> fall back to fetch_draft_only when the deployed row is missing and the caller passed get_draft=true. Mirrors scripts/flows/apps. OpenAPI: - Shared IncludeDraftOnly parameter under components/parameters, wired into the 11 trigger list endpoints + listRawApps. Inline declarations on listVariable / listResource / listSchedules / listAzureTriggers. - draft_only field on ListableVariable, ListableResource, Schedule, TriggerExtraProperty. Frontend: - variables, resources, schedules, and the 10 trigger list pages (routes + 9 *_triggers) pass includeDraftOnly: true on the initial fetch and render <DraftBadge draft_only> on synthesized rows. Trigger pages got a sed/perl bulk update — pattern is the same across kinds. * fix(drafts): swap crypto.randomUUID() for the project's randomUUID helper crypto.randomUUID() is gated on a secure origin (HTTPS or localhost). Self-hosted Windmill instances often run on a bare HTTP origin or a LAN IP where the WebCrypto API is unavailable, so the /add redirect would throw before issuing the 307. Use the existing RFC4122 v4 helper in FlowChatManager that the rest of the codebase already imports for this exact reason. * fix(editor): leading-edge fire + max-wait cap on Monaco debounce The Editor debounced `onDidChangeModelContent` purely on the trailing edge — every keystroke rescheduled a 500ms timer, and uninterrupted typing held the bindable `code` prop stale until a pause. Stacked behind our 1.5s autosave debouncer that meant our clock didn't even start ticking until 500ms after the user paused, and the `code` binding never updated mid-burst for downstream consumers (lint, live preview, change listeners). Switch to leading + trailing + max-wait: * First keystroke of a burst fires `updateCode` synchronously, then stamps a wall-clock chain start. * Each subsequent keystroke (re)arms a trailing timer at `min(now + changeTimeout, chainStart + maxChangeTimeout)` — the cap is what makes continuous typing materialize at least once per maxChangeTimeout window instead of indefinitely. * When the trailing fires it resets the chain so the next keystroke after a pause is a fresh leading fire. New prop `maxChangeTimeout` (default 1000ms) sits next to the existing `changeTimeout` (default 500ms). Dispose path clears the chain stamp alongside the timer. * feat(drafts): wire Ctrl/Cmd+S to flush the pending autosave immediately Each builder already had a Ctrl/Cmd+S keybinding routed through a saveDraft() no-op left over from the LS-era — the comment said "persistence happens via the page-level UserDraft autosave" but the shortcut was the user's only way to actually force a save without waiting for the 1.5s debounce. Restore the intent. * UserDraftDbSyncer.flush({ workspace, itemKind, path }) — new method that re-submits whatever's queued in pendingSaveOpts with immediate: true. No-op when nothing's pending. * Editor.svelte.flushPendingChanges() — exposes a synchronous updateCode() with chain reset, so callers can drain Monaco's own trailing debounce before asking the syncer to flush. Without this step a Ctrl+S within ~500ms of typing would POST the pre-burst content. * ScriptBuilder.saveDraft() — editor?.flushPendingChanges() → await tick() → UserDraftDbSyncer.flush(). Toast on result. * FlowBuilder.saveDraft() — no direct Monaco ref (flows have many per-module editors); just flushes the syncer. Editor.svelte's new 1s max-wait cap means at most the last <1s of typing in a module Monaco won't be in this POST; it follows in the next autosave round. * RawAppEditor.handleKeydown — adds a 's' case that flushes before the focus guard, so the shortcut fires regardless of where focus is in the editor pane. * fix(drafts): low-code apps — drop spurious autosave on /edit + remount on Load from server Two bugs in low-code app editor (raw apps use a separate code path): 1. Every /edit visit looked like an autosave because loadApp() called UserDraft.discard('app', path, undefined). The comment claimed "this load doesn't POST" but discard always POSTs value: null server-side — that surfaced as a DELETE-my-draft on every page load AND a flash in the AutosaveIndicator. The discard was originally intended to wipe the in-memory cell so AppEditor remounts "fresh". But the path-change $effect upstream already sets app = undefined before each loadApp, which unmounts AppEditor and releases the handle's entry — so a remount via app = backendApp naturally starts with an empty handle. Drop the discard. 2. The conflict modal's "Load from server" called loadApp() but didn't remount AppEditor. Since AppEditor's stateApp is captured once at mount and doesn't react to prop changes, the editor kept showing the conflicting local edits even after a successful reload. Wrap the onLoadFromServer to await loadApp() then bump redraw to force a fresh mount. * feat(drafts): home-page Draft badge — show user-initial circles, drop the '+' The home-page Draft badge previously showed '+Draft' as a flat label. Add per-user awareness: up to 3 user-initial circles render to the left of the label, ordered alphabetically; with 4+ users we collapse to the first 2 + a '+N' overflow circle so rows stay compact. Backend: * New `DraftUserRef { username: Option<String> }` in windmill-types::user_drafts, re-exported from windmill-common so the list endpoints in scripts/flows/apps crates share one import path (windmill-types/windmill-common can't be reordered without a cycle). * ListableScript / ListableFlow / ListableApp gain a `draft_users: Option<sqlx::types::Json<Vec<DraftUserRef>>>` field. The list SQL adds a per-row subquery `SELECT json_agg(...) FROM draft d LEFT JOIN usr u ...` that aggregates the workspace users with a per-user draft at this path. NULL (no drafts) decodes to None; LEFT JOIN against `usr` lets orphaned drafts (user removed from workspace) still surface with username = None. * Synthesized draft-only rows set draft_users to a single-element vector with the authed user (those rows come from `email = $2`). OpenAPI: `draft_users` added to listScripts / listFlows / ListableApp response shapes as an array of `{ username }` with nullable username. Frontend DraftBadge: * Accepts `draft_users: { username?: string | null }[]`. Renders up to MAX_CIRCLES (3) initial circles; at 4+ users renders first 2 + a gray '+N' overflow circle. * Initials: 'john.doe'/'john_doe' → 'JD', 'alice' → 'AL', the legacy NULL-email row → '?'. * Color picked deterministically from a 6-entry palette so the same user gets the same circle color across rows. * Label is now just 'Draft' (dropped the '+'). 'Draft only' is unchanged. * Tooltip lists every user in full. ScriptRow / FlowRow / AppRow thread `draft_users` through their prop types and pass it to DraftBadge. * fix(drafts): suppress 'You have unsaved changes' banner when deployed baseline is null A brand-new variable/resource/trigger (no deployed row yet) has `getDeployed() == null`, but the caller's `show` prop is computed off `current != deployed` which is trivially true while the user types. Result: the banner appeared with 'Show diff' (no-op — the drawer early-returns on null deployed) and a 'Discard' that's semantically backwards (there's nothing to revert to). Gate `show` internally on `getDeployed() != null`. The check sits in the banner rather than each caller because every caller would otherwise need the same boilerplate guard. * fix(drafts): hide LocalDraftBanner when deployed and current match the DiffDrawer's compare Earlier I gated the banner on `getDeployed() != null`, but the user still saw it fire on entries where 'Show diff' opens to 'No changes detected'. That means `show` (the caller's coarse dirty check) flagged a difference the DiffDrawer treats as a no-op — typically toggle defaults (`false ↔ undefined`), removed empty arrays, or key-ordering noise that `cleanValueProperties + orderedYamlStringify` collapses. Replicate the drawer's comparison inside the banner: stringify both sides through the same pipeline and only render when the keys differ. A single `diffKey()` helper keeps the logic local; the catch-and-empty fallback survives a non-serializable side rather than throwing. * ui(drafts): nest user-initial circles inside the Draft badge Previously the circles sat alongside the Badge in a parent flex container; the result read as two separate UI elements. The Badge component already exposes its children as a snippet rendered inside its own flex row, so moving the circles into it makes them feel like part of the same chip. Knock-on tweaks: shrunk the circles from h-4/w-4 to h-3.5/w-3.5 so the badge stays compact, and tinted each circle's ring with the badge's indigo palette (instead of plain white) so the overlap reads as a deliberate stack rather than dots floating on top of the chip. * feat(drafts): drop the authed user's circle, mark own drafts with a '*' suffix Three tweaks to the home-page Draft badge: 1. Filter the authed user out of `draft_users` before rendering circles. The row already signals 'this user has a draft' via the asterisk (below), so a circle for them would be redundant noise. New `currentUsername` prop on DraftBadge — pass `$userStore?.username` from each row. The tooltip still lists every user (with `(you)` next to the authed one) so the full picture is one hover away. 2. The badge already showed whenever `is_draft || draft_users.length > 0` (per-user OR any-user). Spelled the rationale out in a comment — no logic change. 3. Append '*' to the displayed summary when `is_draft` is true. Falls back to `draft_path`/`path` when summary is empty so the marker never decorates an empty string. Threaded the same expression into ScriptRow / FlowRow / AppRow. Slice/overflow math now keys on the post-filter `otherUsers` list, so dropping the authed user doesn't silently shrink the visible count (e.g. 3 users incl. self → 2 circles, not 1 circle + a '+1' bubble). * feat(drafts): clone per-user drafts when forking a workspace `clone_workspace_data` clones every other workspace-scoped table on fork creation (resources, variables, scripts, flows, apps, raw apps, triggers, schedules) but quietly dropped the `draft` table. With per-user drafts that meant any open editor in the parent lost its pending edits the moment a fork was created — surprising and inconsistent with how forks treat the deployed surface. New `clone_drafts` mirrors the existing clone helpers: a single INSERT...SELECT into the target workspace, preserving `path`, `typ`, `value`, `created_at`, and `email`. The `email` FK targets `password.email` which is instance-scoped so it carries across workspaces without remap. `created_at` is preserved on purpose so the per-tab `last_sync` baseline lines up with the parent's timeline — otherwise the fork's next autosave would race a stale `last_sync` and trip the conflict modal on every cloned draft. Plain INSERT (not UPSERT) is safe because the fork target is empty at create time; no conflict against the partial unique indexes (`draft_pkey_with_user` / `draft_pkey_legacy`). The synthetic BIGSERIAL `id` PK is regenerated by the default so it stays out of the column list. * ui(drafts): pin the authed user to the first circle instead of hiding them Previously the authed user was filtered out of the circle row entirely on the theory that the row's '*' suffix already signalled 'this user has a draft'. New requirement: they should always lead the circle row when they have a draft so the visual half of the signal lines up across rows (consistent leading-slot identity, easy scan). Switch from a filter to a sort: `orderedUsers` finds the authed user in `draft_users` and splices them to index 0; everyone else keeps the backend's alphabetical order behind. Slice/overflow math now keys on `orderedUsers`, which guarantees the authed user never falls into the '+N' bubble — they're at position 0 and the slice keeps the head. The popover's '(you)' annotation moves to the circle's title attr too, so hovering the leading circle confirms the identity. * feat(drafts): drop draft_only column from script/flow/app Drafts now live in the `draft` table exclusively — `draft_only` stubs in script/flow/app are redundant. Migration `INSERT INTO draft ... ON CONFLICT (workspace_id, path, typ) WHERE email IS NULL DO NOTHING` so real per-user drafts already at the same path are preserved; only rare stubs that lost their draft get a synthesised workspace-level row. Stubs are then deleted (FKs cascade to *_version) and the column is dropped. List endpoints keep a synthesised `draft_only: true` on rows sourced from the draft table itself (sqlx default on the struct field). Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * ui(drafts): surface draft state in AutosaveIndicator instead of toast+auto-modal The "Loaded your saved draft" toast and the auto-opening OtherUsersDraftsModal both surprised users on every editor mount. Move both signals into the AutosaveIndicator label: "Loaded from draft" or "Others are working on this {kind}" (priority) sits where Saving/Saved do, with a one-shot light-green flash behind the indicator that fades to transparent. Saving/Saved still win when they fire. The popover gains a "See others' drafts" button that flips the modal open on demand; the modal itself is now externally controlled via a bindable \`isOpen\` threaded through DraftEditorModals. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * ui(drafts): per-user View JSON / Fork actions in DraftBadge popover Hover popover used to be a plain text list of usernames. Now each row gets a colored circle icon + name + "(you)" for the authed user, and every OTHER user's row carries View JSON / Fork buttons mirroring the OtherUsersDraftsModal. For draft-only entries owned solely by the authed user, the popover ends with "Only you can see this {kind}" so the row's privacy is obvious. ScriptRow / FlowRow / AppRow thread workspace + itemKind + path + editPathFor through; AppRow switches between app / raw_app on app.raw_app. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * nit * fix(drafts): clone only the forker's per-user drafts on workspace fork clone_drafts copied every user's drafts, but only the forker gets added to the fork's usr table. Drafts owned by absent users LEFT-JOIN to NULL in the home page's draft_users aggregate, surfacing as multiple legacy-style rows at one path and crashing the popover with each_key_duplicate. Filter the clone to email = forker OR email IS NULL, and key the popover's #each by index defensively so future legacy collisions can't crash the page either. Also re-adds `draft_only: None` to NewScript/CreateFlowBody literals in tests — the auto-generated windmill-api-client still carries the field and the previous commit dropped them too aggressively. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * fix(drafts): always populate other_drafts_users in maybe_overlay_draft Reset-to-deployed reloads the deployed payload with get_draft=false, which made the backend return other_drafts_users=[]. The route then reassigned otherDraftsUsers to the empty list, dropping the count to 0 and hiding "See others' drafts" in the AutosaveIndicator popover — but the other users' drafts hadn't actually gone anywhere. Fetch the list independently of get_draft so the popover stays accurate across reset reloads. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * feat(drafts): alert user when their draft is older than the latest deploy Open a modal on editor mount when the per-user draft was saved before the latest deploy at the same path — i.e. a teammate deployed a new version while this user's draft was sitting. Two choices: discard the stale draft and pick up the deploy, or keep editing the older draft. DraftEditorModals computes the staleness from the timestamps each route threads in (script.created_at, flow.edited_at, app_version.created_at) and the "Load latest deploy" callback reuses the route's existing reset-to-deployed logic. Wired for script / flow / app / raw_app editors; trigger / resource / variable drawer editors follow a different pattern and aren't covered here. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * fix(drafts): deploy only wipes the deployer's draft, not everyone else's Script / flow / app deploys ran an unconditional DELETE on every draft at the path, so a teammate's deploy silently destroyed any other user's pending draft. After the wipe, the other user's tab kept auto-saving — re-creating the row at a NOW timestamp newer than the deploy — and StaleDraftModal never fired because draft_saved_at had been bumped past the deploy. Filter the DELETE to email = deployer (plus the legacy NULL row), so other users' drafts persist and the stale-draft prompt actually fires on their next reload. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * fix(drafts): surface save failures in AutosaveIndicator instead of pretending Saved postSave caught network errors with `console.error` and let the runner finish normally. The indicator read the saving → none transition as a successful save and flashed "Saved" even when the request had thrown. Track failed keys in a SvelteMap, expose `'failed'` as a new UserDraftSyncState, render "Save failed" in red with a CloudOff icon. Failure clears on the next successful save for the same key, or when recordRemoteSync seeds a fresh authoritative timestamp. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * fix(drafts): surface 'Save failed' inside the AutosaveIndicator popover too The popover used to repeat the cheerful "All changes are saved as a draft on the server..." copy even when the inline label said "Save failed", which read as contradictory. Add a red, text-xs warning at the top of the popover body when the sync state is `failed`, explaining that the latest edits didn't reach the server and that editing again retries the save. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * fix(drafts): surface the actual error message in the AutosaveIndicator popover Replace the generic "your latest changes did not reach the server" copy with the real failure detail. The syncer now stores the extracted message in the failures map (formatSaveError walks body / message / statusText) and exposes it via the state handle's `failureMessage` getter. Popover renders it in red, monospaced, scrollable so a long server traceback doesn't blow out the popover. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * fix(drafts): suppress Saving/Saved indicator during a reset-to-deployed discard A `value: null` POST is a discard, not a save, but it ran through the same runner the indicator watched — so resetting to deployed flashed "Saving..." → "Saved", reading as "your draft just landed" while we were actually wiping it. Track in-flight discards in a SvelteSet, expose a distinct `'discarding'` UserDraftSyncState, and the indicator stays quiet for it: no spinner, no label change, and the `discarding → none` transition deliberately skips the "Saved" flash. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * Revert "fix(drafts): suppress Saving/Saved indicator during a reset-to-deployed discard" This reverts commit 625a47c5d2. * fix(drafts): flush pending autosaves when the editor hook unmounts SPA navigation doesn't fire `pagehide`, so a debounced edit (up to maxDebounceMs old) silently disappeared when the editor was unmounted mid-typing. `UserDraft.useMany`'s onDestroy now walks every acquired entry and fires `UserDraftDbSyncer.flush(query)` before releasing, re-submitting the pending opts with `immediate: true`. The POST rides the runner's own lifetime and survives the component teardown. `use` / `useReactive` are thin wrappers around `useMany` so they inherit the flush automatically. Editors that don't go through the hook (sessions' `ScriptEditorView`, `AppJsonEditor`, copilot adapter, DraftBadge fork action) only call `UserDraftDbSyncer.save` for one-shot operations and don't need lifecycle flush. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * nit * feat(ui): Modal2 fixedHeight='adaptive' sizes the modal to its content The fixed-height steps force either wasted whitespace or clipped content for small dialogs. `adaptive` emits no height rule (still capped by max-h-screen-80) so the modal hugs its content. Use it in StaleDraftModal, which only has two lines of copy and a button row. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * feat(drafts): 'Create test drafts' button on the home page Dev/QA helper that seeds one per-user draft for every supported kind (script, flow, app, raw_app, trigger_schedule, resource, variable) at fixed u/{me}/draft_<kind> paths, so the draft surfaces (home badges, editors, stale-draft modal, others' drafts modal) can be exercised without hand-creating items. Re-clicking overwrites the same paths. Value shapes mirror what each editor's autosave writes, matching the backend list synthesizers that parse them back. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * fix(drafts): dedupe app list rows when a path holds both app and raw_app drafts The apps list LEFT JOINed draft with typ IN ('app', 'raw_app') for the is_draft flag — a path holding BOTH kinds for the same user (easy to hit: open a raw-app draft path in the regular app editor and its autosave writes the second kind) fanned the row out into two identical entries and crashed the home list with each_key_duplicate. Join a DISTINCT (path, workspace_id) subquery instead. Same dedup for the draft-only synthesis block via DISTINCT ON (path) keeping the most recently saved kind. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * feat(drafts): asterisk on resource/variable/schedule/trigger rows with own draft Add an is_draft flag to ListableVariable / ListableResource / ScheduleLight / BaseTrigger list rows — a scalar EXISTS subquery on the draft table for the authed email (no join, so no row fan-out), plus is_draft: true on the synthesized draft-only rows. The list pages (variables, resources, schedules, all trigger kinds) append `*` to the displayed name when set, mirroring the home page's convention. Also fixes draft-only resources never appearing on the resources page: the page always lists with resource_type_exclude=cache,state,app_theme (its tab split) and the synthesis gate bailed on any type filter. The gate now keeps synthesizing and applies resource_type / resource_type_exclude per-row against the draft JSON instead. list_triggers (trait default) takes an authed_email: Option<&str> — Some from the list endpoint, None from workspace export. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * Revert "feat(drafts): 'Create test drafts' button on the home page" This reverts commit 1f244a2a8b. * fix(drafts): P1 hardening — save authz, secret scrubbing, hot-path index 1. save_draft had no authorization check (a regression from the old create_draft's require_writer_of_path): any workspace member could plant drafts in another user's u/ namespace or unwritable folders, and those drafts get surfaced to every reader of the path (home circles, others'-drafts modal, View JSON / Fork). New require_can_write_path: admins; own u/ namespace; g/ namespace when in the group; f/ folders with the write/owner bit (with the same folder-claim refresh deploy endpoints use). Operators are rejected outright — they're excluded from every other draft surface. 2. Secret variable values were persisted in the draft table in plaintext. save_draft now blanks variable.value for is_secret drafts at write time (the editor never round-trips secret values anyway — it fetches with decrypt_secret=false), and a migration scrubs rows persisted before the guard. 3. fetch_other_drafts_users runs on every get-by-path request with (workspace_id, path, typ) and no email predicate — neither partial unique index covers it, so it seq-scanned a table that accumulates per-user autosaves across all workspaces. Add a plain btree index; it also serves get_draft_for_user's IS NOT DISTINCT FROM lookup. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * fix(drafts): Ctrl/Cmd+S flush narrates via the indicator, not a toast The "Draft saved" toast fired even with the network down — flush never rejects (postSave catches errors internally and routes them to the failures map), so the success branch always ran. Drop the toasts from the script / flow / raw-app Ctrl+S handlers; the AutosaveIndicator already narrates the flush truthfully (Saving... → Saved / Save failed in red). Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * fix(drafts): Ctrl/Cmd+S always flashes Saved in the indicator After dropping the toast, an explicit Ctrl/Cmd+S with nothing pending (the common case — autosave already landed everything) gave zero feedback: flush() no-ops when pendingSaveOpts is empty and no state transition fires. flush() now bumps a reactive per-key counter on completion (no-op path included), exposed as flushCount on the state handle; the AutosaveIndicator flashes "Saved" on the bump when the pipeline is idle. Real flushes keep narrating through Saving... → Saved / Save failed as before. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * ui(drafts): Ctrl/Cmd+S replays the green backdrop flash on the indicator Decouple the one-shot light-green → transparent backdrop from the load hint label: triggerFlash() owns the keyed span (mounted only while the animation runs), and both the on-mount hints and the Ctrl/Cmd+S confirmation route through it. The flush bump fires after the POST lands, so a real flush flashes too — not just the no-op path. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * feat(drafts): 'Create test drafts' button on the home page Dev/QA helper that seeds one per-user draft for every supported kind (script, flow, app, raw_app, trigger_schedule, resource, variable) at fixed u/{me}/draft_<kind> paths, so the draft surfaces (home badges, editors, stale-draft modal, others' drafts modal) can be exercised without hand-creating items. Re-clicking overwrites the same paths. Value shapes mirror what each editor's autosave writes, matching the backend list synthesizers that parse them back. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * fix(drafts): Ctrl/Cmd+S reaches the raw-app flush from every editor surface The raw-app window keydown handler never fired in practice: the file editor is a VS Code workbench in a same-origin iframe (keydowns don't cross documents) and the inline-script / YAML Monacos swallow Ctrl+S via addCommand. Two hooks: - attach a capture-phase keydown listener inside the iframe document on each load (no preventDefault — VS Code's own save still runs, we flush the pending autosave alongside it); - Editor.svelte / SimpleEditor.svelte re-broadcast their swallowed Ctrl+S as a `wm-monaco-save-shortcut` window event, which RawAppEditor listens for. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * fix(drafts): editing a draft-only item opens create mode prefilled from the draft Variable / resource / schedule / trigger editors treated every loaded path as deployed and routed saves through the update endpoints, which 404 for draft-only items ("Resource not found at name ..."). The get-by-path responses already mark the case (`no_deployed` from fetch_draft_only) — editors now flip to create mode when it's set: - VariableEditor / ResourceEditor: existedInitially = !no_deployed - ScheduleEditorInner + all 10 trigger editor inners: loadTrigger / loadSchedule return { overlay, noDeployed } and openEdit sets edit = !noDeployed The form opens prefilled from the draft and deploys via create, whose endpoints already delete the creator's draft on success. (The "Could not load schedule: Not Found" half of the report was a stale dev backend — getSchedule?get_draft=true verified working on the current build.) Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * fix(drafts): leading-edge draft saves for raw apps (no double debounce) Raw-app file changes reach the parent already coalesced — the UI Builder iframe holds a ~1s trailing debounce on its rebuild and only posts setFiles when it fires. The syncer then stacked its own 1.5s trailing window on top, so the draft landed ~2.5s after the user stopped typing. The debouncer now supports a leading edge (run immediately when the key is idle and cooled down; later schedules in the window coalesce trailing with the max-wait ceiling, mirroring the classic editor's first-keystroke-materializes-immediately logic), and raw_app saves opt into it. The app build keeps its own trailing debounce inside the iframe — only draft persistence is affected. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * ui(drafts): blue flash for load hints, green for save confirmations The backdrop flash now carries meaning: green = "your save landed" (Ctrl/Cmd+S), blue = informational on-mount hints ("Loaded from draft", "Others are working on this ..."). Color is passed as an inline CSS custom property the keyframe reads, so the single keyframe serves both variants. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * Revert "fix(drafts): leading-edge draft saves for raw apps (no double debounce)" This reverts commit 1b996fd73a. * feat(drafts): 'Enable auto-save' toggle in the AutosaveIndicator popover Browser-wide preference (default on, persisted in localStorage). While off, the reactive keystroke mirror never POSTs — saves marked `auto: true` park their latest opts in pendingSaveOpts instead of scheduling, and the unload keepalive flush is skipped, so nothing leaves the tab except explicit actions: Ctrl/Cmd+S flush (sends the parked latest content), discard / reset-to-deployed, fork, conflict overwrite. The indicator shows a muted cloud-off while disabled (the idle check-mark would otherwise read as "everything saved") and the popover copy explains the Ctrl/Cmd+S-only behavior. Re-enabling re-schedules every parked unsaved draft so edits made while off catch up immediately. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * Revert "feat(drafts): 'Create test drafts' button on the home page" This reverts commit fd7013b399. * feat(drafts): Review & Deploy covers variables/resources/schedules/triggers The drafts review page only assembled scripts/flows/apps from three paginated list endpoints, so drafts of every other kind were invisible. New GET /w/{ws}/drafts/list returns every draft of the authed user in one query over the draft table, with a per-kind draft_only flag (deployed-table EXISTS per kind); getDraftItems switches to it, which also drops the 3×N-page fan-out. CompareDrafts renders the new kinds (icon via a UserDraftItemKind → layout-Kind mapping, gray kind badge, list-page edit links for drawer-based editors), diffs them through a generic overlay GET, and deploys them by replaying the editor save: create/update for variables and resources, saveScheduleFromCfg for schedules, the per-kind save*TriggerFromCfg helpers for the ten standalone trigger kinds. Also fixes two paths stale since the draft_only column removal: draft-only flows/apps now deploy via create (update 404s — there is no row anymore), and discard always deletes the draft row (the old delete-the-item branch 404'd for the same reason). Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * feat(drafts): optimistic asterisk while editing in list-page drawers The `*` suffix on variable/resource/schedule/trigger rows came from the server's is_draft flag, which only updates on a refetch — editing an item in the drawer didn't mark its row until much later. New localDraftHints module (SvelteSet-backed): editors publish their dirty state (the same condition that shows the "You have unsaved changes" banner) and the 13 list pages OR the hint into the asterisk condition, so the suffix appears the moment the form diverges and clears on discard/teardown. Wired once in useTriggerDraftSync (covers the schedule editor and all ten trigger editors) plus VariableEditor and ResourceEditor. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * ee repo * fix(drafts): draft hints persist past editor teardown, re-sync on reopen Clearing the optimistic asterisk on drawer close was wrong: the divergence the editor observed is autosaved server-side, so the draft outlives the drawer and the asterisk should too. Hints are now corrected rather than expired — while an editor is settled on an item it publishes the observed truth in both directions (divergence sets, sitting at the deployed baseline clears), so a draft discarded from another tab loses its stale asterisk the next time the item is opened. No teardown cleanup anywhere. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * fix(drafts): list-page asterisk mirrors the editor's banner, not stale is_draft The asterisk was `is_draft || hint` — an OR can turn the asterisk on optimistically but can never turn it OFF, so after discarding a draft (or editing back to the deployed value) the stale server flag kept the asterisk until the next list refetch. Make the local hint a tri-state override instead: the editor publishes the live banner state (true/false) into a SvelteMap, and the list pages read `getLocalDraftHint(...) ?? is_draft` — the editor's observed truth wins over the stale server flag in both directions. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(drafts): autosaves equal to the deployed value delete the draft instead When the user edits back to exactly the deployed value, the reactive autosave mirror used to persist a baseline-equal copy — a useless draft row that kept `is_draft` (and the list asterisk) on after refetch. Add a `discardIfEqualTo` baseline getter to `UserDraft.useMany` specs: when the cell's value deep-equals the deployed baseline, the mirror POSTs `value: null` (delete) instead of the value. The variable and resource editors pass their `initialStates` baseline, guarded on `existedInitially` — draft-only/new items have no deployed copy, so equality must never delete their only data. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * Draft encryption for secret variables * fix(drafts): discardIf predicate + deploys clear the asterisk and draft row Two follow-ups on the baseline-equal-autosave-deletes change: 1. `discardIfEqualTo` (baseline getter + raw deepEqual) becomes `discardIf` (predicate). Raw deepEqual reported spurious diffs after a refresh: drafts round-trip through JSON, which strips undefined-valued keys, so a restored draft (`{}`) never compared equal to the freshly built baseline (`{ labels: undefined }`) and the delete never fired. The editors now pass the SAME comparison that drives their "unsaved changes" banner — a new exported `draftValuesEqual` (JSON-normalized deep equality) used by both — so the banner and the synced draft can never disagree. 2. Truly saving (deploying) clears the asterisk and the draft row: - variable/resource editors: replace post-deploy `UserDraft.remove` (blanks the cell to `undefined`, which reads as dirty and keeps the banner + asterisk on) with `discard` to the just-saved state, and refresh `initialStates`/`existedInitially` so the editor settles clean. - trigger editors: `useTriggerDraftSync.discard` publishes the hint off explicitly — after a deploy the editor's `deployed()` baseline is stale, so the hint effect alone would keep the asterisk on. - Review & Deploy page: `deployDraft`/`discardDraft` clear the hint. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * revert encryption just for the resources part * fix(drafts): required const DRAFT_KIND on TriggerCrud; deploy/delete cover raw_app The TriggerCrud::user_draft_item_kind() default matched on TRIGGER_TYPE and panic!'d on any unmapped string — a runtime crash on the first draft save for a trigger that forgot to map. Replace it with a required associated const DRAFT_KIND, so a missing mapping is a compile error. user_draft_item_kind() now just returns Self::DRAFT_KIND; every impl (OSS + EE) declares the const. Also fix the app deploy/delete draft cleanup to cover raw_app: raw apps deploy and delete through the same internal path, but the cleanup filtered typ = 'app' only, leaving raw_app drafts dangling (create_app_internal apps.rs:1465, update path apps.rs:2077) or un-archived on delete (apps.rs:1687). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(drafts): deleting an item wipes every user's draft, not just the caller's Scripts/flows/apps already wiped all users' drafts on delete, but resources/variables/schedules/triggers called delete_user_draft (caller-scoped), so a teammate's draft on the just-deleted item lived on forever — surfacing through fetch_other_drafts_users with no item left to deploy onto. Add delete_all_drafts_for_path (all emails + the legacy NULL row) and use it in every delete handler; keep delete_user_draft for the discard-my-own-draft flow where the item lives on. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * perf(drafts): skip other-drafts query on non-editor reads (get_draft=false) maybe_overlay_draft ran fetch_other_drafts_users (a usr join) on every get-by-path, including worker/CLI reads of MB-scale flows & apps that pass get_draft=false and never render the draft overlay or "others editing" surfaces. Gate the query behind get_draft — only editor reads pay for it. Reset-to-deployed editor reloads still get it (they pass get_draft=true). (Eliminating the serde_json::to_value materialization of the deployed payload needs WithDraftOverlay to become generic over T, which is folded into the get-by-path choreography refactor.) Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * refactor(drafts): single-source the kind→table mapping via deployed_table() The kind→table dispatch lived in three places that could drift: the TriggerCrud string-match (already replaced by const DRAFT_KIND), the table_for_kind access-check map, and a hand-written draft_only CASE in list_drafts. Add UserDraftItemKind::deployed_table() as the single source (plus an ALL enumerator). table_for_kind now delegates to it, and the list_drafts draft_only CASE is generated from it at runtime (table names come from the closed enum, never user input — no injection). Drift between the access check and the existence check is now impossible by construction. Webhook and the native triggers (poll/cli/nextcloud/google/github) map to None: they have no path-keyed backing table and aren't draftable, so they report draft_only=true and use a path-only access check. This also fixes a latent bug where table_for_kind mapped native kinds to native_trigger, which has no `path` column — the access query `SELECT 1 FROM native_trigger WHERE path = $1` would have errored. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * ee repo * fix(drafts): close variable draft-secret laundering oracle (sentinel + rehydrate) save_draft encrypts secret variable values with the workspace key, but the ciphertext was round-tripped to the client and the deploy endpoints decrypted whatever $encrypted: ciphertext the client submitted (variables.rs create/update). Any workspace member who can write a variable path could take an arbitrary workspace-key ciphertext (another user's secret draft via GET /drafts/get with only path-read, or a deployed secret's stored value) and submit it as their own secret variable's value — the server decrypted it and, since they own the path, they read the plaintext back. That bypasses the audited decrypt_secret permission. Fix: the ciphertext never leaves the server. get_variable swaps a draft secret's $encrypted: value for an opaque $draft_secret sentinel (both the draft overlay and the draft-only inner stand-in). On deploy the client sends the sentinel back and the server rehydrates the plaintext from the caller's OWN draft row — the only ciphertext it ever decrypts is one it encrypted for this exact (workspace, path, email). A raw $encrypted: submitted by a client is now rejected outright. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(drafts): don't clobber a secret draft when autosaving the $draft_secret sentinel After reload the client holds the $draft_secret sentinel for a secret variable (never the ciphertext). Editing some OTHER field (description, labels) triggers an autosave carrying value="$draft_secret" — and save_draft's encrypt_secret_variable_value, seeing a non-empty, non-$encrypted: string, encrypted the literal sentinel, overwriting the real ciphertext in the draft row and losing the secret. Treat the sentinel as "secret unchanged": restore the $encrypted: ciphertext already stored in this user's draft row instead of encrypting the placeholder (falling back to empty only if there's no prior ciphertext). The new lookup reuses the same query shape as the deploy- time rehydrate, so no new offline cache entry. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * Revert "$draft_secret" sentinel approach for variable draft secrets Reverts 339c259fce and b2c38ef407. Instead of round-tripping a sentinel and rehydrating server-side, we close the laundering vector more simply by disabling cross-user draft visibility for triggers/resources/variables (next commit) — an attacker can no longer read another user's secret draft ciphertext to launder it. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * feat(drafts): keep drafts private to their owner for resource/variable/trigger kinds Replaces the reverted $draft_secret sentinel: instead of laundering-proofing the ciphertext round-trip, simply don't expose other users' drafts for the drawer kinds (resource/variable/triggers). A viewer can no longer obtain another user's secret-variable draft ciphertext, so it can't be laundered into plaintext via deploy. UserDraftItemKind::shares_drafts_across_users() — true only for script/flow/app/raw_app. maybe_overlay_draft skips other_drafts_users for non-sharing kinds, and get_draft_for_user (View JSON / Fork) returns 404 for them. Own-draft load/save is unchanged. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * refactor(drafts): make the list-page asterisk hint a shadow of UserDraftDbSyncer The optimistic `*` hint was written by three open-editor publishers, so draft deletions that didn't go through an editor (banner discard, autosave-back-to-baseline, Review & Deploy) left a stale asterisk that a server refetch couldn't clear (the hint overrides is_draft). Move ownership to the syncer — the one choke point where a draft's existence actually changes: - postSave sets the hint on a saved write (value !== null) and clears it on a delete (null), so every syncer-routed delete clears it for free. - save() lights it optimistically when a real save is scheduled, so the asterisk still tracks the editor's banner without the debounce lag. The editors no longer SET the hint; they only CLEAR it when settled at the deployed baseline (so a draft discarded from another tab disappears on reopen). discardDraft drops its explicit clear (postSave covers it); deployDraft keeps one (it deletes server-side, bypassing the syncer). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * chore(migrations): fold draft index + secret scrub into the base sync migration Merge 20260610095349_draft_workspace_path_typ_index and 20260610100018_scrub_secret_variable_drafts into the base 20260528143710_draft_user_sync_schema migration (the index creation + secret-draft scrub in .up, the index drop in .down; the scrub stays irreversible). 20260609165313_remove_draft_only remains standalone. Verified the full chain applies and reverts cleanly on a fresh DB. (Rewrites an already-applied migration — existing dev DBs need a reset.) Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * refactor(drafts): promote the get-by-path draft choreography to one helper The "Some(deployed) → overlay / None+get_draft → draft-only / None → 404" dance was copy-pasted across the get-by-path handlers and had drifted (different 404 text, the trigger one missing the draft-only fallback at first). Promote it to windmill_common::overlay_or_draft_only<T>, which takes the deployed entity as Option<T> and a per-route not_found closure. Converts scripts, flows, apps, schedules, and triggers onto it. Resources keeps its own (it runs an async explain_resource_perm_error on the 404 path) and variables keeps its own (secret-decrypt logic interleaved with the draft fetch) — both genuinely diverge from the common shape. (The serde_json::to_value elimination via a generic WithDraftOverlay<T>, and the list-only draft synthesis dedup, remain as follow-ups.) Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * perf(drafts): serialize the deployed overlay payload in one pass maybe_overlay_draft materialized the deployed entity into a serde_json::Value tree (serde_json::to_value) and then serialized that tree again into the response — two passes plus a full Value allocation over what can be an MB-scale flow or app, on every get-by-path (including get_draft=false worker/CLI reads). Hold WithDraftOverlay.inner as a boxed erased_serde::Serialize trait object instead, so the deployed payload flattens straight into the response in one pass. The struct stays non-generic, so the helper and all seven handler return types are unchanged; only the deployed type now needs Send + 'static (already true — they're owned rows; added 'static to TriggerCrud::Trigger to say so). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * refactor(drafts): one helper for the draft-only list synthesis query The "draft rows at paths with no deployed counterpart" query was copy-pasted into the variable / resource / schedule / trigger list handlers, each hardcoding its own typ literal and NOT EXISTS table — a drift hazard. Promote it to windmill_common::fetch_draft_only_list_rows, which derives the absence-check table from kind.deployed_table() (the same single source as the access check and draft_only flag). Each handler keeps its own include_draft_only gating and per-type row mapping (genuinely entity-specific); only the shared SQL is deduped. The trigger handler's prior generated-SQL version is folded in too. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(drafts): route raw-app draft deploys through the raw-app endpoint [P1] deployDraft's raw-app guard was `kind === 'app' && rawApp`, but Review & Deploy passes `kind === 'raw_app'` (raw apps are their own DRAFT_KIND), so the guard never fired and the row fell into the visual-app branch. There `d.value` is undefined (a RawAppDraft has files/runnables/data, no `value`), so AppService.updateApp did a partial update — resetting policy to the publisher default, never bundling/deploying the files — the backend then deleted the user's raw_app draft rows, and the UI reported "deployed". The work-in-progress was destroyed without ever deploying. Route `kind === 'raw_app'` (or the editor's `app` + rawApp) through deployRawAppDraft. The now-unreachable `raw_app` arm of the visual-app branch is dropped. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(drafts): allow draft saves for item-level extra_perms writers [P1] require_can_write_path only accepted namespace rules (own u/, member g/, writable f/), dropping the item-level extra_perms check the old create_draft had. A user granted write on e.g. u/alice/script via the Share dialog could still deploy it (the update endpoints go through RLS) but could no longer save a draft — and because the editors autosave continuously with no permission gate, editing a shared item produced a persistent "Save failed: you don't have write permission" and Ctrl/Cmd+S failures. Add the item-level fallback: when a deployed row exists at the path, check its extra_perms for a write grant (every deployed table has extra_perms; the table comes from the closed deployed_table() mapping). Draft-only items have no row and stay governed by the namespace rules. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(drafts): pass rawApp on get-app for never-deployed raw-app drafts [P2] A raw app that has only ever been drafted has no `app` row, so get_app resolves the draft kind from the `rawApp` query param. getDraftDiffValues ("Show diff") and deployRawAppDraft both fetched with getDraft=true but without rawApp, so the backend looked up the visual-app draft kind, found nothing, and 404'd. Pass rawApp so the raw_app draft is found. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * feat(drafts): surface the localStorage→DB migration with toasts migrateUserDraftsToDb already uploaded legacy "userdraft/..." entries and cleared them on success (and runs after the v1→userdraft normalizer). Add the user-facing surface: when real legacy entries are detected, show an info toast "Migrating local storage drafts ..."; on a per-draft failure show an error toast "Could not migrate draft <path> in workspace <X>" with a "Delete draft" action that drops the stuck localStorage entry (otherwise it retries every mount). Unparseable junk is still cleared silently up front, so the toast only fires for genuine drafts. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * test(drafts): cover the autosave pipeline's pure-logic utilities [P2] The deleted draft tests left the new debouncer + coalescing runner — the core of the autosave pipeline — with zero coverage. Add vitest suites (16 cases) for debouncerByKey (debounce window, latest-task-wins, maxDebounceMs ceiling under a trickle, fresh-chain-after-fire, cancel, key independence) and coalescingRunner (immediate run when idle, coalesce burst to in-flight + latest, displaced-task drop, submitAndWait resolve/reject/displaced, cancel semantics, key independence). Broader replacement (save_draft conflict semantics + the require_can_* checks as backend integration tests) still outstanding. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * feat(drafts): add UserDraft.seed — a one-shot baseline load that never POSTs The page editors bracket their new-draft / deployed-baseline loads with stopSync + restartSync so the programmatic write isn't synced as the user's edit. Forgetting restartSync silently disables autosave for the session — the footgun behind the three divergent resume strategies the review flagged. `UserDraft.seed(kind, path, value)` is the scoped alternative: it sets the cell (all reactive readers update) and arms a single-shot `seedNextWrite` flag the sync effect consumes — adopting the value as the new baseline and skipping exactly that one POST, with no suspension to resume. Additive: stopSync/restartSync are untouched and still used for the writes that fan out across editor components (initContent cascades). Foundation for converting the editor bootstraps off the bracket. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * refactor(drafts): extract usePageDraftSync; convert the scripts editor onto it First step of unifying the four page editors' hand-rolled draft orchestration (three divergent handle-ownership models + an easy-to-forget recordRemoteSync). usePageDraftSync is the single model — the page analogue of useTriggerDraftSync — owning the re-keyed autosave handle, the live-editor-draft registry entry, recordRemoteSync (now a method, not a per-page ritual), seedBaseline (via UserDraft.seed), and draft removal. The scripts editor is converted as the reference adoption: its inline useReactive handle, live-editor-draft effect, recordRemoteSync, and the two UserDraft.remove calls now go through draftSync. The new-draft stopSync bracket stays (it spans ScriptBuilder's initContent cascade). Verified in a real browser against the dev stack: load fires no spurious save, a code edit triggers exactly one save_draft POST + a draft row, and the draft persists across reload. Flows / apps_raw / apps conversions follow. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * refactor(drafts): convert the flows editor onto usePageDraftSync Replace the inline useReactive handle + UserDraftDbSyncer.recordRemoteSync + UserDraft.remove with draftSync. effectivePath is omitted — flows register their live-editor-draft entry through FlowBuilder (liveEditorDraftStoragePath), so the composable doesn't double-register. The new-draft stopSync + armRestartOnFirstInteraction bracket stays (it spans FlowBuilder's seed cascade). flowStore reads/writes draftSync.draft. Verified in a real browser: load fires no spurious save, a summary edit triggers exactly one save_draft POST + a draft row, and the edit persists across reload. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * refactor(drafts): convert the apps_raw editor onto usePageDraftSync Replace the UserDraft.use handle + mirror, UserDraftDbSyncer.recordRemoteSync, and UserDraft.remove with draftSync. `path` is a mount-scoped plain `let` (the editor remounts per path), so the composable's useReactive re-keys only on workspace change — equivalent to the prior capture-once use(). effectivePath omitted (RawAppEditor owns the live-editor-draft entry); the new-draft stopSync + armRestartOnFirstInteraction bracket stays. Type-checked and behavior-equivalent (handle mechanism unchanged; the centralized recordRemoteSync/remove read the same `path`). Not browser-exercised here — no existing raw app in the dev workspace and the new-draft template-picker flow isn't scriptable quickly; scripts and flows (same composable) were verified live. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(drafts): remove app autosave at its canonical key after deploy/rename AppEditor keys the app autosave on the URL draft path and passes it down as userDraftPath, but AppEditorHeader's post-deploy cleanup re-derived the key from the just-typed deploy path (createApp) / the live $appPath (updateApp) instead. For a new app the autosave lives at u/{user}/draft_{uuid} while the typed path is the user's chosen name, and a rename leaves the autosave at the original key — so removing at path/$appPath missed the real draft row and orphaned it. Use the canonical userDraftPath AppEditor already provides. This is the "children re-derive the UserDraft key" fragility from the review, addressed without giving apps a page-level handle — apps deliberately lets AppEditor own the handle so the entry is destroyed on unmount (a page handle would keep it alive and reintroduce spurious autosaves on every /edit visit). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * test(drafts): integration tests for save_draft conflict semantics + authz [P2] Replaces the deleted drafts.rs (which targeted the removed /drafts/create API) with tests for the new surface: - save_draft upsert → stale-last_sync conflict (rejected, value unchanged) → force overwrite → delete, the optimistic-concurrency contract. - require_can_write_path: own namespace allowed, another user's namespace rejected, operators rejected. - the item-level extra_perms fallback — a user granted write on a deployed item can save a draft on it (regression test for the authz drop). - cross-user draft privacy: GET /drafts/get is 404 for the drawer kinds (variable/resource/triggers), not blocked for script/flow/app. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * chore(sqlx): refresh offline cache after the main merge The merge auto-combined both branches' additions inside the resource get-by-path query_as! (our draft_only/is_draft columns + main's folder_labels(...) inherited_labels), producing query text neither branch had cached — so the offline build failed for it. Regenerate the entry (rename to the new content hash) and refresh a re-described workspace query. Feature-gated/EE entries the local prepare can't compile are left as committed. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * ee repo ref * chore(system_prompts): regenerate for draft_only/is_draft trigger schema fields The openapi.yaml trigger/schedule schemas gained draft_only + is_draft, but system_prompts/generate.py wasn't rerun, failing the freshness check. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * refactor(drafts): defer save_draft write authz to RLS via a FOR UPDATE probe require_can_write_path re-implemented the item-level extra_perms write rule in Rust (SELECT extra_perms + get_perm_in_extra_perms_for_authed) — a third copy of rules whose canonical home is the RLS policies, and the exact lane that regressed once already. Replace it with an RLS write-probe: `SELECT 1 FROM {deployed_table} WHERE path/workspace ... FOR UPDATE` through UserDB. Postgres applies UPDATE policies to rows locked via FOR UPDATE, so a returned row means the canonical policies (see_own / see_member / folder-write / see_extra_perms_*_update / admin_policy) would let this user UPDATE the row — no write rule re-implemented, no drift possible. The probe's row lock is released by the immediate commit. The claim-based namespace checks stay, evaluated FIRST: they read the same JWT claims RLS does (so outcomes are identical), they spare the autosave hot path a DB round-trip for the common own-namespace case, and they are the entire check for draft-only paths — where no deployed row exists, so there is structurally nothing for RLS to evaluate. The u/own + folder-owner part now goes through the shared windmill_api_auth::require_owner_of_path instead of bespoke code. Adds a read-only-grant test case (extra_perms value false): the row is visible under the SELECT policy but FOR UPDATE filters it under the UPDATE policy — pinning the semantics the probe relies on. All 4 draft integration tests pass. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * chore: point ee-repo-ref at the EE branch merge (has DRAFT_KIND consts) ee-repo-ref was set to main's EE commit (d45b9a6) while the EE branch was unpushed; building OSS (which requires const DRAFT_KIND on TriggerCrud) against that EE ref fails with E0046 on every EE trigger impl. The EE branch head e936e9a — the merge of d45b9a6 into the EE remove-workspace-drafts branch, carrying the DRAFT_KIND consts — is now pushed; point at it. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(drafts): ignore permissioned_as fields in the unsaved-changes comparison The schedule cfg carries permissioned_as / preserve_permissioned_as — run-as deploy directives, not user-edited draft content — and the editor round-trips them asymmetrically (preserve_… is rebuilt as !!cfg.permissioned_as on load but `|| undefined` on build), so the banner comparison could report a phantom diff. Extract the normalization into a shared normalizeDraftForCompare (JSON round-trip + a DRAFT_COMPARE_IGNORED_FIELDS list with the two fields) and use it from BOTH comparators: draftValuesEqual (variable/resource banner + discardIf) and useTriggerDraftSync's cfgDiffers (schedule and trigger banners, the persist-effect's at-baseline discard, restore) — one ignore-list, no way for the two to disagree. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * nit * fix(drafts): at-baseline discard is auto-gated and only fires with a draft Two related fixes to useTriggerDraftSync's persist-effect: 1. The reactive at-baseline discard bypassed the "Enable auto-save" toggle: with autosave off, value saves were parked (correct) but the discard's value:null still POSTed — so the editor never wrote drafts yet kept reactively DELETING them, and the only network traffic was discards. Thread `auto` through UserDraft.discard to the syncer; the persist-effect passes auto:true (parked for Ctrl/Cmd+S when the toggle is off), explicit discards (banner button, post-deploy cleanup, reset-to-deployed) stay ungated. 2. The discard fired unconditionally whenever the form sat at the deployed baseline — including a spurious value:null POST on every drawer open. Guard on cfgDiffers(h.draft, deployed): undefined on a fresh open (nothing to discard) and equal to deployed right after a discard (no repeat per cfg recompute). Verified live as a non-admin user on a schedule: toggle on → no POST on open, edit → one value save, revert → one discard; toggle off → zero POSTs (everything parked), banner still functional. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * feat(drafts): scope the "Enable auto-save" toggle to the page editors Add a canBeDisabled opt (default false) to UserDraft.use / useReactive / useMany specs, threaded through acquireEntry into the reactive mirror's save opts. The syncer's auto-save gate (and the pagehide-flush skip) now only applies to saves whose handle opted in: the four full-page editors — script / flow / raw app via usePageDraftSync, app via AppEditor's use() — which are exactly the surfaces whose AutosaveIndicator carries the toggle. Drawer editors (variables / resources / schedules / triggers) keep the default and always sync regardless of the toggle — previously a toggle flipped off in some browser silently disabled their autosave and the optimistic asterisk (both sit behind the same gate) with no toggle UI anywhere on those surfaces to explain it. Verified live: schedule edit with the toggle off now POSTs the value save (and the discard on revert); script editor with the toggle off still parks everything for Ctrl/Cmd+S. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(drafts): consume the import handoff stores in the new-draft bootstrap The /add pages used to read importStore / importFlowStore / importScriptStore / sessionStorage rawAppImport to seed the editor from "Import from YAML/JSON", "Build app" (from a script/flow), and the workflows-as-code import. Since /add became a pure redirect to /{kind}/edit/u/{user}/draft_{uuid}?new_draft=true, the writers kept firing but nothing consumed the payload — every import landed in an empty editor. Consume them (one-shot read + clear) in the four edit pages' new_draft branches, layering the imported content over the empty template with path kept '' so the friendly-name generation still runs: - scripts: $importScriptStore spread over the empty script (non-empty content also keeps ScriptBuilder's template bootstrap from overwriting it — that cascade is gated on content == ''). - flows: $importFlowStore spread over the empty flow. - apps: $importStore — wrapped exports ({summary, value, policy}) and bare App values, mirroring main's /add. - raw apps: $importStore then sessionStorage rawAppImport (the full page reload for cross-origin isolation drops in-memory stores); honored only when the payload carries files (rendering gates on them), skipping the framework picker; otherwise the template seed. Verified live: "Build app" from a script lands on /apps/edit with the canvas seeded from the script instead of an empty editor. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * chore(drafts): remove dead delete_user_draft + its stale doc [C4] The doc claimed item delete handlers call it, but those all moved to delete_all_drafts_for_path (an item delete is for everyone); the caller-scoped discard goes through the save_draft route with value:null. That left delete_user_draft with zero callers (OSS and EE) — remove it and its orphaned sqlx cache entry, and reword the contrast note on delete_all_drafts_for_path. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * chore(migrations): retire the sync_drafts-era index comment + right-size it [C6] The draft_user_sync_idx comment described the deleted sync_drafts polling endpoint (editors polling created_at ranges every 2-10s) — that design was replaced by recordRemoteSync + save_draft last_sync, and nothing range-scans draft.created_at anymore. Since this migration only exists on this branch, fix it before it ships: the index's real consumer is GET /drafts/list (workspace_id + email equality, ORDER BY path), so swap the vestigial trailing created_at for path (rows come back in output order) and rename to draft_user_listing_idx. Chain re-verified on a fresh DB. (Byte-for-byte migration edit — dev DBs that already applied it need a reset, as with the earlier consolidation.) Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(drafts): discardDraft awaits the delete POST before refetching [I5] UserDraftDbSyncer.save resolves at enqueue time for debounced saves, so discardDraft's await finished ~1.5s before the value:null POST and the invalidateWorkspaceDrafts refetch re-listed the just-discarded draft. Use immediate: true (resolves after the POST lands), matching every sibling delete-then-refetch path. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(drafts): replace stale draft_only gates in the builders [I6] draft_only was dropped from the get-by-path wire shape (the column is gone; overlays carry no_deployed instead), so these four reads were always undefined: - ScriptBuilder "Exit & See details" gate and TriggersEditor's isDeployed treated every draft-only script as deployed → now keyed on savedScript.no_deployed like the sibling reads right next to them. - FlowBuilder's deploy path never took the direct-save branch for draft-only flows (no deployed version exists to compare against), and "Exit & see details" was offered for draft-only flows (404 details page) → both now keyed on the newFlow prop (driven by no_deployed), which the rest of the file already uses. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(drafts): Ctrl/Cmd+S flushes the draft in the low-code app editor [I7] The app editor's keydown handler swallowed the shortcut with a bare preventDefault() — every other page editor flushes the pending autosave (UserDraftDbSyncer.flush) so the AutosaveIndicator narrates Saving... → Saved and parked edits (autosave toggle off) actually persist. Wire the same flush, skipped in the AI session pane where no UserDraft handle exists. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(drafts): AI tool strings no longer describe drafts as localStorage [C2] The copilot tool results/messages still told the model drafts were "saved to local storage" / "a browser-only local draft" — drafts are per-user rows in the server-side draft table now. Misleading the model about the storage medium produces wrong explanations to users (e.g. "your draft will be lost if you clear your browser data"). Reword all occurrences to "draft" / "per-user draft (saved server-side)". Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(openapi): drop stale draft_only request props, fix OtherDraftUser, regen deref [D4][C5] - The create-script (NewScript), createFlow, createApp and createAppRaw request bodies still documented draft_only — the backend request structs no longer read it, so an older CLI sending draft_only: true is silently ignored and fully deploys. Remove the property from the spec so generated clients can't offer it. (Response-side draft_only on the Listable* rows stays — the list synthesis populates it.) - UserDraftOverlay.other_drafts_users item schema declared email and a required draft_saved_at; OtherDraftUser serializes only username (nullable for the legacy row — emails never leave the server). Align the schema. [C5] - Regenerate openapi-deref.yaml/.json (served at runtime via include_str!) — they still advertised getScriptByPathWithDraft and the deleted draft surface, and now carry the drafts/save_draft routes. Frontend gen client regenerated; check:fast clean. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(sessions): stop session pane from clobbering server-side raw-app drafts [P1] loadRawApp seeded the session runtime from result.value (the deployed payload), ignoring the .draft pocket returned by the get-by-path overlay. The subsequent UserDraft.save then POSTed deployed content with no last_sync recorded, silently overwriting the user's server draft. Now the no-draft branch consumes result.draft when present (matching the flow/script branches) and records draft_saved_at via recordRemoteSync so later session saves are conflict-checked instead of treated as fresh. Also corrects the header and aiDraft-branch comments that claimed the overlay merges drafts into top-level fields — it never does. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * fix(rust-client): pass new get_draft arg to variable_api::get_variable getVariable gained a GetDraft query parameter (per-user draft overlay), so the generated client fn takes a sixth argument. Verified with the same generate+check pipeline CI runs (rust-client/dev.nu --check). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * nit: Workspace fork mention * fix(drafts): don't leak other_drafts_users on draft-only private kinds [P2] fetch_draft_only built the other_drafts_users list unconditionally, while the deployed-overlay path gates it on shares_drafts_across_users. For the drawer kinds (resource/variable/triggers) drafts are private to their owner, so a draft-only GET was the one route that still told a viewer who else has a draft at the path. Apply the same kind gate. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * perf(drafts): probe a single row in the RLS write-probe [P2] The script table keeps one row per version at the same path, so the FOR UPDATE probe locked the entire version history and serialized against concurrent deploys. LIMIT 1 locks one row — any UPDATE-policy visible row proves writability (same pattern as scripts.rs's latest-version lock). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * fix(drafts): consume the /add?param= seeding intents in new_draft branches [D2] The /add routes' redirect preserves query params, but the edit pages' new_draft branches only consumed the YAML/JSON import stores — every other intent the old /add pages handled landed in a blank editor: - scripts: ?hub= and ?template= forks (with locked language and a `<source>_fork` path suggestion), ?wac=python|typescript (WAC editor template + language), ?lang=, ?initial_args= (URL form), and the base64-JSON #hash payload (run page "Fork", workspace_settings handler-template buttons; WAC detection restored for imports too) - flows: ?hub= (preprocessor placeholder replacement + env-variables panel), ?template=/?template_id=, ?fork=true (fork_flow localStorage / window.opener handoff), #state, ?tutorial= - apps: ?hub= (fromHub inputs panel), ?template=/?template_id=, ?tutorial= The redirect itself also dropped the URL hash — SvelteKit forbids url.hash in load, so it forwards window.location.hash (correct for all hash producers: they arrive as full page loads via window.open / target=_blank). Seeding priority and toasts mirror main's /add pages. Verified live: hub/template/wac/hash/fork intents for scripts and flows, hub for apps (dev hub returns empty payloads, code path confirmed via toast + inputs panel); no autosave POSTs fire during seeding. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * fix(drafts): LS→DB migration no longer clobbers fresher server drafts [P2] The one-off localStorage migration POSTed every entry with force: true, unconditionally overwriting whatever the user had since saved server-side from another browser. It now passes the LS copy's lastWrittenAt as last_sync (epoch 0 when absent), so the server's conflict rule arbitrates: empty slot → insert; server draft fresher → conflict, LS copy dropped; LS copy fresher → upload wins. Verified all three outcomes against the live save_draft endpoint. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * refactor(raw_apps): drop banned $bindable(default) on template picker open [P2] `open = $bindable(false)` on an optional prop is the AGENTS.md-banned pattern (the default masks the undefined state). The only caller always binds a boolean, so `open` is now a required prop with a plain `$bindable()`. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * fix(drafts): fork others' drafts via the import handoff, not an eager save The Fork actions (OtherUsersDraftsModal + DraftBadge popover) saved the fetched draft server-side immediately and navigated to the fork path, which surfaced three problems: a server draft existed before the user edited anything, the Path widget treated the slot as an existing item ("Only the owner can change the path"), and the value's draft_path kept the source path while the URL said X_owner_fork. Forking now routes through the same one-shot import handoff as the "Import from YAML/JSON" actions (new shared forkDraftToImport helper): stash the value in the kind's import store, navigate to /add, and let the new_draft branch seed a brand-new own item — nothing saved until the first real edit, fresh renamable path, no source identity riding along. The editPathFor/currentUserUsername plumbing that only served the old flow is removed from both fork surfaces and their callers. The new_draft branches also clear the previous path's draft-presence state (otherDraftsUsers, loadedFromDraft, stale-draft timestamps) — the page component is reused across same-route navigation, so forking from an editor with collaborators used to carry the "Others are working on this" hint onto the fresh draft. Verified live: fork of a legacy draft seeds content+summary on a fresh u/{user}/draft_{uuid} slot with zero save_draft requests and no leftover collaborator hints. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * refactor(drafts): replace deprecated Popover with meltComponents Popover - Migrate from old Popover.svelte to meltComponents/Popover.svelte - Convert to new trigger/content snippet pattern with openOnHover=true - Maintain hover behavior with debounceDelay=100 - Add key to visibleUsers each block for Svelte 5 compliance * feat(drafts): seed forked drafts with the source path in the forker's namespace Forking u/admin/myflow as guest now seeds the Path widget with u/guest/myflow instead of a random friendly name — everything after the source path's first two segments is kept, so f/folder/my/flow becomes u/guest/my/flow. The re-homed path travels from forkDraftToImport to the new_draft branches as a ?seed_path= param (the redirect preserves query params; plain ?path= would be eaten in transit by ScriptBuilder's legacy collab-param cleanup, which deletes path/collab from the live searchParams object). The script editor also passes initialPathChosen for any seeded path — MetadataGen fires onChange for a non-empty summary at mount, and the summary→path auto-slug would otherwise overwrite the explicit seed (hub/template forks and URL-hash payloads included). Verified live: forking a draft on u/admin/hard_working_script seeds path u/admin/hard_working_script (with the "path already used" warning), keeps the drafted summary/content, and still fires no save_draft until the first edit. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * fix(drafts): DiffDrawer "Restore deployed" actually discards the draft [P1] All four restoreDeployed implementations POSTed the delete through the debounced pipeline and reloaded with getDraft defaulting to true: the reload's draft write re-entered the autosave mirror (the one-shot seed guard was consumed on first load), and debouncerByKey displaced the queued value:null with the new save — the delete never reached the server and the editor re-rendered the draft it was told to discard. They now funnel through runResetToDeployed (the stopSync-bracketed delete the AutosaveIndicator reset already uses) with each page's proven reset body (getDraft: false reload), so the suspension mutes the mirror while the delete flushes and sync re-arms on first interaction. Also fixes the raw-app drawer navigating to the visual app editor (/apps/edit) instead of /apps_raw/edit [P2]. Verified live on the script editor: Restore deployed issues exactly one save_draft ({value:null} answered status=saved), the server row is gone, and the editor re-renders the deployed content. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * fix(drafts): deploying a draft-only item reliably deletes its draft Two bugs left the slot draft (u/{user}/draft_{uuid}) alive after a successful deploy: - RawAppEditorHeader.createApp removed the draft at the just-typed deploy path instead of the URL slot key (the visual header documents exactly this trap), orphaning the real row for every draft-only raw-app deploy. - Everywhere else the delete went through bare UserDraft.remove, which only QUEUES the value:null in the per-key debouncer. Editors that stay mounted through the post-deploy navigation (AppEditor, RawAppEditor — and timing-dependently the script/flow builders' post-deploy draft_triggers mirror) keep mirroring their working value, and one such write displaces the queued delete with a fresh save — observed live: deploying a new visual app re-saved the full grid value at the slot right after deploy. New discardDraftAfterDeploy helper (userDraftToast.ts) applies the same bracket runResetToDeployed uses: stopSync to mute the mirror, remove + immediate flush so the displacement window closes, re-arm on first interaction. Wired into the script/flow pages' onDeploy and both app headers' create/update paths (session-pane guards preserved). Verified live for all three kinds: draft-only deploy issues the value:null (status saved), the slot row is gone, and no post-deploy save re-creates it. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * fix(drafts): forks-compare deploy clears drawer-kind drafts too The script/flow/app deploy endpoints delete the deployer's draft server-side, but the drawer kinds' (variable / resource / schedule / triggers) create/update endpoints never touch the draft table — their editors discard client-side after a save. deployDraft replayed the save but not the discard, so "Deploy n drafts" on /forks/compare deployed those kinds correctly and left the drafts listed forever. deployDraft now issues the canonical value:null delete (immediate) for the drawer kinds after a successful save. Verified live: deploying a draft-only variable from /forks/compare creates the variable and the draft row is gone. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * fix(drafts): StaleDraftModal "Load latest deploy" actually discards the stale draft [P2] The modal invoked onLoadLatestDeploy directly — the draft = undefined write queued the delete and the reload's deployed-payload write displaced it, overwriting the stale draft with a deployed-identical copy (is_draft stuck on, asterisk persists, modal can't re-fire since draft_saved_at moved past the deploy). All four pages now run the callback through runResetToDeployed, same as the DiffDrawer restore. Verified live: stale-draft scenario → Load latest deploy → exactly one value:null POST, draft row gone, editor renders the newer deploy. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * fix(drafts): don't acquire a sync entry for empty-path specs [P2] The read-only historical-hash view (/scripts/edit/x?hash=...) computes draftPath '' but useMany still acquired a live entry at ws/script/ — every edit mirror-POSTed to /drafts/save_draft/script/ (unroutable), populating the failures map and pinning the AutosaveIndicator on "Save failed" with a retry per debounce window. Empty-path specs now get a detached local-only handle: bind: works, nothing syncs — which is what usePageDraftSync's doc always claimed. Verified live: editing in the hash view fires zero save_draft requests. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * fix(drafts): no spurious conflict after bfcache restore of a flushed page [P2] flushOnPageHide advances the server rows with unreadable keepalive POSTs and leaves lastSyncMap stale — correct when the document dies, wrong when bfcache resurrects it: the next autosave carried the pre-flush last_sync and the server rejected the user's own write as a conflict, opening DraftSyncConflictModal. The flushed keys are now remembered and dropped from lastSyncMap on pageshow with event.persisted, so the first post-restore save takes first-push semantics against this document's own flush. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * ui(drafts): draft asterisk sits on the trigger row's main title The draft hint rendered at the end of the secondary path line (u/admin/item*) on the http/websocket/nats/kafka/email trigger lists — easy to miss. It now renders at the end of the row's bold title, and on the azure/gcp lists it moves from mid-title (after the path, before the topic suffix) to the end of the line. mqtt/postgres/sqs/schedules already had it on the title. Verified visually on the HTTP routes list. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * fix(drafts): trigger editors save the FIRST edit, not the second Three interlocking fixes in the trigger autosave path: - The entry's one-shot first-write seed guard (skipNextWrite) was never consumed for trigger entries: the drawers don't write the cell on open (the form holds the state, unlike variables/resources which pass a defaultValue), so the guard stayed armed and silently swallowed the user's FIRST edit — banner on, no asterisk, no save until a second change. maybeRestore now seeds the cell with the post-load baseline (server draft overlay if any, deployed otherwise) via UserDraft.seed, consuming the guard without POSTing. - Guard hygiene in the cell's sync effect: a programmatic write consumes BOTH one-shot guards, and a no-op write (same serialization — e.g. the trigger pages fire openEdit twice per row click, re-seeding the same value) defuses a lingering seedNextWrite instead of leaving it armed to eat the next real edit. - The at-baseline auto-discard is now deferred + revalidated (600ms): with the cell seeded, the double-openEdit churn transiently shows form-at-deployed + cell-holds-draft and an immediate discard deleted the server draft on open; the recheck skips the transient state while a genuine user revert still discards. Verified live on the HTTP route editor: open-with-draft restores the draft with zero POSTs, the very first field edit saves, and reverting the form to the deployed value deletes the server draft. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * ui(drafts): underscore-separated uuids in draft slot paths u/{user}/draft_{uuid} now uses underscores instead of dashes in the uuid — path segments elsewhere in Windmill are [a-zA-Z0-9_] words and downstream consumers treat '-' as a foreign character. Nothing parses the uuid back, so existing dashed slots stay valid. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * fix(drafts): cascade draft cleanup on bulk-delete and rename Drafts have no SQL FK to their underlying items (only to password.email), so deletion and rename must cascade programmatically. Two gaps remained: - Bulk delete of variables/resources did not wipe per-user drafts at the deleted paths (single delete already did via delete_all_drafts_for_path). Cascade them — including the linked resource/variable rows the bulk delete fans into — so no orphaned draft-only rows survive. - Renaming a variable/resource/trigger left the per-user draft stranded at the old path. Add delete_own_draft_for_path and clear the deployer's own (+ legacy NULL) draft at the old path on rename, mirroring the script/flow/app rename path; teammates keep theirs (StaleDraftModal). Variable/resource renames also move the linked counterpart, so both kinds' drafts at the old path are cleared. Schedules have no rename path. Note: sqlx offline cache not yet regenerated for the new/changed queries. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * fix(drafts): surface legacy NULL-email drafts and migrate pathless /add keys Legacy workspace-scoped drafts (pre-per-user rows + the remove_draft_only migration, all email IS NULL) stopped showing up because every per-user lookup matched only email = self. Match (email = self OR email IS NULL) everywhere a draft is surfaced or opened, with the owned row taking precedence (DISTINCT ON / ORDER BY email NULLS LAST): the home drafts list, the script/flow/app/drawer draft-only list syntheses, and the get-by-path overlay/fallback. The localStorage->DB migration also dropped pathless legacy keys (userdraft/w/{ws}/{kind}/ with no path) — the new-item /add autosave — because parseKey rejected an empty path, leaving them stranded in LS. Mint a fresh u/{user}/draft_{uuid} slot for those (same convention as the editors' /add redirects) so they migrate as regular draft-only items. Note: sqlx offline cache not yet regenerated for the changed macros. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * chore(sqlx): regenerate offline cache for draft cascade + legacy-draft queries Adds the offline entries for the queries changed in the two preceding draft fixes (delete_own_draft_for_path, the maybe_overlay_draft/fetch_draft_only NULL-email fallback, and the script/flow/app draft-only syntheses). Also forwards the `http_trigger` feature from windmill-api-openapi to windmill-store: that crate imports `try_get_resource_from_db_as` unconditionally, but the fn is cfg-gated behind a trigger feature, so the openapi targets failed to compile in isolation (e.g. `--all-targets` under resolver 2) — which blocked `cargo sqlx prepare`. The feature was already present transitively in whole-workspace builds; this just makes it explicit where the symbol is used. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * fix(drafts): resolve own draft owner in the admins workspace The draft-owner surfaces (home-page badge, "others' drafts", View JSON / Fork) resolve a draft's email to a username via the `usr` table. The `admins` workspace has no `usr` rows — there a user's "username" IS their email — so the join missed every owner and returned NULL, which the badge renders as "Legacy workspace draft". A user editing a deployed item in `admins` thus saw their OWN draft plus the genuine legacy NULL-email row both labelled "Legacy workspace draft" (the reported duplicate). Add the identity fallback `COALESCE(u.username, CASE WHEN workspace_id = 'admins' THEN email END)` to the script/flow/app draft_users aggregations and fetch_other_drafts_users, and accept username==email in get_draft_for_user. The genuine legacy row keeps username NULL (its email is NULL, so the CASE yields NULL too), so it alone reads "Legacy workspace draft" while the user's own draft now reads "<email> (you)". Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * fix(drafts): keep "See others' drafts" after reset-to-deployed other_drafts_users is only computed by the backend when getDraft is true (the cross-user lookup is skipped otherwise). Reset-to-deployed reloads with getDraft:false, so the editors were overwriting the known list with the empty response — hiding the "See others' drafts" button until a full page reload recomputed it. Discarding one's own draft is independent of other users' drafts, which are untouched on the backend. Only assign otherDraftsUsers on a getDraft:true load. Applied to the script, flow, app and raw-app editors. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * disable fork for operators * Path reactivity issue * docs(drafts): tighten draft-feature comments and drop dead code The draft feature accumulated many multi-paragraph comments that risked code-comment drift. Compact them to the AGENTS.md bar (constraints not narration, state-once, no drafting-history), de-duplicating the repeated draft_users / cascade / draft_only-synthesis rationale to one canonical version per theme with terse cross-references elsewhere (~1300 fewer lines). Also fixes three stale/contradictory comments surfaced while trimming: - the operator authz note claimed operators are "excluded from every draft surface", contradicting require_can_read_path (they can read some drafts, never write) — reworded to match the code; - a migration comment named a non-existent index (draft_user_sync_idx); - a syncer comment documented the wrong map-key separator. Removes notifyDraftLoaded (orphaned exported helper, no callers). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * refactor(drafts): rename save_draft route to /update for CRUD consistency The draft write route was POST /drafts/save_draft/{kind}/{path}, which stutters with the /drafts prefix and uses a non-house verb. Rename it to POST /drafts/update/{kind}/{path} (operationId saveDraft -> updateDraft) to match the codebase's CRUD convention (/list, /get/{path}, /update/{path}). /list and /get/{kind}/{path} already matched and are unchanged. Updates the handler, openapi spec + dereferenced bundles, the two DraftService callers, the hand-built keepalive page-unload URL (it bypasses the generated client, so it wouldn't be caught by regeneration), and the integration tests. Response status values ("saved"/"conflict") are unchanged, so there is no behavior change. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> --------- Co-authored-by: windmill-internal-app[bot] <windmill-internal-app[bot]@users.noreply.github.com> Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com>	2026-06-15 10:23:16 +02:00
Ruben Fiszel	765f50c474	feat: folder-level label inheritance for scripts, flows and jobs (#9524) ... * feat: folder-level label inheritance for scripts, flows and jobs Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix: use SECURITY DEFINER folder_labels() for RLS-consistent inheritance Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * feat: extend folder label inheritance to apps, resources, variables, schedules Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> --------- Co-authored-by: Claude Fable 5 <noreply@anthropic.com>	2026-06-11 07:49:02 +00:00
hugocasa	192574ab8f	fix(forks): keep trigger/schedule operational state owned by the parent - WIN-2019 (#9476) ... * fix(forks): defer trigger/schedule state to parent for clean git merge Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(forks): read parent trigger/schedule state on non-RLS pool for complete substitution Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(forks): read schedule fork-ness on non-RLS pool; clarify mutator-rule wording Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>	2026-06-08 16:18:12 +00:00
Ruben Fiszel	5e909b2b4f	feat(git-sync): sync extra_perms for flows/scripts/apps (#9162) ... * feat(git-sync): sync extra_perms for flows/scripts/apps * fix(git-sync): hash extra_perms, fix script no-op + up-to-date checks * refactor(git-sync): route extra_perms through /acls/* instead of update endpoints * fix(git-sync): dispatch raw_app perm changes via DeployedObject::RawApp * fix(git-sync): wire applyExtraPermsDiff into pushRawApp + per-change logs * feat(git-sync): opt-in tarball perms, no-op when yaml omits, audit logs, validation * fix(git-sync): replace remaining bool literal in EE-only trigger export call * fix(git-sync): raw_app /acls/* hits app table; refetch after create * fix(git-sync): drop unnecessary post-deploy refetch (folder perms not merged) * fix(git-sync): raw_app /acls/get; reject malformed local extra_perms * audit: distinct raw_apps prefix so dashboards can split raw_app ACL events	2026-05-14 08:16:24 +00:00
Ruben Fiszel	1174d7d77f	refactor: replace SELECT * with explicit column lists (#9010) ... * refactor: replace SELECT * with explicit column lists Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * chore: update sqlx offline query cache * chore: update sqlx offline query cache * chore: update sqlx offline query cache with EE support * chore: update sqlx offline query cache, no deletions * chore: update sqlx offline query cache after rebase * fix: correct column names in explicit script query lists - concurrency_limit → concurrent_limit (matches DB column name) - runnable_settings → runnable_settings_handle (matches DB column name) Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * fix: add missing delete_after_secs column to script queries Also add integration test covering all explicit-column export queries. Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * test: add workspace export integration test covering all explicit-column queries Covers tarball_workspace (folder, script, resource, resource_type, variable, schedule, usr, group_) and the mcp_oauth_client SELECT query from windmill-mcp. Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * test: add tarball export integration test covering all explicit-column queries Single test creates one of each entity type and exercises every runtime-checked explicit-column query in tarball_workspace. Uses archive_type=tar to avoid zip feature-gate in CI. Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * docs: fix stale verification step and CI contradiction in update-sqlx skill - Regenerate current_files.txt after EE cache restoration so step 4 reports accurate diff - Scope "Never use SQLX_OFFLINE=true" to local prepare (CI legitimately uses it) Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * docs: remove Co-Authored-By from commit skill template * refactor: extract SCRIPT_COLUMNS const to single source of truth Replaces 5 duplicated 44-column lists with a shared const in windmill-types. Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>	2026-05-04 08:39:45 +00:00
hugocasa	d60dd745e4	feat(forks): handle triggers and schedules in workspace forks (#8976) ... * feat(forks): strip operational state from triggers/schedules on git-sync export When the source workspace is a fork (`wm-fork-*`), the tarball export now omits `mode` from triggers and `enabled` from schedules. The trigger update handler also preserves the existing DB `mode` when both fields are absent from the request, instead of falling back to the BaseTriggerData default. This prevents a fork's git-sync round-trip from flipping the parent workspace's enabled/disabled state when a merge applies the fork's YAML back to main. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * feat(forks): opt-in fork_triggers flag clones triggers/schedules disabled Adds `workspace.fork_triggers` (default false) and a matching field on CreateWorkspaceFork. When the user opts in, fork creation also runs clone_triggers_and_schedules: every row in schedule and the ten *_trigger tables is copied to the fork with mode='disabled' / enabled=false. Listener identifiers (group_id, replication_slot_name, subscription_name, …) are copied verbatim — the runtime suffix that prevents the fork from competing with the parent ships in a follow-up PR. native_trigger is intentionally skipped: those triggers manage external webhook state we don't want duplicated. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * feat(forks): warn before enabling triggers/schedules that conflict with parent set_trigger_mode and schedule's set_enabled now check whether the parent workspace has the same path actively enabled. If so, the call is rejected with a `fork-conflict:<kind>:<parent_id>` error unless the request includes `force=true`. The frontend interprets the prefix to surface a confirm-to- proceed dialog. This is the placeholder safety net until the Phase 3 listener-suffix work removes the conflict for the namespaceable kinds (Kafka/MQTT/NATS/Postgres/ Azure/GCP-CreateNew). For SQS, GCP-Existing, and schedules — where there's no namespacing fix — the warning is the durable solution. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * feat(forks): UI: opt-in clone-triggers checkbox + confirm-on-fork-conflict Adds the user-facing surface for the fork-trigger work: - CreateWorkspaceInner: new "Clone triggers and schedules" toggle in the fork-creation dialog (default off). Sends fork_triggers in the request. - forkConflict utility: detects the `fork-conflict:<kind>:<parent_id>` error string from the backend, shows a confirm() dialog explaining why the action is blocked, retries with `force: true` if accepted. - Wires withForkConflictRetry into every trigger setMode and the schedule setEnabled call, both in the per-kind editor components and the +page.svelte list views (HTTP, websocket, kafka, NATS, SQS, MQTT, GCP, Azure, Postgres, email, schedule). OpenAPI spec gains the `force` field on each setmode/setenabled body. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * feat(forks): CLI --fork-triggers flag, fork-trigger docs, skill update - Adds --fork-triggers boolean to wmill workspace fork; passes fork_triggers through to the create_fork API call. - New docs/fork-triggers.md describing the model end-to-end (default, opt-in clone, merge-direction filter, conflict warning, future runtime-suffix work). - Updates the adding-a-trigger SKILL.md to mention the fork-export ignore-keys participation and the clone_triggers_and_schedules block that new trigger kinds must extend. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * chore: regenerate sqlx offline query cache for fork-trigger SQL * fix(forks): replace browser confirm() with ConfirmationModal for fork conflict The fork-conflict warning previously used the browser's native confirm() which doesn't match Windmill's design system. Switches to a singleton ConfirmationModal mounted at the (logged) layout root, driven by a new forkConflictModal store. The withForkConflictRetry helper now sets the store and awaits the user's choice via a Promise, instead of blocking on window.confirm. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * fix(forks): filter unchanged triggers in merge UI, add diff view, surface parent-only ones The fork merge UI listed every trigger from the fork as a deployable item regardless of whether it differed from the parent — so a fork created with fork_triggers=true (which clones triggers in disabled state, otherwise identical) showed every trigger as a "Fork-only" change. The 'Update current' tab also missed triggers newly created in the parent that the fork hadn't pulled yet. This refactor: - fetchAllTriggers now lists both fork and parent in parallel for each trigger kind, then merges by path. - Computes a per-trigger `changeKind` (new / modified / deleted-in-source) using a JSON comparison that strips runtime + fork-local fields (mode/enabled/server_id/last_server_ping/edited_at/edited_by/etc.) so the disabled-on-clone difference doesn't show up as a change. - Filters the trigger items in deployableItems by the current direction: Deploy mode shows fork-side new/modified, Update mode shows parent-side new/modified. - Replaces the always-on "Fork-only" badge with proper New/Modified badges and surfaces a Diff button (modal Drawer + Monaco DiffEditor) for modified triggers — the diff strips the same ignored fields so users see only the meaningful config differences. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * fix(forks): always clone triggers/schedules disabled, drop opt-in flag Disabled triggers and schedules are inert — no listener attaches, no cron fires — so cloning them by default is safe by construction. Drops the fork_triggers opt-in flag introduced earlier in this PR: - Drops workspace.fork_triggers column (migration removed) - Removes fork_triggers from CreateWorkspaceFork (API + OpenAPI) - Removes the conditional in create_workspace_fork — clone always runs - Removes the toggle from the fork-creation dialog - Removes --fork-triggers from `wmill workspace fork` - Updates docs/fork-triggers.md and adding-a-trigger SKILL.md The merge UI continues to exclude triggers from the deploy/update default selection, so a routine merge from a fork doesn't accidentally push trigger config the user hasn't intentionally changed. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * fix(http-triggers): scope route exists check by workspace, skip non-workspaced clones in forks The non-CLOUD branch of `route_path_key_exists` self-excluded by trigger path alone, which silently masked cross-workspace collisions once forks started cloning trigger rows verbatim. Tighten it to exclude only the exact `(workspace_id, path)` row. Fork creation also now skips non-workspaced HTTP triggers — their URL has no workspace prefix, so a clone collides with the parent at the matchit router (which silently drops one of two duplicates) and there is no namespacing escape hatch. The clone copies all rows when CLOUD_HOSTED or HTTP_ROUTE_WORKSPACED_ROUTE forces every route workspaced regardless. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * fix(forks-ui): silent cancel on enable conflict, clean up trigger rows in compare view forkConflict helper now returns undefined when the user dismisses the modal instead of throwing, so the redundant 'Cannot enable: undefined' toast no longer appears. CompareWorkspaces trigger rows now mirror the script row layout: drop the redundant Disabled badge and the Trash/Details buttons (both belong on the dedicated trigger pages, not in the deploy/compare view); pass triggerKind through so RowIcon picks the right kind-specific icon; move extraLabel into the summary line; replace the yellow Modified badge with the same green ↗ ahead / blue ↘ behind treatment scripts use. Trigger diff drawer: switch JSON → YAML for parity with DiffDrawer, fix zero-height monaco render with className=!h-full, drop the redundant Original/Modified label banner above the diff. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * fix(email-trigger): scope local_part exists check, skip non-workspaced clones in forks Mirrors the HTTP route fix for the email-trigger non-CLOUD `email_exists` check (in EE) which had the same path-only self-exclusion bug, and the fork clone of `email_trigger` rows which copied non-workspaced `local_part` verbatim. Skip non-workspaced rows in the clone unless the instance is CLOUD_HOSTED (where lookup is workspace-scoped natively). EE companion change in windmill-trigger-email/src/handler_ee.rs. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * chore: update ee-repo-ref to 78512dd73b4a1c9f70574cff863374179e3a621b This commit updates the EE repository reference after PR #554 was merged in windmill-ee-private. Previous ee-repo-ref: 1ac77f50747b58e720a11162dfd309bc252a24ab New ee-repo-ref: 78512dd73b4a1c9f70574cff863374179e3a621b Automated by sync-ee-ref workflow. * fix(forks): always-warn on parent row, kind-specific modal copy, cancel-aware toggles - Conflict check now fires whenever the parent has the path (regardless of parent's mode), since the cloned upstream identifier is shared by construction; closes the Postgres slot-takeover gap when the parent is disabled. Schedule's set_schedule_enabled gets the same treatment. - Skip the warning entirely for HTTP and Email via a new TriggerCrud::FORK_CONFLICT_ON_ENABLE const — both kinds are workspace- scoped at runtime so cloned rows can't collide with the parent. - Modal copy branches by failure family: split-events (Kafka/NATS/MQTT/SQS/ GCP/Azure), duplicate-firing (Websocket/Schedule), slot-takeover (Postgres). Generic fallback for unknown kinds. - withForkConflictRetry now returns boolean (true=committed, false= cancelled). TriggerModeToggle reuses its existing innerTriggerMode local state via a function binding for the regular Toggle, snapping back to the prop when onToggleMode signals a cancel — needed because the native bind:checked diverges from the parent's prop after a click and Svelte's reactivity won't re-push a same-valued prop down. Schedule list page uses {#key} on a reset version since it renders Toggle directly. - Editor inners revert mode = previousMode on cancel; list pages skip the re-fetch (loadTriggers/loadSchedules) on cancel to avoid pointless network traffic and the schedule "Job stats loading..." flash. - Drop withForkConflictRetry from HTTP and Email editors + list pages since the backend never emits the conflict for those kinds. * fix(forks-ui): widen onToggleMode types, scope schedule toggle reset by path - TriggerEditorToolbar and TriggerSuspendedJobsModal forwarded onToggleMode as `(mode) => void`, dropping the new boolean return so any caller wired through them would silently no-op the cancel-revert. Match the wider TriggerModeToggle signature. - Schedule list page used a single resetVersion counter for every row's {#key}, so cancelling on any one schedule remounted every <Toggle> on the page. Switch to a per-path Record<string, number> bumped only for the affected row. * chore: bump ee-repo-ref to c3a4553 (email FORK_CONFLICT_ON_ENABLE override) * fix(forks): include Suspended in conflict gate, use parent_workspace_id for fork detection Three fixes from the Claude review on PR #8976: - Suspended mode still attaches the listener (it just pauses auto-run of queued jobs); two suspended fork+parent listeners would still split Kafka events / share a PG slot. Gate set_trigger_mode on `mode != Disabled` instead of `mode == Enabled` so Suspended also surfaces the warning. - workspaces_export.rs::fork_*_ignore_keys keyed off the wm-fork-* prefix while set_trigger_mode and set_schedule_enabled key off parent_workspace_id. Switch the export filter to query parent_workspace_id once at the top of tarball_workspace and pass is_fork through. The column is the contract; the prefix is a creation-time naming convention that could in principle drift. - TriggerModeToggle's suspend-dropdown action reassigned the non-bindable `triggerMode` prop instead of the local `innerTriggerMode` mirror, leaking inconsistent state if the dispatch was cancelled. Now writes to innerTriggerMode like the Toggle's on:change handler does. * fix(cli): skip setScheduleEnabled when local YAML lacks `enabled` Tarball export from a fork strips `enabled` from schedules so the fork→parent git-sync round-trip can't flip the parent's operational state. The CLI's pushSchedule called setScheduleEnabled whenever `localSchedule.enabled != schedule.enabled`, which evaluates truthy when local is undefined (fork-pulled YAML) and remote is true/false — sending `{ enabled: undefined }` that serializes to `{}` and gets rejected by the backend (`SetEnabled.enabled` is required). Skip the call when `localSchedule.enabled === undefined` so a sync push of fork-pulled YAMLs preserves the target's existing enabled state instead of erroring out. Trigger updates were already safe — the backend's update_trigger preserves `mode` when the request omits it. * Revert "fix(cli): skip setScheduleEnabled when local YAML lacks `enabled`" This reverts commit 23ba7e72fc. * feat(cli): --force flag and friendlier error on fork-conflict for schedule enable `wmill schedule enable foo/bar` against a fork whose parent has the same path used to surface the raw `fork-conflict:schedule:<parent>` error body. The CLI now: - accepts `--force` to bypass the warning (mirrors the API field and the UI's "Enable anyway" confirmation), - detects the `fork-conflict:` prefix on errors and prints a one-screen explanation pointing at --force instead of the raw body. Disable doesn't trigger the warning (the gate fires only on transitions to listener-attaching modes), so no flag there. Trigger enable/disable isn't exposed as a standalone CLI command — sync push goes through updateTrigger which has its own backend mode-preservation, so no fork-conflict surfaces from the CLI for those. * chore: regenerate cli-commands docs after adding --force to schedule enable * chore: update ee-repo-ref to 967f961f0a88b027d894aebd03977181129477a8 This commit updates the EE repository reference after PR #555 was merged in windmill-ee-private. Previous ee-repo-ref: c3a4553296473932e15392a06415dd7fb9aa6591 New ee-repo-ref: 967f961f0a88b027d894aebd03977181129477a8 Automated by sync-ee-ref workflow. * fix(forks): address CI dead-code, claude/cubic review feedback - backend: cfg-gate `fork_trigger_ignore_keys` to match its already-gated callsite. CI compiles with `-D warnings`, so the unused-fn under feature combos that disable all trigger crates was breaking check_oss/check_ee/ cargo_test/test-linux/test-windows. - cli: re-apply the `pushSchedule` undefined-skip (originally 23ba7e7, reverted in 4d172a1). Tarball export from forks strips `enabled`, so fork-pulled YAMLs that get sync-pushed back via `wmill schedule push` would otherwise serialize `{ enabled: undefined }` → `{}` and the backend's required `SetEnabled.enabled` rejects the body. Skipping preserves the target's existing flag, which is the round-trip-safe behavior. (`wmill workspace merge` extension to triggers/schedules is tracked in #9001 — until then sync push is the only CLI path.) - TriggerModeToggle suspend-dropdown action awaits onToggleMode and resets `innerTriggerMode = triggerMode` on cancel, matching the Toggle on:change handler. Without this, dismissing the fork-conflict modal on a Suspend transition leaves the toggle stuck in 'suspended'. - forkConflict: when a new modal opens with a previous resolver still pending, resolve the older promise to false. Avoids a dangling promise if the user clicks toggles on two rows in quick succession. - schedules list: bump `toggleResetVersions[path]` on the permission-denied branch so the Toggle re-mounts back to the prop's `enabled` value. Without this, a user without write permission could click the toggle and have it stick visually flipped. - docs/fork-triggers.md: switch the merge-direction filter description from `wm-fork-*` prefix to `parent_workspace_id IS NOT NULL` (matches the code after 4dd38fe). Drop the misleading "merge-direction filter strips identifier columns too" line in Future Work — the runtime suffix is applied at listener attach, the stored column never carries it, so no export filtering is needed there. --------- Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com> Co-authored-by: windmill-internal-app[bot] <windmill-internal-app[bot]@users.noreply.github.com>	2026-05-01 20:55:24 +00:00
Ruben Fiszel	4483d0cab9	fix(workspaces): split get_settings into admin-only + public endpoint (#8990) ... * fix: redact GitHub App tokens and Slack OAuth secret for non-admins `GET /workspaces/get_settings` returned the full `git_app_installations` JSONB to any workspace member. That column caches the GitHub App JWT and installation token used by git-sync; the installation token is refreshed on every git-sync action and valid for ~55 minutes, so the value sitting in the DB is essentially always live. Null it out for non-admins, matching the existing `slack_oauth_client_secret` redaction. The tarball export's v2 settings format (added in #8935) included `slack_oauth_client_secret` with no admin gating, regressing the same redaction. Mirror the admin check there. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * refactor: split get_settings into admin-only + public endpoint Adds `WorkspacePublicSettings` and `GET /workspaces/get_public_settings`, which returns only fields safe for any workspace member to read (workspace_id, slack/teams team identity, mute_critical_alerts, deploy_ui, large_file_storage, datatable). `get_settings` is now admin-only via `require_admin`. Migrates frontend callers: every caller that read non-sensitive fields (deploy_ui on trigger pages, mute_critical_alerts on the root layout, slack team identity for handler pickers, etc.) now uses `getPublicSettings`. The admin-managed settings UI, git-sync admin context, operator settings, checkout polling, and full settings page stay on `getSettings`. This replaces the field-level redactions added in the previous commit: the type system itself defines the public surface, so adding a sensitive column to `workspace_settings` no longer defaults to leaking — it stays out of `WorkspacePublicSettings` unless explicitly added. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>	2026-04-30 14:02:54 +00:00
Alexander Petric	95d4c6a94d	feat(cli): non-interactive Slack connect/disconnect + sync round-trip fixes (#8935) ... * feat(cli): non-interactive Slack connect/disconnect Extract create_slack_workspace_artifacts / create_slack_instance_artifacts from the browser OAuth callbacks and expose them via two new endpoints that accept a pre-minted xoxb bot token: - POST /w/{workspace}/workspaces/connect_slack (admin) - POST /oauth/connect_slack_instance (super-admin) Both produce bit-for-bit identical DB state to the UI browser flow. Wire three CLI commands as thin wrappers: - wmill workspace connect-slack - wmill workspace disconnect-slack - wmill instance connect-slack Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * fix(cli): round-trip stability for workspace settings handlers wmill sync push was destroying UI-configured error_handler/success_handler state on every deploy. Two orthogonal bugs: (a) pushWorkspaceSettings called editErrorHandler with `path: undefined` when the YAML lacked the handler block, which the backend treats as a clear — so syncing settings.yaml that didn't mention the handler wiped the DB row. Fix: skip the call entirely when absent from YAML. (b) edit_error_handler omitted muted_on_cancel / muted_on_user_path when false, but the CLI always sends them, causing perpetual deepEqual drift and a spurious editErrorHandler call on every sync push. Fix: always persist both booleans. migrateToGroupedFormat now preserves explicit `null` on error_handler / success_handler as a "clear remote" signal distinct from absence. Widen ErrorHandlerConfig | null / SuccessHandlerConfig | null to make this explicit in the type. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * feat(cli): sync support for workspace-level Slack OAuth override Add slack_oauth_client_id and slack_oauth_client_secret to the v2 tarball export and to pushWorkspaceSettings, so the workspace-level OAuth override is now fully managed as code through settings.yaml. Semantics: - both defined and truthy → setWorkspaceSlackOauthConfig (upsert) - both defined but falsy (e.g. empty strings) and remote has a value → deleteWorkspaceSlackOauthConfig - either omitted → leave remote alone ("not managed by git") Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * refactor(cli): normalize workspace settings sync to "omit = clear" Earlier commits on this branch introduced an "omit = keep" rule for error_handler / success_handler / slack_oauth_client_{id,secret} that diverged from every other workspace setting (webhook, deploy_to, etc. all treat YAML as canonical: absence = clear). Normalize: - v2 tarball always emits these 4 fields (null when remote is NULL) so round-trip is bijective and settings.yaml is a complete snapshot. - pushWorkspaceSettings drops the absent-from-YAML guards; YAML is canonical. Absence and explicit null both clear the remote — same rule as every other field. - set_slack_oauth_config / delete_slack_oauth_config now fire handle_deployment_metadata so UI mutations reach git-sync-enabled workspaces' committed settings.yaml. Policy for users: pull before push (same as every other setting). On first post-upgrade pull, explicit `null` keys appear for any workspace whose handlers / oauth override are unset — one-time YAML diff, no semantic change. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * test(cli): add unit + integration coverage for Slack settings sync Unit tests (settings_unit.test.ts): cover migrateToGroupedFormat preserving explicit `null` on error_handler / success_handler, and passthrough of slack_oauth_client_id / _secret (both populated and null values). Integration tests (slack_settings_sync.test.ts, skipped on CI per the same convention as datatable_settings_sync.test.ts): exercise the full backend via withTestBackend to verify 1. pull emits null for unset error_handler / success_handler / slack_oauth_client_id / _secret; 2. round-trip with all-null handlers is idempotent; 3. push of populated slack_oauth_config upserts; 4. omitting the slack_oauth keys from YAML clears remote (universal "omit = clear" rule); 5. explicit null error_handler in YAML clears remote; 6. round-trip preserves a populated error_handler exactly, including the always-persisted muted_on_cancel / muted_on_user_path booleans. Also feature-gates `use crate::oauth2_oss::workspace_connect_slack` and its route registration behind `cfg(feature = "oauth2")`: the import caused a build failure on subsets of the workspace without the oauth2 feature, surfaced by the integration test harness. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * chore: bump ee-repo-ref to 59b6123 Pins windmill-ee-private to the tip of branch alp/slack_cli, which contains the companion EE changes (helper extraction, non-interactive Slack connect handlers, git-sync for Slack settings mutations). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * Update SQLx metadata * chore: regenerate system prompts for new slack CLI commands Captures the new workspace connect-slack, workspace disconnect-slack, and instance connect-slack commands in the auto-generated files that CI enforces via system_prompts/check-freshness.sh. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * chore: update ee-repo-ref to b4a5ca11e3b96ff03793c2bd396dbc1fe6ea1022 This commit updates the EE repository reference after PR #550 was merged in windmill-ee-private. Previous ee-repo-ref: d7e44d0519327ec9077625130365e887826f324b New ee-repo-ref: b4a5ca11e3b96ff03793c2bd396dbc1fe6ea1022 Automated by sync-ee-ref workflow. --------- Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com> Co-authored-by: windmill-internal-app[bot] <windmill-internal-app[bot]@users.noreply.github.com> Co-authored-by: Ruben Fiszel <ruben@windmill.dev>	2026-04-24 17:14:08 +00:00
hugocasa	d6c642b170	feat: add Azure Event Grid triggers (#8888) ... * feat: add Azure Event Grid triggers (EE) Introduces a new enterprise trigger kind `azure` that supports three modes via a single unified trigger type: - basic_push: Azure Event Grid basic — custom topics, system topics (Storage, Resource Manager, Key Vault, etc.), domains (push only) - namespace_push: Event Grid Namespace topics (CloudEvents over HTTP push) - namespace_pull: Event Grid Namespace topics (HTTP pull with lock-token ack/reject for dead-lettering) Auth uses a Service Principal resource (tenant_id, client_id, client_secret, subscription_id). Subscriptions are created in CloudEvents 1.0 schema so the push webhook handler and the pull listener share one payload parser. Backend - New crate `windmill-trigger-azure` (OSS stubs + EE impl symlinked from windmill-ee-private) - Migration `azure_trigger` table with CHECK constraints enforcing mode/columns coherence - `TriggerKind::Azure`, `JobTriggerKind::Azure`, `DeployedObject::AzureTrigger` variants - Push route `/api/azure/w/{workspace}/*path` handles classic Event Grid SubscriptionValidation handshake and CloudEvents 1.0 abuse-protection OPTIONS handshake - Optional inbound JWT validation (audience check only for v1) - Feature flag `azure_trigger` propagated through windmill-api, windmill-store (resource helper), and added to ee_core Frontend - `triggers/azure/` editor with mode toggle (basic/namespace-push/ namespace-pull) and per-mode config (topic ARM id / namespace + topic name / subscription / filters / push auth / pull options) - Registered in icon map, display names, save functions, badge, wrapper, editor, add-trigger menu OpenAPI - `AzureTrigger`, `AzureTriggerData`, `AzureMode`, `AzureSubscriptionMode`, `AzureDeliveryConfig`, `TestAzureConnection` schemas; `/azure_triggers/*` endpoints; client regenerated Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * chore: update ee-repo-ref to eaa7c3a9cb37a9ccc93f10a2535d929365acd2d8 This commit updates the EE repository reference after PR #541 was merged in windmill-ee-private. Previous ee-repo-ref: 9689014e8c12c36c1059fd8fa5758d550b8b8bc9 New ee-repo-ref: eaa7c3a9cb37a9ccc93f10a2535d929365acd2d8 Automated by sync-ee-ref workflow. * feat(azure-trigger): secret-auth push, ARM discovery, capture isolation, CLI + parity Frontend: - Split mode selector into Namespace/Basic + Pull/Push - ARM resource dropdowns (namespaces, Basic topics, namespace topics) populated from the service principal; cascade with stale-selection reset on SP / edition change - Remove stale authenticate toggle + audience input (server-managed push_auth_config has replaced them) - Azure listing page: "Create from template" button; "Also delete Azure subscription" toggle in the delete modal; simplified trigger label falling back to path - AzureCapture.svelte: "Test subscription name" with -wm-capture suffix - CompareWorkspaces.svelte: wire Azure for fork/compare - Drop Trigger-deployed/event-loss warning (capture subscription is isolated with -wm-capture) Backend: - Shared-secret push auth (see EE crate for detail) - JSONB push_auth_config column (renamed from delivery_config), #[serde(skip)] so clients/CLI/exports never see it - Drop redundant enabled column; mode supersedes - Azure capture infra: AzureTriggerConfig + set_azure_trigger_config + azure_payload route + TriggerKind::Azure arm; PT15M queue TTL on capture subscriptions so they bound storage after tab close - Granular ACLs, users offboarding, trash, git-sync deployed-object: all include azure_trigger CLI: - Add azure to TRIGGER_TYPES, pushObj dispatch, getTypeStrFromPath, trigger commands (get/update/create/list/template), sync delete switch + regex; e2e test for `trigger new --kind azure` - system_prompts: SCHEMA_MAPPINGS + schema_names include AzureTrigger; auto-generated/* regenerated Skill: - .claude/skills/adding-a-trigger/ checklist covering every file that needs editing when wiring a new trigger type (learned from this PR) ee-repo-ref bumped to b0e490cbf3724b7b64c6a5b010e3bdf24acd873c. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * fix(azure-trigger): ci — ShareModal Kind + regenerated system_prompts - frontend/src/lib/components/ShareModal.svelte: add 'azure_trigger' to the Kind type so the listing page's "Permissions" action compiles (ts2345 — caught by npm_check on CI, missed by fast-check locally). - system_prompts/auto-generated/: regenerate to drop the stale delivery_config / AzureDeliveryConfig fields from the Azure schema (check-freshness on CI). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * refactor(azure-trigger): use workspace constant_time_eq crate Drop hand-rolled constant-time compare in favour of the workspace constant_time_eq crate (same one used by http_trigger_auth). ee-repo-ref bumped to 9659382d47286e7f7f66d01b6f5dd8d4ed34848b. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * fix(azure-trigger): pass placeholder + disabled via inputProps `TextInput`'s `placeholder` and `disabled` go through its `inputProps` prop — CI's `npm run check` caught the stale top-level passing that `npm run check:fast` missed. Align with the DefaultEmailConfigSection pattern. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * fix(azure-trigger): correct LATEST_GIT_SYNC_SCRIPT_PATH version to 28213 The hub deploy of the azure-aware sync-script is version 28213, not 28214. Backend was pinning a non-existent hub script, which broke the git_sync_e2e suite (every deploy's sync step 404'd). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * fix(azure-trigger): add azure_triggers to token scope selector + skill - windmill-api/src/token.rs: `build_trigger_scope_domains` was missing `("azure_triggers", "Azure Event Grid")`, so the CreateToken UI's scope selector didn't surface azure_triggers:read/write. Backend already had `ScopeDomain::AzureTriggers` wired (scopes.rs), this just exposes it. - .claude/skills/adding-a-trigger/SKILL.md: capture both scope-related files under the hardcoded-arrays section so future triggers don't miss the UI surface. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * docs(adding-a-trigger-skill): clarify token.rs scope effect Not a regression — nothing was working before. Skipping TRIGGER_DOMAINS just means the scope works via API/CLI but has no UI checkbox. * docs(adding-a-trigger-skill): trim token.rs bullet * fix(azure-trigger): regen openapi-deref + swap textarea for TextInput - Run build_openapi.sh to regenerate openapi-deref.{yaml,json} with the 12 azure_triggers paths + schemas. These files are served by the runtime (include_str! in windmill-api/src/lib.rs) to external SDK consumers; without this regen the new endpoints wouldn't be advertised. - Replace the raw <textarea> for event type filters with the design-system TextInput in textarea mode (frontend/CLAUDE.md bans raw HTML elements). Addresses cubic + claude PR review items. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com> Co-authored-by: windmill-internal-app[bot] <windmill-internal-app[bot]@users.noreply.github.com>	2026-04-23 16:30:18 +00:00
Ruben Fiszel	bbb564c142	fix: omit default_permissioned_as from tarball export when empty ... Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>	2026-04-22 19:31:37 +00:00
Ruben Fiszel	2d18a68099	feat: add scheduled job deletion with configurable retention period (#8753) ... * feat: add scheduled job deletion with configurable retention period Extends delete_after_use with delete_after_secs to enable configurable retention periods for job args/result/logs. At completion, jobs can be scheduled for future deletion via a new job_delete_schedule table, processed by a monitor task. Supports per-script, per-flow, and per-flow-step configuration. Backward compatible. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * feat: add integration tests, revert query! macros, fix review issues - Add integration tests for resolve_delete_after_secs, schedule_job_deletion, flow-level and module-level delete_after_secs, backward compat - Revert sqlx::query() back to sqlx::query!() macros for compile-time safety - Regenerate sqlx offline cache - Fix FlowModule/NewScript/FlowValue constructions in all test files - Fix autoscaling_ee.rs for updated script_path_to_payload return type Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * chore: update ee-repo-ref.txt for autoscaling_ee fix Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: gate cleanup_scheduled_job_deletions behind enterprise feature Prevents dead_code warning (which CI treats as error via -D warnings) when compiling without enterprise feature. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * chore: regenerate sqlx cache after merge with main Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: address review feedback on scheduled deletion - Monitor: roll back transaction on any cleanup error so schedule rows survive for retry on next cycle (instead of best-effort then discard) - Migration: add FK with ON DELETE CASCADE to job_delete_schedule.job_id to prevent orphan rows when jobs are deleted through other means - Simplify bool-to-Option conversion with .then_some(true) Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * refactor: stop setting delete_after_use alongside delete_after_secs No mixed-version deployment scenario exists, so delete_after_secs alone is sufficient. The backend's resolve_delete_after_secs handles (None, Some(secs)) correctly without needing delete_after_use set. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * refactor: remove delete_after_use from public API surface Remove delete_after_use from OpenAPI spec, API client, runtime client, and workspace export. Only delete_after_secs is exposed going forward. The field remains in Rust backend types with #[serde(skip_serializing)] for backward-compatible deserialization of existing scripts/flows that were saved with delete_after_use: true. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * chore: update ee-repo-ref to 1d4b7a31fc115d6aba8640f7cd3fd5a01abe6806 This commit updates the EE repository reference after PR #519 was merged in windmill-ee-private. Previous ee-repo-ref: 9eba09a13b778caafc6ae65098b90e53c91984d3 New ee-repo-ref: 1d4b7a31fc115d6aba8640f7cd3fd5a01abe6806 Automated by sync-ee-ref workflow. * fix: regenerate system prompts, remove unused import - Regenerate auto-generated system prompts after openflow schema change - Remove unused serde_json::json import in test file (CI -D warnings) Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: insert dummy v2_job row in schedule tests for FK constraint The job_delete_schedule table has a FK to v2_job, so tests need a real v2_job row before inserting into the schedule table. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * chore: trigger CI re-run * fix: remove heavy flow integration tests to avoid CI worker contention The flow integration tests spawn workers that compete for CPU with the existing relock_skip tests under --test-threads=10, causing consistent 60s timeouts in CI. Keep only the lightweight unit tests and DB integration tests. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: restore correct ee-repo-ref for our branch The ref was overwritten to main's EE ref during a rebase. Restore to our branch's EE commit that includes the autoscaling tuple fix. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * chore: retrigger CI on fresh runner * fix: remove FK constraint from job_delete_schedule to unblock CI The FK with ON DELETE CASCADE to v2_job may have caused performance overhead during test DB setup (each sqlx::test creates a fresh DB with all migrations). Remove the FK — orphan schedule rows are harmlessly cleaned by the monitor. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * ee-ref --------- Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com> Co-authored-by: windmill-internal-app[bot] <windmill-internal-app[bot]@users.noreply.github.com>	2026-04-08 04:15:28 +00:00
Ruben Fiszel	c4c9ef5fd7	feat: add optional labels to scripts, flows, apps, schedules, triggers (#8609) ... * feat: add optional labels to scripts, flows, apps, raw apps, schedules, and triggers Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: update sqlx cache, make labels optional in openapi, regenerate system prompts Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * feat: add minimal labels input UI to script, flow, and schedule editors Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: reduce gap between summary and labels input Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * feat: add labels to script/flow detail pages and summary/path popover Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: move labels inside SummaryPathDisplay trigger for clickable area, reduce gap Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: display labels inline to the right of summary, not below Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: increase gap between summary and labels Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * feat: add labels to resources/variables, make labels nullable, add home page label filter badges Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * feat: add labels to workspace export/import, resources, variables + test coverage Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: make migration idempotent, regenerate sqlx cache after merge Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: pass labels in script create and flow create/update API calls Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * feat: add labels input UI to resource and variable editors Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: remove negative margin from LabelsInput to prevent overlap Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: add top and left margin to LabelsInput for better spacing Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: reduce left margin on LabelsInput Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: widen label input to w-32 Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: use inline-flex so LabelsInput doesn't stretch full width Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: remove flex-wrap so label input stays on same line as badges Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * feat: add label filter presets to resources, variables, and schedules search Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: use max-w-32 on label input to prevent stretching Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: pull labels closer to summary with negative top margin Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: increase negative margin to pull labels even closer to summary Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: pass labels in schedule create/update API calls Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: use COALESCE to preserve existing labels when not provided in schedule/flow update Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: add labels to CreateResource, EditResource, CreateVariable, EditVariable in OpenAPI spec Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * feat: display label badges on resource and variable list pages Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * feat: display label badges on schedule and all trigger list pages Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * feat: add folder and label presets to schedules search filter Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: apply user_folders_only filter on all workspaces including admins Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * feat: add label presets to resources and variables search filters Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: derive folder presets from loaded items, not all workspace folders Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: add label query parameter to resource and variable list endpoints in OpenAPI Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * feat: display label filter badges inline with folder filters on home page Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * Revert "feat: display label filter badges inline with folder filters on home page" This reverts commit 6767a50aa6. * feat: support comma-separated label filters (allowMultiple) in all list endpoints Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: append label presets with comma for allowMultiple filters instead of duplicating key Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: hide label presets that are already in the comma-separated filter value Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: replace unsafe manual SQL ARRAY construction with parameterized queries, add labels to ScriptWDraft Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: complete down migration, add labels to Resource/Variable OpenAPI schemas, remove type cast, add label length validation Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: add labels field to Schedule test fixture Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: add labels field to Rust client struct constructions Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: regenerate sqlx cache with --all-features for EE builds Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * chore: regenerate sqlx cache and package-lock after merge with main Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: squash two migrations into one, use IF NOT EXISTS for idempotency Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: track label changes in SummaryPathDisplay to enable save button Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: use JSON string comparison for label dirty tracking in popover Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: navigate to script by path after save from popover to load new version Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: update initialLabels after save so subsequent label changes enable save again Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: use onchange callback for label dirty tracking instead of derived comparison Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: reload script by path after label save to fetch new version Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * feat: propagate script/flow labels to jobs at push time Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * feat: show script/flow labels on runs page, merge with wm_labels for completed jobs Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: change job labels type from JSONB to text[], show labels on job detail page, fix type mismatch Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: add labels to QueuedJob struct, fix get_job queries to return v2_job.labels Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: replace +Label text with icon only Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * feat: add tag icon before labels on job detail page Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: move tag icon inside badge on job detail page Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: use blue badge with tag icon in RunBadges, remove duplicate labels from JobDetailHeader Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: set icon position to left so tag icon renders in badge Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: render Tag icon inline in badge children instead of via icon prop Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: retry icon prop with small badge and position left Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * feat: add hover tooltip showing "Label: X" on job label badges Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * feat: include v2_job.labels in runs page label filter and broad search Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * chore: regenerate sqlx cache and system prompts after merge with main Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: add labels to EE JobPayload constructions, regenerate sqlx cache with --all-features Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: regenerate sqlx cache CE-only (without EE symlinks that cause conflicts) Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: update remaining wm_labels JSONB queries to use text[] merge expression Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: simplify job labels to just read v2_job.labels (wm_labels already merged at completion) Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: consistent label badge spacing with gap-0.5 wrapper and px-0.5 on badges Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: add labels: None to test utils JobPayload construction Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: add labels to all test fixture JobPayload/NewFlow/EditApp constructions, regenerate sqlx cache Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: fix vertical content shift by fixing container and input height to h-5 Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: npm_check errors - unused imports, combinedItems order, flow.labels type, badge px-1 padding Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: remove unused FolderService imports, fix label badge alignment in RunBadges Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: restore deleted service imports in variables page, remove empty loadFolders Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * chore: trigger CI with updated ee-repo-ref * chore: update ee-repo-ref to merged EE companion PR Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * chore: trigger fresh CI run for updated ee-repo-ref * fix: match label badge size with other badges in RunBadges using {large} prop Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: remove icon from RunBadges label badge to fix vertical alignment Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: shorten "Job kind" to "Kind" in run badges Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: add small inline tag icon (10px, -mt-px) to label badge without disrupting height Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: add "Label: X" hover tooltip to all label badges, show hidden labels on +N hover Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * feat: add tag icon and "Label: X" tooltip to home page label filter badges Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: show LabelsInput even when path is hidden in ResourceEditor Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * feat: add labels input to new resource creation drawer (AppConnectInner) Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * iterate * fix: add LabelsInput to all resource creation steps in AppConnectInner Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: reduce LabelsInput top margin from -mt-3 to -mt-1 Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: increase negative margin to -mt-2 for tighter spacing Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: split the difference with -mt-1.5 Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: adjust to -mt-1 for label spacing Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: per-site label spacing via class prop instead of global negative margin Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * feat: make label badges clickable to toggle label filter on resources, variables, schedules Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: use proper array indexOf for label filter toggle, set undefined correctly on removal Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: use delete instead of undefined to properly clear label filter Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * feat: add /labels/list endpoint and autocomplete dropdown to LabelsInput Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: use inline preventDefault for Svelte 5 event handling Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * feat: add "Create new" option in label autocomplete, regenerate sqlx cache with update_sqlx.sh Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * feat: add GIN indexes on labels column for all 16 tables Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: remove CONCURRENTLY from GIN index creation in migration Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * test: add comprehensive label coverage for pull, edit, removal across all item types Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: simplify job label filters to only use v2_job.labels, remove wm_labels back-compat Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * test: add integration tests for job label propagation, display, and filtering Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: address PR review findings — missing labels in fetch_script_for_update, app rename, escape key bug - Add `labels` to SELECT in `fetch_script_for_update` to prevent lost labels on script clone - Pass `labels` in app branch of `moveRenameManager.ts` so app renames preserve labels - Clear `inputValue` before `adding = false` in LabelsInput escape handler to prevent accidental label add via onblur - Fix `test_job_label_filter` to complete jobs via SQL (label filtering only works on completed jobs) - Add `test_wm_labels_from_result_merged_with_static_labels` integration test using Bun Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>	2026-04-03 17:39:32 +00:00
Ruben Fiszel	a46aa641f9	feat: add R language support (#8263) ... * feat: add R language support Add R as a new supported scripting language in Windmill, following the same pattern used for Ruby. Includes: - Backend: ScriptLang::Rlang enum variant, DB migration, tree-sitter-r parser crate with tests, WASM parser binding, R executor with NSJail sandboxing, job dispatch and signature parsing - Frontend: language picker, R icon, syntax highlighting, editor bar insertions (Sys.getenv, get_variable, get_resource), schema inference, init code template, BETA badge - CLI: .r extension mapping, sync support, bootstrap template R scripts use `main <- function(...)` syntax, jsonlite for JSON serialization, and system curl for the Windmill client helper. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * feat: add R package resolution and installation Parse library()/require() calls from R scripts to extract dependencies. Resolve versions from CRAN, cache lockfiles in pip_resolution_cache, and install packages to a shared R library cache. The run step sets R_LIBS_USER so installed packages are available to the script. - Parser: parse_r_requirements() extracts package names from AST - Executor: resolve() generates lockfile, install() installs from CRAN - Worker lockfiles: wire up R resolve for dependency jobs Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * feat: add nsjail sandboxing for R resolve and install phases Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * fix: fix R get_variable/get_resource and add sandbox annotation + e2e tests Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * fix: fix R arg inference with JS fallback parser and get_variable/get_resource Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * fix flake * nsjail * nits * fix: R install improvements - suppress verbose output, flat lockfile logging, Dockerfile R support, rlimits - Suppress renv verbose output during resolve and install (controlled by #verbose annotation) - Filter renv from install list (already loaded, causes noisy restart message) - Log compact "resolved N packages" instead of full renv.lock JSON - Add R (r-base, r-cran-renv) to DockerfileFull and DockerfileFullEe - Use disable_rl for nsjail install config (R compiles from source) - Reduce default concurrency from 20 to 5 - Add rlang to openflow.openapi.yaml - Fix MainArgSignature (no_main_func -> auto_kind) after main merge Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * final * fix: remove accidental R install from multiplayer Dockerfile Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * fix: remove R from Windows build and DockerfileExtra Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * fix: rename R migration to avoid timestamp collision with trigger_filter_logic Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * all * fix: R install improvements - suppress verbose output, flat lockfile logging, Dockerfile R support, rlimits Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * fix: add clear error when Rscript binary is missing Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * fix: fix type errors in R fallback parser, use format! in wrap(), add R system prompts Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com> Co-authored-by: pyranota <pyra@duck.com>	2026-04-01 06:11:37 +00:00
Ruben Fiszel	0317668089	fix: require admin for workspace encryption key export (#8523) ... Move the require_admin check from blocking the entire tarball export to only guarding the include_key=true path. Non-admins can still export tarballs for workspace sync/git, but only admins can export the raw workspace encryption key. Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>	2026-03-25 14:33:20 +00:00
hugocasa	efb4a27d51	fix: replace email with permissioned_as for triggers/schedules (#8439) ... * refactor: replace email with permissioned_as for triggers/schedules Add a new `permissioned_as` column (format: `u/{username}`, `g/{group}`, or raw email) to all trigger tables and schedule. This value is used directly for job permission checks, removing the need for email lookups when creating/updating triggers. - Migration: add permissioned_as to all 9 trigger tables + schedule, drop email from trigger tables (schedule keeps it for backwards compat) - Backend: resolve_email() (async, DB) -> resolve_permissioned_as() (sync) - Email cache: get_email_from_permissioned_as() with quick_cache for places that still need email (fetch_api_authed, schedule backwards compat) - Frontend: rename email/preserve_email -> permissioned_as/preserve_permissioned_as in deploy data and OpenAPI schemas - Tests updated for new field names and u/{username} format Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix sqlx/build * update ee ref * refactor: simplify resolve_edited_by to always use authed username Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix compile + migration * update ref * test: add trigger trait method tests for permissioned_as queries Add tests that call TriggerCrud and Listener trait methods directly to verify dynamic SQL correctly references the permissioned_as column. Covers get_trigger_by_path, list_triggers, set_trigger_mode, and fetch_enabled_unlistened_triggers for all trigger types. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * update sqlx * fix: use permissioned_as directly for schedules and fix audit RLS for groups - Schedule: permissioned_as only set on create, not on edit/set_enabled - Schedule: stop reading email column, use get_email_from_permissioned_as - Triggers: use fetch_api_authed_from_permissioned_as instead of edited_by - Triggers: rename listener fields for clarity (username -> edited_by) - Fix audit author username for group permissioned_as (g/test -> group-test) to match session.user, preventing RLS policy violations on audit_partitioned - OpenAPI: remove permissioned_as/preserve_permissioned_as from EditSchedule - Add backwards-compat comments for schedule email writes Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * chore: regenerate system prompts for permissioned_as field Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix build * refactor: generalize onBehalfOf naming, add permissioned_as to EditSchedule - Frontend: rename onBehalfOfPermissionedAs -> onBehalfOf with comments explaining it carries emails for flows/scripts and permissioned_as for triggers/schedules - Frontend: rename getOnBehalfOfEmail -> getOnBehalfOf, getOnBehalfOfPermissionedAsForDeploy -> getOnBehalfOfForDeploy, customOnBehalfOfEmails -> customOnBehalfOf - Backend: add optional permissioned_as/preserve_permissioned_as to EditSchedule with COALESCE (only updates when provided) - Backend: add on_behalf_of audit log for schedule edit - Backend: remove unused resolve_on_behalf_of_permissioned_as - Tests: remove email assertions from schedule update test (email is just backwards compat, only permissioned_as matters) Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: preserve email column when permissioned_as is preserved on schedule edit Derive email from the preserved permissioned_as via cache lookup instead of always writing authed.email. This keeps the email column consistent with the old behavior for backwards compat with old workers. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: update deploy UI labels from "edited by" to "run as" for triggers Triggers now use permissioned_as (not edited_by) for permissions, so update the deploy UI wording to reflect this. Also update wm_deployers group description to mention schedules and permissioned_as. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: use u/username format for custom trigger/schedule deploy selection When picking a custom user for trigger/schedule deployment, store u/${username} (permissioned_as format) instead of the email. Flows/scripts continue to use email format for on_behalf_of_email. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: show u/username format for "me" option in trigger deploy selector Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * refactor: simplify OnBehalfOfSelector to return the right format per kind OnBehalfOfSelector now handles the email vs permissioned_as format internally based on kind: - triggers: returns u/username, displays u/username in all options - flows/scripts/apps: returns email, displays username The onSelect callback now takes (choice, value?) where value is already in the correct format. Parent components just store it directly without needing to know about the format difference. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: always show u/username format in OnBehalfOfSelector for all kinds Display is now consistent: all kinds show u/username in the selector. The returned value still differs (email for flows/scripts, u/username for triggers) since the backend APIs expect different formats. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: replace email with permissioned_as in http_trigger test insert The email column was dropped from trigger tables in the migration. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: review fixes — migration, app policy, capture cleanup, naming - Migration: remove DEFAULT '', use nullable → populate → SET NOT NULL - App policy: set both on_behalf_of and on_behalf_of_email for all choices - OnBehalfOfSelector: return OnBehalfOfDetails {email, permissionedAs} instead of ambiguous value - Remove unused email field from Capture struct and query - Rename getSourceEmail/getTargetEmail → getSourceOnBehalfOf/getTargetOnBehalfOf - Rename test functions from preserve_email to preserve_permissioned_as Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: add permissioned_as to all test schedule INSERTs Since the migration no longer uses DEFAULT '', all INSERTs must explicitly provide permissioned_as. Updated test fixtures and schedule_push tests. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: strip permissioned_as from exports/sync, fix OpenAPI required field - Add permissioned_as to workspace export strip list (like edited_by) - Add permissioned_as to CLI TriggerFile Omit list - Fix TriggerExtraProperty.required: email → permissioned_as - Regenerate frontend and CLI types Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: remove accidentally committed generated files These directories are gitignored and should not be tracked. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * chore: regenerate system prompts for permissioned_as schema changes Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: remove permissioned_as from CLI TriggerFile Omit list Already stripped in workspace export, no need to also omit from the type. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: optimize email cache key and revert TriggerFile Omit change - Use single concatenated string for cache key instead of (String, String) tuple - Remove permissioned_as from CLI TriggerFile Omit (already stripped in export) Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: zero-allocation email cache lookups using Equivalent trait Use a borrowed EmailCacheKey(&str, &str) for cache lookups via quick_cache's Equivalent support. Only allocates (String, String) on cache miss for insert. This is called on every trigger fire and schedule push. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: add permissioned_as to Schedule required fields in OpenAPI spec The backend always returns permissioned_as (non-optional String), so the schema should reflect that. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: handle group- prefix in migration UPDATE statements edited_by can be 'group-{name}' for group-owned triggers/schedules. The migration now correctly maps these to 'g/{name}' format instead of incorrectly producing 'u/group-{name}'. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * Revert "fix: handle group- prefix in migration UPDATE statements" This reverts commit 0971392b38. * fix: use superadmin email to resolve permissioned_as in schedule migration For users upgrading from older versions where edited_by may not reflect the actual schedule owner, check if the email belongs to a superadmin and look up their username. Otherwise fall back to edited_by. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: fall back to superadmin email when not in workspace usr table If the superadmin isn't a member of the workspace, use their email as raw permissioned_as instead of falling back to edited_by. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: always update permissioned_as and email on schedule edit Consistent with pre-refactor behavior where email and edited_by were always updated on every edit. permissioned_as is now always set (to editing user or preserved value), removing the COALESCE that previously preserved it when not provided. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * feat: add schedule permission tests and centralize group prefix constants Tests: schedule create/update for normal user, workspace admin, and superadmin not in workspace. Verifies schedule fields (email, permissioned_as, edited_by) and pushed job fields (permissioned_as, permissioned_as_email). Constants: centralize "u/", "g/", "group-" as PERMISSIONED_AS_USER_PREFIX, PERMISSIONED_AS_GROUP_PREFIX, USERNAME_GROUP_PREFIX. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: use @unknown.windmill.dev for synthetic email fallback Prevents privilege escalation: a user with username like 'superadmin_secret' would get superadmin via the synthetic email matching SUPERADMIN_SECRET_EMAIL. Using a different subdomain avoids any collision with hardcoded @windmill.dev emails. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * update ee ref * sqlx * chore: regenerate system prompts after main merge Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * chore: update ee-repo-ref to bda51bc33bcb573659e7ff07d0a23ff6e23b8148 This commit updates the EE repository reference after PR #468 was merged in windmill-ee-private. Previous ee-repo-ref: 8cf1802f8fe183f430830590b4f3172a50207843 New ee-repo-ref: bda51bc33bcb573659e7ff07d0a23ff6e23b8148 Automated by sync-ee-ref workflow. --------- Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com> Co-authored-by: windmill-internal-app[bot] <windmill-internal-app[bot]@users.noreply.github.com>	2026-03-20 16:28:38 +00:00
Ruben Fiszel	31d6660d56	feat: script module mode with CLI sync, preview, and WAC UI improvements (#8380) ... * feat: add script module mode with folder model for Bun and Python Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * fix: add missing modules field to RawCode in bun_executor Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * sqlx * feat: enrich WAC templates with checkpoint and replay semantics Add prominent comments explaining that all computation must happen inside task/step/taskScript or it will be replayed on resume/retry. Clarify that waitForApproval does not hold a worker and that approve/reject URLs are available in the timeline step details. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * fix(cli): script module sync idempotency, per-module hash tracking, and preview support - Fix pull→push idempotency: use `??` instead of `||` for module lock field so empty strings are preserved (matches API's `lock: ""`) - Add per-module hash tracking in wmill-lock.yaml following the flow inline script pattern (SCRIPT_TOP_HASH + per-module subpath hashes) - Selective module lock regeneration: only regenerate locks for modules whose content actually changed, not all modules - Use unfiltered rawWorkspaceDependencies for module hashes to match what updateModuleLocks passes to fetchScriptLock - Show changed module names in stale script output for clarity - Add module support to `script preview` command: read modules from __mod/ folder and pass them in the preview API request - Add preview tests for taskScript pattern (flat and folder layout) - Update test assertion for module stale detection output Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * feat(frontend): WAC UI improvements — reorder templates, module tab rename, import consolidation - Reorder WAC template buttons: TypeScript before Python in ScriptBuilder, CreateActionsScript, and CreateActionsFlow - Remove dropdown items from +Script button (simplify to direct link) - Move "Import Workflow-as-Code" to +Flow dropdown with dedicated drawer - Add module tab rename: pencil icon on hover opens popover with validation, fixed-width icon container prevents layout shift Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * feat: remaining module-mode changes from working branch - Backend parser updates for WAC detection - CLI sync/types updates for raw app path and module support - Frontend UI polish (Dev.svelte, ScriptRow, script hash page) - Test fixture updates Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * test(cli): add test for module modification detection in generate-metadata Verifies that modifying a single module file re-triggers stale detection and only the changed module is listed, not all modules. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix(backend): critical fixes from PR review - Fix hardcoded dev path in bun_executor.rs WAC v2 wrapper — use "windmill-client" import instead of absolute filesystem path - Fix missed no_main_func → auto_kind rename in parser TS test - Add modules column to clone_script SQL (windmill-common and windmill-api-workspaces) so cloned scripts retain their modules - Add modules: None to RawCode structs in worker tests - Restore complete sqlx cache (merge main's cache + our new queries) Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix(backend): fix clone warning treated as error in CI Change `.clone()` on double reference to `*k` dereference in scripts.rs hash implementation. Update sqlx cache with new query hashes from modified clone_script SQL. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix(frontend): use published parser wasm versions for CI build The local file:// paths for windmill-parser-wasm-py and windmill-parser-wasm-ts don't exist in the Cloudflare Pages build environment. Revert to published npm versions (1.655.0). Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix(frontend): update parser wasm packages to 1.657.2 Use newly published windmill-parser-wasm-ts and windmill-parser-wasm-py v1.657.2 which include auto_kind/WAC detection changes. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix(frontend): regenerate package-lock.json for npm ci compatibility Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix(frontend): use main's lockfile as base, update only parser wasm packages Regenerating package-lock.json from scratch pulled different dependency versions causing svelte-check type errors. Instead, start from main's lockfile and only update the two changed packages. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix(backend): add modules column to fetch_script_for_update query The Script<SR> struct has a modules field (FromRow), but fetch_script_for_update didn't SELECT modules, causing a runtime error "no column found for name: modules" when the worker processed dependency jobs. This was the root cause of the relock_skip test timeout. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix(backend): fix script module execution for Python and Bun - Fix modules not passed through job queue: inject _MODULES into PushArgs.extra when pushing Code jobs so worker can extract them - Fix Python module imports: use relative imports (from .helper) and add sys.path.insert for module directory in wrapper - Fix Python tests: use relative imports and empty lock to prevent pip from resolving module names as packages - Add local file check in Bun loader for module resolution - Ignore Bun module test (bundle mode loader integration tracked separately) - Add missing modules column to fetch_script_for_update query Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix(backend): remove unnecessary empty lock in Python module tests Relative imports (from .helper) are not parsed as pip packages, so the empty lock workaround is not needed. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix(backend): fix module execution for Python and Bun — all tests pass Python modules: - Use relative imports (from .helper import greet) since scripts run as packages - Add sys.path.insert for module directory in wrapper to ensure local modules take precedence over pip packages with same name Bun modules: - Use bundled output (./out/main.js) as wrapper import when modules are present — the bundled output has module content inlined by Bun.build, avoiding runtime loader resolution issues - Add local file check in loader.bun.js onResolve to short-circuit API URL resolution for module files on disk Job queue: - Inject _MODULES into PushArgs.extra when pushing Code jobs so the worker can extract them at execution time Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * refactor: address PR review — simplify, fix correctness, remove dead code Critical fixes: - Replace all CLI `no_main_func` references with `auto_kind` (string) to match the backend migration and API changes - Remove duplicated `compute_python_module_dir` in worker.rs, use the canonical version from python_executor.rs High priority: - Auto-create `__init__.py` in intermediate directories for nested Python modules so imports like `from .utils.math import add` work without users manually creating __init__.py files - Remove redundant `sys_path_insert` — relative imports use Python's package system, not sys.path Medium: - Fix lock file base name extraction: use regex to strip only the final extension (`.replace(/\.[^.]+$/, '')`) instead of `indexOf(".")` which breaks for files like `helper.test.ts` Simplification: - Remove dead `{#if false}` Popover block in ScriptEditor.svelte - Guard loader.bun.js local file check to only run for relative paths (matching the Windows loader pattern) - Add clarifying comment on Bun dual mechanism (build + run phases) - Add maintenance comment on manual Hash impl for NewScript Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: final review fixes — stale cleanup, baseName, auto_kind export - Fix sync.ts baseName extraction using indexOf(".") → regex (same fix as script.ts/metadata.ts, missed this instance) - Add stale module file cleanup in writeModulesToDisk: removes files from __mod/ that are no longer in the modules map before writing, fixing the pull→push cycle that couldn't delete modules - Log warning when _MODULES serialization fails in job push instead of silently dropping modules - Use strict equality (===) for auto_kind comparison - Exclude auto_kind from workspace export — it is auto-detected by the parser at deploy time from script content Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix(cli): remove auto_kind from push, comparison, and metadata auto_kind is auto-detected by the parser at deploy time, so the CLI should not send it, compare it, or write it to script.yaml. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: remove erroneously added backend/backend/.sqlx directory Duplicate .sqlx cache was committed at the wrong nested path. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: address PR review feedback + fix CI dead_code warning Frontend (ScriptEditor.svelte): - Fix switchToMain() missing lastSyncedCode update — prevents stale code sync on external changes while editing a module tab - Fix formatAction saving module code to main script's localStorage draft — now saves main code when on a module tab - Fix non-null assertion on inferModuleLang in renameModule — fall back to original language instead of force unwrap - Remove redundant activeModuleTab truthy check in runTest CLI (script.ts): - Clean up empty directories after removing stale module files in writeModulesToDisk Backend: - Add path traversal guard in write_module_files — reject module paths containing ".." - Fix dead_code warning on auto_kind field in workspace export struct Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * feat(frontend): improve auto_kind UX + address review findings - Rename "Include without main function" toggle to "Include library scripts" in script list (ItemsList.svelte) - Update NoMainFuncBadge: "No main" → "Library" with clearer tooltip - Filter module file extensions by main script language — Python scripts only allow .py modules, TypeScript only .ts, etc. - Split flushModuleState into flushModuleContent (no UI side-effect) and flushModuleState (flush + reset tab), reducing duplication - Dynamic placeholder and hint text in add module popover based on main script language Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>	2026-03-17 01:20:09 +00:00
Ruben Fiszel	f2be625348	feat: store hashed tokens instead of plaintext (#8217) ... * feat: store hashed tokens in the token table instead of plaintext Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * fix: address review issues in token hash migration - Update all base.sql fixtures to include token_hash/token_prefix columns - Keep plaintext token for webhook tokens (needed for URL reconstruction) - Restore get_token_by_prefix to query DB for webhook tokens - Fix down migration to delete NULL-token rows before restoring NOT NULL - Update parser fixture standalone schema - Update EE dedicated_worker_ee.rs to use token_hash/token_prefix Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * fix: restore sqlx offline cache (only add new query files) Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * refactor: keep writing plaintext token column for backward compat Write to token column alongside token_hash until MIN_VERSION_SUPPORTS_TOKEN_HASH (1.649.0) is reached. This ensures older workers can still authenticate during rolling upgrades. Remove the separate UPDATE in new_webhook_token since create_token_internal now writes plaintext directly. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * feat: branch on MIN_VERSION to write plaintext token or null Check MIN_VERSION_SUPPORTS_TOKEN_HASH at runtime: write plaintext to token column while old workers exist, switch to NULL once all workers are >= 1.649.0. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * fix: set MIN_VERSION_SUPPORTS_TOKEN_HASH to 1.650.0 Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * fix: use token_hash for email lookup and expiry notifications Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * refactor: rotate webhook tokens instead of recovering plaintext from DB Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * refactor: use token_hash for native trigger token lookups and deletes Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * sqlx * refactor: drop webhook_token_prefix from native_trigger table Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * fix: backward compat for token rotation and make webhook_token_hash NOT NULL Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * fix: prevent panic on short superadmin secret token prefix Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * fix: prevent panic on short superadmin secret token prefix Replace all `token[0..TOKEN_PREFIX_LEN]` slicing with `token.get(..TOKEN_PREFIX_LEN).unwrap_or(token)` to prevent panics when a token shorter than 10 chars is provided (e.g. malformed Authorization header, short superadmin secret). Co-authored-by: hugocasa <hugocasa@users.noreply.github.com> * fix: prevent panic on short token prefix slicing Replace all `token[0..TOKEN_PREFIX_LEN]` with safe `token.get(..TOKEN_PREFIX_LEN).unwrap_or(token)` to prevent panics on malformed tokens shorter than 10 characters. Co-authored-by: hugocasa <hugocasa@users.noreply.github.com> Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * Revert "fix: prevent panic on short superadmin secret token prefix" This reverts commit 37ec2e5ad5. * revert: remove unnecessary defensive token prefix slicing Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * fix: add token_hash to end_user_email test fixture Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * test: add integration tests for token hash migration Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * fix: correct token_hash test assertions for cache and version Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * chore: add plaintext column removal reminder to test fixtures Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * fix: log count of orphaned triggers deleted during migration Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * fix: preserve orphaned triggers with error instead of deleting Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * fix: rename token_expiry_notification.token to token_hash and copy owner/expiration in rotate Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * fix: hash existing plaintext values before renaming token_expiry_notification column Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * fix: remove unnecessary length check in token_expiry_notification migration Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * update dates and version * updat ee ref + sqlx * improve mcp migration * fix: atomic token rotation with rollback on trigger update failure rotate_webhook_token now atomically inserts the new token and deletes the old one in a single transaction, preventing token leaks. Returns new_token_hash so callers can clean up the new token if their subsequent trigger update fails (which involves external HTTP calls and cannot be in the same DB transaction). - Handler: wraps post-rotation work; deletes new token on failure - Google renewal: deletes new token if service_config update fails - Tests updated to match new atomic semantics Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * higher min version * fix: defer old token deletion to avoid breaking triggers on update failure rotate_webhook_token now keeps the old token alive and returns old_token_hash. Callers delete it only after the trigger row has been successfully updated. If the external service call or DB update fails, the trigger keeps working with the old token. Worst case: if the best-effort delete fails, the old token leaks as an extra DB row — harmless compared to breaking the trigger. Also update summarized_schema.txt for renamed columns. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * chore: update ee-repo-ref to 2d0823a471014e2bc2d898c63518323946b7474f This commit updates the EE repository reference after PR #437 was merged in windmill-ee-private. Previous ee-repo-ref: 7aef8b06cb6f54c2bc89dd57b70947deed72553c New ee-repo-ref: 2d0823a471014e2bc2d898c63518323946b7474f Automated by sync-ee-ref workflow. * fix: prevent panic on short tokens by using safe prefix extraction Add safe_token_prefix() helper that uses .get(..TOKEN_PREFIX_LEN).unwrap_or(token) instead of direct slice indexing, preventing panics when tokens are shorter than 10 characters (e.g., short superadmin secrets or malformed Bearer tokens). Co-authored-by: Ruben Fiszel <rubenfiszel@users.noreply.github.com> Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com> Co-authored-by: HugoCasa <hugo@casademont.ch> Co-authored-by: claude[bot] <41898282+claude[bot]@users.noreply.github.com> Co-authored-by: hugocasa <hugocasa@users.noreply.github.com> Co-authored-by: windmill-internal-app[bot] <windmill-internal-app[bot]@users.noreply.github.com> Co-authored-by: Ruben Fiszel <rubenfiszel@users.noreply.github.com>	2026-03-17 01:15:38 +00:00
Alexander Petric	5df37fb0db	feat: add datatable config support to CLI settings sync and backend export (#8024) ... Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>	2026-03-14 04:44:11 +00:00
Alexander Petric	39e77ecd00	feat: add slack connection fields to workspace settings export/import (#8287) ... Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>	2026-03-09 21:39:03 +00:00
Ruben Fiszel	424ca59dfe	feat: make WINDMILL_DIR configurable via environment variable (#8215) ... * fix: auto-heal corrupted python runtime cache on remote workers Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * Revert "fix: auto-heal corrupted python runtime cache on remote workers" This reverts commit 0ea013a554. * feat: make WINDMILL_DIR configurable via environment variable Allow users to configure the base directory for Windmill's tmp/cache files via the WINDMILL_DIR env var (default: /tmp/windmill). This fixes Python runtime cache corruption on RHEL systems where systemd-tmpfiles-clean removes files from /tmp. Converts TMP_DIR (renamed to WINDMILL_DIR) and all derived cache directory constants from compile-time const &str (concatcp!) to runtime lazy_static String values. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * chore: update ee ref Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * chore: update ee ref Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * fix: deref ERROR_DIR lazy_static for AsRef<Path> and Display traits Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * chore: update ee ref to branch name for CI compatibility Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * fix: deref lazy_static constants in all executor files Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * chore: update ee ref Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * chore: update ee ref Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * chore: update ee ref Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * fix: panic if WINDMILL_DIR has trailing slash Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * fix: also reject trailing backslash in WINDMILL_DIR for Windows Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * fix: deref GO_BIN_CACHE_DIR in test utils Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * fix: replace remaining hardcoded /tmp/windmill paths and validate empty WINDMILL_DIR Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * fix: nsjail powershell mount dst, Windows path assumptions, pwsh deref consistency Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * fix: restore Windows /tmp path translation in go and bun executors The Windows path translation replaces /tmp with the Windows temp dir (e.g. C:\tmp) before normalizing slashes. Without this, the default WINDMILL_DIR=/tmp/windmill produces paths without a drive letter on Windows. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * chore: update ee-repo-ref to 6fd5a2ce908235a17975ad4dbdf0051cd89334f3 This commit updates the EE repository reference after PR #436 was merged in windmill-ee-private. Previous ee-repo-ref: e8c03e16720833230ebd1878b4c63642ecc6c80f New ee-repo-ref: 6fd5a2ce908235a17975ad4dbdf0051cd89334f3 Automated by sync-ee-ref workflow. --------- Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com> Co-authored-by: windmill-internal-app[bot] <windmill-internal-app[bot]@users.noreply.github.com>	2026-03-04 08:53:25 +00:00
Ruben Fiszel	20bb97fc1f	reorg mcp_tools	2026-02-11 06:44:10 +00:00
Ruben Fiszel	ccc4806b9a	refactor: extract windmill-types crate from windmill-common (#7851) ... Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>	2026-02-08 14:15:07 +00:00
Ruben Fiszel	9ff8a85af6	refactor: extract windmill-api into subcrates for parallel compilation (#7845) ... * refactor: extract windmill-api into 4 subcrates (api-auth, store, api-sse, api-jobs) Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * refactor: eliminate refresh_token OnceLock bridge in windmill-store Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * refactor: eliminate FromRequestParts OnceLock bridge in windmill-api-auth Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * refactor: wire subcrates into workspace and clean up unused re-exports Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * fix: resolve cargo check --all-features errors in subcrate wiring Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * sqlx * all * chore: update ee-repo-ref for warning fixes Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * refactor: extract windmill-trigger crate and expand windmill-api-jobs Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * refactor: extract windmill-trigger-kafka crate from windmill-api Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * refactor: extract windmill-trigger-postgres crate from windmill-api Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * refactor: extract windmill-trigger-websocket and windmill-trigger-mqtt crates Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * refactor: extract windmill-trigger-nats, sqs, gcp, and email crates Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * refactor: extract windmill-trigger-http crate from windmill-api Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * refactor: move token creation and permission helpers to windmill-api-auth Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * refactor: extract windmill-native-triggers crate from windmill-api Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * sqlx * all * refactor: extract windmill-api-embeddings crate and fix CI warnings Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * fix: resolve type mismatch in oauth2_oss and remaining warnings Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * fix: use correct HTTP_CLIENT config in embeddings crate (30s timeout, cert override) Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * all * fix: gate oauth_refresh_ee on oauth2 feature to fix warnings Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * all --------- Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>	2026-02-07 22:12:55 +00:00
hugocasa	456dd478d8	fix: 404 triggers listing in CE (#7705)	2026-01-27 17:30:36 +00:00
dieriba	6418c4bcc6	feat: nextcloud native triggers (#6797)	2026-01-26 16:49:52 +00:00
claude[bot]	05fa3cd013	feat: add workspace setting to disable error handler for u/ scripts/flows (#7634) ... * feat: add workspace setting to disable error handler for u/ scripts/flows Add a new workspace setting `error_handler_muted_on_user_path` that allows disabling the error handler for jobs related to scripts/flows that start with the "u/" prefix (user scripts/flows). Changes: - Add migration to add `error_handler_muted_on_user_path` column to workspace_settings - Update WorkspaceSettings struct and get_settings query - Update EditErrorHandler struct and edit_error_handler API endpoint - Update error handler cache to include the new setting - Add skip logic in send_error_to_workspace_handler for u/ paths - Add toggle in workspace settings UI The implementation uses the existing cached settings to avoid additional database queries, making it efficient. Closes #7633 Co-authored-by: windmill-internal-app[bot] <windmill-internal-app[bot]@users.noreply.github.com> * all * update ee private * combine migrations * sqlx * fix: migration fixes for auto_invite JSONB column - Fix TEXT[] to JSONB conversion using to_jsonb() - Add GIN index on auto_invite for efficient ? operator queries - Add index cleanup to down migration Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * sqlx * all * all * all --------- Co-authored-by: claude[bot] <41898282+claude[bot]@users.noreply.github.com> Co-authored-by: windmill-internal-app[bot] <windmill-internal-app[bot]@users.noreply.github.com> Co-authored-by: Ruben Fiszel <ruben@windmill.dev> Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>	2026-01-24 19:26:06 +00:00
Pyra	fe56191422	feat(internal): runnable settings (#7298) ... * rework everything again Signed-off-by: pyranota <pyra@duck.com> * updcate sqlx Signed-off-by: pyranota <pyra@duck.com> * update ref Signed-off-by: pyranota <pyra@duck.com> * fix things Signed-off-by: pyranota <pyra@duck.com> * fix function Signed-off-by: pyranota <pyra@duck.com> * final fixes Signed-off-by: pyranota <pyra@duck.com> * update sqlx Signed-off-by: pyranota <pyra@duck.com> * fix script creation Signed-off-by: pyranota <pyra@duck.com> * address todo Signed-off-by: pyranota <pyra@duck.com> * cleanup Signed-off-by: pyranota <pyra@duck.com> * remove dbg Signed-off-by: pyranota <pyra@duck.com> * cleanup Signed-off-by: pyranota <pyra@duck.com> * fix Signed-off-by: pyranota <pyra@duck.com> * fixups Signed-off-by: pyranota <pyra@duck.com> * fix cargo.toml Signed-off-by: pyranota <pyra@duck.com> * update ee repo Signed-off-by: pyranota <pyra@duck.com> * fix ci Signed-off-by: pyranota <pyra@duck.com> * nit Signed-off-by: pyranota <pyra@duck.com> * ref Signed-off-by: pyranota <pyra@duck.com> * fix Signed-off-by: pyranota <pyra@duck.com> * ee repo Signed-off-by: pyranota <pyra@duck.com> * sqlx Signed-off-by: pyranota <pyra@duck.com> * ee ref Signed-off-by: pyranota <pyra@duck.com> * remove dbg Signed-off-by: pyranota <pyra@duck.com> * sqlx Signed-off-by: pyranota <pyra@duck.com> * chore: update ee-repo-ref to 505eadbff32d102ea5245a2bef88ce6f1bb95395 This commit updates the EE repository reference after PR #348 was merged in windmill-ee-private. Previous ee-repo-ref: 195243e56cc0eab55f8890fa57297206bfe2c18c New ee-repo-ref: 505eadbff32d102ea5245a2bef88ce6f1bb95395 Automated by sync-ee-ref workflow. * ci: force runnable settings Signed-off-by: pyranota <pyra@duck.com> --------- Signed-off-by: pyranota <pyra@duck.com> Co-authored-by: windmill-internal-app[bot] <windmill-internal-app[bot]@users.noreply.github.com>	2025-12-16 21:17:06 +00:00
Ruben Fiszel	6f5489c7dd	raw apps v2 (#7251)	2025-11-29 01:23:47 +00:00
Pyra	d38c96db36	feat: workspace dependencies (#7124) ... * commit raw requirements Signed-off-by: pyranota <pyra@duck.com> * raw requirements Signed-off-by: pyranota <pyra@duck.com> * implement `parse_annotation` Signed-off-by: pyranota <pyra@duck.com> * more progress on wdeps Signed-off-by: pyranota <pyra@duck.com> * more progress Signed-off-by: pyranota <pyra@duck.com> * fixes Signed-off-by: pyranota <pyra@duck.com> * more progress Signed-off-by: pyranota <pyra@duck.com> * fixes Signed-off-by: pyranota <pyra@duck.com> * cli improvements + raw deps Signed-off-by: pyranota <pyra@duck.com> * cleanup Signed-off-by: pyranota <pyra@duck.com> * fix python versions Signed-off-by: pyranota <pyra@duck.com> * progress Signed-off-by: pyranota <pyra@duck.com> * update :) Signed-off-by: pyranota <pyra@duck.com> * add MaybeLock Signed-off-by: pyranota <pyra@duck.com> * go WIP Signed-off-by: pyranota <pyra@duck.com> * fix python ignoring py version from requirements Signed-off-by: pyranota <pyra@duck.com> * optimize php Signed-off-by: pyranota <pyra@duck.com> * require admin to alter Signed-off-by: pyranota <pyra@duck.com> * fix(cli): flow generateLocks raw deps Signed-off-by: pyranota <pyra@duck.com> * progress in checklist Signed-off-by: pyranota <pyra@duck.com> * fix agent workers Signed-off-by: pyranota <pyra@duck.com> * nits Signed-off-by: pyranota <pyra@duck.com> * nits Signed-off-by: pyranota <pyra@duck.com> * nit: remove default features Signed-off-by: pyranota <pyra@duck.com> * oh-wow Signed-off-by: pyranota <pyra@duck.com> * remove dbg! Signed-off-by: pyranota <pyra@duck.com> * nits Signed-off-by: pyranota <pyra@duck.com> * add indexes Signed-off-by: pyranota <pyra@duck.com> * cleanup Signed-off-by: pyranota <pyra@duck.com> * nits Signed-off-by: pyranota <pyra@duck.com> * remove todos Signed-off-by: pyranota <pyra@duck.com> * fix cli Signed-off-by: pyranota <pyra@duck.com> * add debug flag Signed-off-by: pyranota <pyra@duck.com> * cli: remove noise Signed-off-by: pyranota <pyra@duck.com> * fix cli Signed-off-by: pyranota <pyra@duck.com> * remove todos Signed-off-by: pyranota <pyra@duck.com> * trigger deps correctly Signed-off-by: pyranota <pyra@duck.com> * fix frontend Signed-off-by: pyranota <pyra@duck.com> * fix frontend again Signed-off-by: pyranota <pyra@duck.com> * finally fix frontend Signed-off-by: pyranota <pyra@duck.com> * ee repo ref Signed-off-by: pyranota <pyra@duck.com> * fix all Signed-off-by: pyranota <pyra@duck.com> * more fixes... Signed-off-by: pyranota <pyra@duck.com> * remove test Signed-off-by: pyranota <pyra@duck.com> * Update backend-test.yml * comment out legacy test Signed-off-by: pyranota <pyra@duck.com> * fix ci Signed-off-by: pyranota <pyra@duck.com> * fix ci? Signed-off-by: pyranota <pyra@duck.com> * comment out thing Signed-off-by: pyranota <pyra@duck.com> * ignore test Signed-off-by: pyranota <pyra@duck.com> * ci Signed-off-by: pyranota <pyra@duck.com> * base fixture Signed-off-by: pyranota <pyra@duck.com> * fix regression Signed-off-by: pyranota <pyra@duck.com> * fix docs links Signed-off-by: pyranota <pyra@duck.com> * update min version Signed-off-by: pyranota <pyra@duck.com> * simplify * implement cache for get_latest Signed-off-by: pyranota <pyra@duck.com> * move to workspace settings Signed-off-by: pyranota <pyra@duck.com> * sqlx + migration Signed-off-by: pyranota <pyra@duck.com> * more migrations Signed-off-by: pyranota <pyra@duck.com> * use box pin Signed-off-by: pyranota <pyra@duck.com> * nit Signed-off-by: pyranota <pyra@duck.com> --------- Signed-off-by: pyranota <pyra@duck.com> Co-authored-by: Ruben Fiszel <ruben@windmill.dev>	2025-11-28 16:00:11 +00:00
Pyra	edece035f8	feat: job debouncing (#6878) ... * feat(EE): job debouncing Signed-off-by: pyranota <pyra@duck.com> * remove 'script' file Signed-off-by: pyranota <pyra@duck.com> * more work Signed-off-by: pyranota <pyra@duck.com> * properly gate it behind enterprise Signed-off-by: pyranota <pyra@duck.com> * update ee repo ref Signed-off-by: pyranota <pyra@duck.com> * change ee repo ref again Signed-off-by: pyranota <pyra@duck.com> * remove unused variable Signed-off-by: pyranota <pyra@duck.com> * feat(EE): implement TODOs and enhance tracing for job debouncing - Add database index on script(workspace_id, debounce_key) for efficient lookups - Update minimum version requirement to 1.564.0 throughout codebase - Add tracing warnings when debouncing is disabled due to worker version mismatch - Fix all documentation links from TODO placeholders to proper URLs - Replace Gauge icon with Timer icon for debouncing UI elements - Update placeholder text and tooltips with clear descriptions Co-authored-by: Pyra <pyranota@users.noreply.github.com> * create -> crate Signed-off-by: pyranota <pyra@duck.com> * remove index Signed-off-by: pyranota <pyra@duck.com> * some updates Signed-off-by: pyranota <pyra@duck.com> * fix once more Signed-off-by: pyranota <pyra@duck.com> * fix it once more Signed-off-by: pyranota <pyra@duck.com> * Remove flow step debouncing, keep top-level flow debouncing - Remove debounce fields from RawScript and FlowScript FlowModuleValue variants - Remove debounce fields from JobPayload::FlowScript and RawCode - Update raw_script_to_payload function signature - Remove debouncing UI from flow step runtime settings - Remove debouncing toggle handler and indicator badge - Preserve top-level flow debouncing in FlowSettings Co-authored-by: Pyra <pyranota@users.noreply.github.com> * cleanup Signed-off-by: pyranota <pyra@duck.com> * fixup claude's work Signed-off-by: pyranota <pyra@duck.com> * cleanup: remove dbg! statements, update min version to 1.566.0, add comprehensive comments - Removed all dbg! macro calls from production code - Updated MIN_VERSION_SUPPORTS_DEBOUNCING from 1.564.0 to 1.566.0 - Added comprehensive documentation comments explaining: - Debouncing feature purpose and mechanics - Database schema for debounce_key and debounce_stale_data tables - Version check logic and guard functions - Improved code clarity and maintainability Co-authored-by: Pyra <pyranota@users.noreply.github.com> * improve fallback Signed-off-by: pyranota <pyra@duck.com> * remove comments from old migration Signed-off-by: pyranota <pyra@duck.com> * fix pull Signed-off-by: pyranota <pyra@duck.com> * fix once more Signed-off-by: pyranota <pyra@duck.com> * Update frontend/src/lib/components/ScriptBuilder.svelte Co-authored-by: claude[bot] <209825114+claude[bot]@users.noreply.github.com> * add sqlx cache Signed-off-by: pyranota <pyra@duck.com> * fix: frontend CI - fix debounce field names and remove leftover flow step debouncing - Fix ScriptBuilder.svelte: change custom_debounce_key to debounce_key - Add debounce_key and debounce_delay_s fields to NewScript schema in openapi.yaml - Regenerate frontend types from OpenAPI spec - Remove leftover flow step debouncing code from FlowModuleComponent.svelte - Remove debounce fields from RawScript in openflow.openapi.yaml - Remove unused Timer import from FlowModuleHeader.svelte All frontend checks now passing (0 errors, 0 warnings) Co-authored-by: Pyra <pyranota@users.noreply.github.com> * fix ci Signed-off-by: pyranota <pyra@duck.com> * remove unused import Signed-off-by: pyranota <pyra@duck.com> * fix ci again Signed-off-by: pyranota <pyra@duck.com> * udpate ee repo ref Signed-off-by: pyranota <pyra@duck.com> * CI doesn't want to be fixed but I still try Signed-off-by: pyranota <pyra@duck.com> * nits Signed-off-by: pyranota <pyra@duck.com> * ci... Signed-off-by: pyranota <pyra@duck.com> * Update ee-repo-ref.txt * safer migration Signed-off-by: pyranota <pyra@duck.com> * reduce noise in logs Signed-off-by: pyranota <pyra@duck.com> * fix cli for scripts Signed-off-by: pyranota <pyra@duck.com> * nit Signed-off-by: pyranota <pyra@duck.com> --------- Signed-off-by: pyranota <pyra@duck.com> Co-authored-by: claude[bot] <41898282+claude[bot]@users.noreply.github.com> Co-authored-by: Pyra <pyranota@users.noreply.github.com> Co-authored-by: claude[bot] <209825114+claude[bot]@users.noreply.github.com> Co-authored-by: windmill-internal-app[bot] <windmill-internal-app[bot]@users.noreply.github.com>	2025-10-23 20:18:08 +00:00
Alexander Petric	e92a99a101	don't include drafts in workspace exports tarball (#6710) ... * don't include drafts in workspace exports tarball * sqlx * ee repo ref --------- Co-authored-by: Ruben Fiszel <ruben@windmill.dev>	2025-09-30 19:32:34 +00:00
wendrul	9325f5636c	fix: Don't reencrypt secrets on workspace forking (#6622) ... * Do not reencrypt variable when performing a workspace fork * Update SQLx metadata * Update SQLx metadata * Remove unused --------- Co-authored-by: GitHub Action <action@github.com>	2025-09-17 16:47:27 +00:00
hugocasa	b8a2371a44	cli and git sync (#6568)	2025-09-10 12:22:28 +00:00
dieriba	f3fd1e90b0	feat: refactor trigger crud (#6472) ... * base * base crate * websocket updated * crud refactored * fix and nits * fix compiler warning, nits and update repo ref * fix oss * fix compilation * update ref * fix * update feature * listener base * add listener * refactor logic done and implemented for postgres * fix capture * websocket * implem for all triggers * update sqlx,repo ref and fix * fix oss * unify struct fix websocket * nits and fix oss runtime axum error * perf cache query * update .sqlx * update .sqlx * fix * fix unused * fix * fix * nits and fix http handler update endpoint * update .sqlx * update repo ref * nits * fix * update ref * fix * update .sqlx	2025-09-02 11:28:06 +00:00
pyranota	11dd4118ce	feat: add ruby support (#5939) ... * implement single line pin * make panic-safe * use pin even if multiple modules imported withing single statement * add repins and make imports respect pins * keep all pins * Allow multiple pins * add comments + handle stuff more safely * fix fully qualified imports * remove ignore * sort nested * apply unique to output requirements list * fix typo * remove mut * update sqlx * sort imports * sort imports * fix formatter and format * feat: add ruby support * fix * update sqlx * add ruby to flake.nix Signed-off-by: pyranota <pyra@duck.com> * upgrade par_install_language_dependencies helper rename fields + allow passing custom payload Signed-off-by: pyranota <pyra@duck.com> * change migrations Signed-off-by: pyranota <pyra@duck.com> * make full pipeline work! Signed-off-by: pyranota <pyra@duck.com> * do a bit fixing * stage Signed-off-by: pyranota <pyra@duck.com> * add ruby in dockerfiles Signed-off-by: pyranota <pyra@duck.com> * implement auto-require + 'windmill/inline' Signed-off-by: pyranota <pyra@duck.com> * create 'windmill/inline' only if does not exist Signed-off-by: pyranota <pyra@duck.com> * add nsjail Signed-off-by: pyranota <pyra@duck.com> * update global settings Signed-off-by: pyranota <pyra@duck.com> * add ruby icon * generate lockfile on deploy (but broken) Signed-off-by: pyranota <pyra@duck.com> * fix(frontend): display deployed script lockfile and other info Signed-off-by: pyranota <pyra@duck.com> * feat: add mini-wmill client Methods: - get_resource - get_variable Additionally add shortcuts on frontend Signed-off-by: pyranota <pyra@duck.com> * add nit warning Signed-off-by: pyranota <pyra@duck.com> * support private repos credentials Signed-off-by: pyranota <pyra@duck.com> * fix compilation error Signed-off-by: pyranota <pyra@duck.com> * found rustc bug + refactor universal installer Signed-off-by: pyranota <pyra@duck.com> * more refactor + fixes Signed-off-by: pyranota <pyra@duck.com> * bug fixing + polishing Signed-off-by: pyranota <pyra@duck.com> * cleanup Signed-off-by: pyranota <pyra@duck.com> * change 'windmill/inline' Signed-off-by: pyranota <pyra@duck.com> * delegate env_clear to universal installer Signed-off-by: pyranota <pyra@duck.com> * update init script Signed-off-by: pyranota <pyra@duck.com> * add cli support Signed-off-by: pyranota <pyra@duck.com> * unhardcode frontend Signed-off-by: pyranota <pyra@duck.com> * fix cli Signed-off-by: pyranota <pyra@duck.com> * fix init script Signed-off-by: pyranota <pyra@duck.com> * make it work on ms windows Signed-off-by: pyranota <pyra@duck.com> * do not clean env in the end Signed-off-by: pyranota <pyra@duck.com> * fix windows (again) Signed-off-by: pyranota <pyra@duck.com> * include PROXY_ENVS (untested) Signed-off-by: pyranota <pyra@duck.com> * feat: self-signed certs + proxy support (tested) Signed-off-by: pyranota <pyra@duck.com> * flake: pin ruby to 3.4 * merge follow-ups Signed-off-by: pyranota <pyra@duck.com> * remove obsolete libc crate Signed-off-by: pyranota <pyra@duck.com> * cleanup parser Signed-off-by: pyranota <pyra@duck.com> * fix java ci Signed-off-by: pyranota <pyra@duck.com> * cleanup Signed-off-by: pyranota <pyra@duck.com> * fix nits Signed-off-by: pyranota <pyra@duck.com> * Update backend/windmill-worker/src/universal_pkg_installer.rs Co-authored-by: ellipsis-dev[bot] <65095814+ellipsis-dev[bot]@users.noreply.github.com> * Update backend/windmill-worker/src/ruby_executor.rs Co-authored-by: ellipsis-dev[bot] <65095814+ellipsis-dev[bot]@users.noreply.github.com> * add nsjail for lock stage Signed-off-by: pyranota <pyra@duck.com> * fix merge Signed-off-by: pyranota <pyra@duck.com> * Update common.rs * regen windmill-api-client Signed-off-by: pyranota <pyra@duck.com> * fix tests Signed-off-by: pyranota <pyra@duck.com> * fix tests again Signed-off-by: pyranota <pyra@duck.com> * reference published ruby parser Signed-off-by: pyranota <pyra@duck.com> --------- Signed-off-by: pyranota <pyra@duck.com> Co-authored-by: ellipsis-dev[bot] <65095814+ellipsis-dev[bot]@users.noreply.github.com>	2025-08-15 13:03:32 +00:00
dieriba	852bf064dc	feat: togglable manual acknowledgement for gcp trigger (#6321) ... * update * done * update repo ref * nits * update repo ref * fix * all --------- Co-authored-by: Ruben Fiszel <ruben@windmill.dev>	2025-08-07 17:45:50 +00:00
dieriba	9fbb1992e5	feat: add email workspace error handler if smtp is configured (#6267) ... * detecting trigger kind on job error * v0 done * done * updat .sqlx and add mgrations * fix: set back vs code settings and unused * fix unused import * add feature flag * update ref * nits * nits * done * update repo ref * remove * add isCloudHosted * update sqlx * big * fix * use extra args instead of new col * nits * update .sqllx * update ref * update email * fix * fix * fix * nits * update script and use env base_internal_url * update sqlx and fix query * remove * fix unused import * update ref	2025-08-07 16:46:53 +02:00
HugoCasa	328ef605ad	feat: triggers error handler and retry (#6138) ... * feat: http triggers error handling and retries * feat: handle sync http triggers * feat: triggers retry and error handling * feat: frontend and preprocessor * fix build * minimum allow dead code * check workspace tags in schedules * fix flow preprocesor + hub urls * update ee ref * nit * nit * final nits * schedule UI nits * migrate workspace settings to svelte 5	2025-07-11 18:55:23 +02:00
dieriba	aba8c01d7f	feat: windmill http triggers and webhooks to openapi spec (#5918) ... * add sum and description to http routes * add to openapi spec * add subsection * add collaspable and action button * added path rendering, * migrate logic to backend * remover server and handle different format generation * add filter for http route and document generated * remove print * handled webhook and integrated server component * done * add download and copy * nits * update .sqlx * remove .vscode * fix npm check * add summary description and fix toggle * added security handling and nits * update .sqlx * nits and rename key * remove unused code * update ref * nits rename var * nits * nits * nits * nits * nits * add token generation for cURL command * create token * update label and remove section * update repo ref * clean * let brower start download before cleaning up resource * handle sync/async for webhook * format fix * nits * nits * reset sum and description	2025-06-17 19:49:00 +02:00