mirror of
https://github.com/windmill-labs/windmill.git
synced 2026-08-18 16:02:10 +00:00
fix/hook-tests-worktree-root
45 Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 hugocasa
|
ee533273dd |
fix: confine jobs:run tokens to the jobs of the runnables they may start (#10635)
* fix: confine path-scoped jobs:run tokens to their runnable's jobs Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix: project singlestepflow onto its runnable and confine kind-only run scopes Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix: keep every by-id job read reachable by a jobs:run token Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix: whitelist the dbt and wac-approval by-id job reads for run tokens Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix: let an apps:run scope satisfy job-read confinement for that app's runs Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix: apply run-scope confinement on top of the approval-token read bypass Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix: confine the resume-secret job reads to the run scope as well Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com> |
||
 Ruben Fiszel
|
633d7bcb2e |
feat: add trigger_history table with source tracking (#10696)
* feat: add trigger_history table with source tracking Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix: gate trigger history reads on scopes and harden its writers Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix: filter trigger history scopes in SQL and match the cleared-handler diff Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix: record a trigger restore from the trashbin in its history Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix: record bulk http trigger creates and document the recording boundary Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix: lock the trigger row when capturing its history preimage Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix: only record an auto-disable that actually flipped the schedule Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * chore: state the auto-disable invariant once instead of at four call sites Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * feat: render trigger history changes as a structured field diff Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix: make a server-initiated disable atomic with its history row Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * docs: note that the auto-disable savepoint takes no pool connection Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * docs: note the flow fallback is the last chance to disable Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix: never leave a trigger enabled because its history row failed Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix: retry the disable history row instead of dropping it on first failure Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix: use the design-system Button for the change-value expander Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix: hold the trigger row lock across its disable history row Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix: keep the history-loss alert out of the listener cancellation race Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix: read the history workspace through the trigger-workspace seam Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com> |
||
 Diego Imbert
|
7e1c1fa3a4 |
feat(apps): use the windmill-client SDK from raw app frontend code (#10377)
* feat(apps): use the windmill-client SDK from raw app frontend code
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_018Gmsk9kAG7p9t2Qy6ADRJz
* fix(apps): bound the raw app SDK token to deployed runnables
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_018Gmsk9kAG7p9t2Qy6ADRJz
* fix(apps): deny dependency jobs and survive a failed SDK mint
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_018Gmsk9kAG7p9t2Qy6ADRJz
* fix(apps): confine the SDK token's users scope to the viewer's identity
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_018Gmsk9kAG7p9t2Qy6ADRJz
* docs: describe the full raw-app SDK sentinel narrowing
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_018Gmsk9kAG7p9t2Qy6ADRJz
* fix(apps): deny workflow-as-code replay for raw app SDK tokens
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_018Gmsk9kAG7p9t2Qy6ADRJz
* fix(apps): deny preview-flow restart replay for raw app SDK tokens
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_018Gmsk9kAG7p9t2Qy6ADRJz
* fix(apps): re-prompt when an app widens its SDK scopes mid-consent
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_018Gmsk9kAG7p9t2Qy6ADRJz
* feat(apps): support the frontend SDK in sandboxed raw apps
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_018Gmsk9kAG7p9t2Qy6ADRJz
* fix(apps): hand the sandboxed SDK token over only once per loaded document
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_018Gmsk9kAG7p9t2Qy6ADRJz
* fix(apps): bind the sandboxed SDK handoff to the document we loaded
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_018Gmsk9kAG7p9t2Qy6ADRJz
* fix(apps): use an unguessable nonce for the sandboxed SDK handoff
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_018Gmsk9kAG7p9t2Qy6ADRJz
* fix(apps): reply to the sandboxed SDK handshake over its own port
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_018Gmsk9kAG7p9t2Qy6ADRJz
* fix(apps): answer the raw app handshake only over a transferred port
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_018Gmsk9kAG7p9t2Qy6ADRJz
* fix(apps): set frontend_sdk_scopes in the S3-gated policy literals
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_018Gmsk9kAG7p9t2Qy6ADRJz
* docs: describe the sandboxed wrapper's credential as it now works
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_018Gmsk9kAG7p9t2Qy6ADRJz
* ui nit
* feat(apps): make the frontend SDK work in the raw app editor preview
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_018Gmsk9kAG7p9t2Qy6ADRJz
* fix(apps): guard the preview token mint and drop superseded responses
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_018Gmsk9kAG7p9t2Qy6ADRJz
* fix(apps): refuse job tokens on every raw app SDK mint path
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_018Gmsk9kAG7p9t2Qy6ADRJz
* docs: correct the mint caller list and the preview retry rationale
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_018Gmsk9kAG7p9t2Qy6ADRJz
* fix(apps): apply the consent response's render mode before rendering
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_018Gmsk9kAG7p9t2Qy6ADRJz
* fix(apps): restart the viewer when a redeploy changes the render mode
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_018Gmsk9kAG7p9t2Qy6ADRJz
* fix(apps): restart on every render-mode change, not just the first
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_018Gmsk9kAG7p9t2Qy6ADRJz
* fix(apps): clear the preview's SDK credential when scopes go away
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_018Gmsk9kAG7p9t2Qy6ADRJz
* fix(apps): remove window.process in the preview instead of blanking it
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_018Gmsk9kAG7p9t2Qy6ADRJz
* docs: cut the raw app SDK comments down to the invariant
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_018Gmsk9kAG7p9t2Qy6ADRJz
* refactor(apps): use randomUUID for the raw app handshake nonce
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_018Gmsk9kAG7p9t2Qy6ADRJz
* fix(apps): re-read the render mode before rendering without a token
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_018Gmsk9kAG7p9t2Qy6ADRJz
* fix(apps): make the raw app handshake nonce unguessable again
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_018Gmsk9kAG7p9t2Qy6ADRJz
* chore: pin the EE ref to a commit that builds against this OSS tree
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_018Gmsk9kAG7p9t2Qy6ADRJz
* refactor(apps): authenticate the raw app preview by session instead of a token
The editor preview is same-origin and unsandboxed, so app code there already
holds the editing user's session cookie. Minting a scoped bearer for it added
an endpoint and a portable 12h credential without containing anything.
Inject only BASE_URL and WM_WORKSPACE: `windmill-client` falls back to
credentialed same-origin requests when it finds no token, so the SDK runs as
the editing user. Drops POST /apps/preview_sdk_token and the mint/race
handling in the editor.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01KPCW1WB5QeYrgJmgwcywNA
* fix(sdk): send credentials only outside the browser
The API answers `Access-Control-Allow-Origin: *` and never sets
`allow_credentials`, so a credentialed cross-origin request fails before the
bearer is read — which is what a sandboxed raw app issues. Keying this on the
browser rather than on `WM_TOKEN` leaves non-browser callers byte-identical,
and browsers keep sending cookies same-origin through fetch's own default.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01KPCW1WB5QeYrgJmgwcywNA
* remove windmill-client from templates
* fix(sdk): drop credentials only for raw app bundles
A sandboxed raw app calls the API from an opaque origin, and the API answers
`Access-Control-Allow-Origin: *`, which a credentialed request can never pair
with. Gate on WM_RAW_APP, set by the two places that build a raw app's
`window.process.env`, so every other windmill-client consumer is untouched.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01KPCW1WB5QeYrgJmgwcywNA
* feat(apps): make frontend SDK access sandbox-only
An unsandboxed bundle runs same-origin with the viewer's full session, so a
consent prompt there implies a boundary that does not exist and the token adds
nothing it could not already do. Advertise scopes and mint only when isolation
is on; turning the toggle off clears the declared scopes with it.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01KPCW1WB5QeYrgJmgwcywNA
* Revert "refactor(apps): authenticate the raw app preview by session instead of a token"
This reverts commit
|
||
|
Ruben Fiszel
|
032300e28e |
feat: run dbt projects as a first-class Windmill runtime (#10326)
* fix: mount only the engine in the dbt jail, reject shadowed and malformed args
Review round 42.
The jail mounted the whole dbt cache directory, whose siblings of the
engine are `repos/` and `packages/` — other workspaces' private checkouts
and package trees, kept apart by cache key rather than by permissions. A
jailed project could read them. It now mounts the engine's own directory,
which the provisioner names; verified from inside the jail that `repos/`,
`packages/` and `state/` are invisible while the engine stays usable.
A `{{ placeholder }}` may no longer take the name of a run argument this
runtime defines. It was silently dropped from the signature, so a
descriptor like `value: "{{ select }}"` deployed and then could not be
run at all: the built-in `select` is an array and the interpolation needs
a scalar. Refused at parse, so the deploy says so.
A `vars` override that is not an object is refused rather than ignored.
Argument-schema validation is opt-in, so a string or an array silently
ran the descriptor's own vars — against a different schema or alias than
the caller asked for. `select` and `exclude` already refused theirs.
* feat(dbt): the project is the script's module bundle, not a git checkout
A dbt script now carries its whole dbt project as its module bundle. The
descriptor is the script content; `<script>__dbt/` holds the project verbatim,
so importing an existing project is `cp -r` plus `wmill sync push`, and the
worker materialises the bundle into the job directory instead of cloning.
Backend
- `prepare_project` writes the script's modules and requires `dbt_project.yml`
at the bundle root. `checkout`, the git-ssh command, the clone cache and the
repository resource are gone, along with `repo`, `project`, `ref` and
`git_ssh_identity` on the descriptor.
- Run identity and the package cache key take a `project_digest` (sorted SHA256
over the bundle) where the commit used to sit, so an edited project cannot
resume a previous run's `run_results.json` or reuse its `dbt_packages`.
- The per-run graph re-ingest is now gated on `vars` placeholders and `$var:`
env alone.
- `capture_dependency_job` takes the script's modules so a dependency job, which
has no generic module-writing step, materialises them itself.
- `dbt deps` caching strips the git remote from every package it cached, not
just the tree root: `packages.yml` can render a token into a `git:` URL.
- `git_clone.rs` is dropped and `ansible_executor.rs` returns to its own copy of
the clone helpers.
CLI
- `wmill sync pull` keeps a dbt script's lock beside its folder rather than
inside it, so the folder holds nothing but the project.
- Directories dbt generates (`target-path`, `packages-install-path`,
`clean-targets` and the usual defaults, read from `dbt_project.yml`) are
excluded from the bundle, from the sync diff and from staleness hashing.
- A module-only edit now pushes its parent dbt script and is reported as a
changed module rather than passing unnoticed.
* fix: keep a script's modules in the worker's file-system cache
The first fetch of a script version reads the database and carries its
modules; every later fetch imports from the worker's cache directory, whose
`RawScript::import` hard-coded `modules: None` and whose `export` never wrote
them. A worker restart therefore started running the script without its own
files, silently — for a dbt script, without its project, which fails with
"carries no project"; for any other script with a module bundle, with the
imports missing.
`modules.json` is now written on every export and required on import, so an
entry written by an older version fails to import and is refetched rather than
serving a stripped script for as long as the directory lives.
Also derives a dbt run's `project_digest` from the bundle the run actually
carries: `handle_dbt_job` was passing `None`, which collapsed every project in
a workspace onto one digest and let `dbt retry` resume a different project's
`run_results.json`.
* fix(dbt): give every phase the script's environment, bound the cache copies
`dbt deps` ran without the script's environment variables on an unsandboxed
worker, so a `packages.yml` resolving a private package URL through
`env_var()` could not see them while the package cache key was still built on
their digest. `with_invocation_env`, applied at three of the four call sites,
is folded into `dbt_command` so no phase can be added without it, and
`DBT_TARGET_PATH` is set after both environments rather than before.
The package cache copies ran through a bare `Command::output()`: the tree is
the project's, so a cancelled or timed-out job held its worker slot until `cp`
finished. Both the restore and the publish now run under the job poller like
every other phase.
* fix(dbt): only offer commands whose writes match the graph, honour packages-install-path
`dbt_command: run` is dropped from the allowed overrides. Asset dispatch fires a
script's deploy-time writes on any successful job, and `dbt run` covers models
only, so a project with seeds or snapshots notified consumers of relations the
invocation left stale. That is the same reason `test` was already excluded.
Narrowing what a run touches is `select`/`exclude`, which scope the graph too.
`dbt deps` writes to the project's `packages-install-path`, so a project that
moved it got no package cache at all: the publish found nothing at
`dbt_packages` and every job resolved its dependencies over the network again.
The path is read from `dbt_project.yml` and validated as project-relative,
since both cache copies are rooted at it.
Also states the sidecar's mutator contract at the module level: the dbt manifest
tables carry no RLS and grant `windmill_user` full access, so a user-scoped
transaction is not enforcement and every caller must have verified write access
to the script itself.
CLI: a module file is now grouped with its parent script for the push. Left in
a group of its own it got its own `alreadySynced`, so a push touching several
files of one bundle deployed the script once per file; the resulting versions
raced, and the asset graph could end up describing none of them.
* fix(dbt): seed a project for browser-created scripts, refuse a no-op retry
A dbt script created in the browser only got a descriptor, and the runtime
refuses a script whose bundle has no `dbt_project.yml`, so the advertised
Create → Deploy → Run path always failed its dependency job. New dbt scripts
now start with a project that builds: pointing `profile.resource` at a
warehouse is the one edit, and growing it is `wmill sync pull` plus a local
editor, which is where dbt development happens.
`dbt retry` builds its graph from the previous run's error, fail and skipped
nodes alone, so retrying an all-green run selected nothing and wrote nothing —
and a job that succeeds having written nothing still dispatches every
deploy-time write, waking every downstream consumer for relations no one
touched. Refused, with the reason.
CLI: a configured `target-path` or `packages-install-path` may be nested
(`build/target`), and `clean-targets` has a block form as well as an inline
one. Both are now parsed, and the exclusion compares the project-relative path
rather than the top-level segment, so a nested generated tree no longer lands
in the bundle and no longer makes a local `dbt run` look like a project change.
* fix(dbt): lock a project once, find the parent on either path separator
A dbt script's modules are its dbt project, not helper code with dependencies
of its own, so the generic per-module lock loop is skipped for it: the parent
lock already ran `dbt deps` and `dbt parse` over the whole project. Locking
each file separately re-materialised the bundle and re-invoked dbt once per
file, so a project of N files paid N project-sized passes and a large one timed
the deploy out. The 13-file fixture went from 14 relock passes to 1.
`pushParentScriptForModule` searched the raw path for `__dbt/`, so on Windows,
where the folder is spelled `__dbt\`, a module-only edit returned without
deploying its parent while the caller still recorded the file as synced. It now
goes through `getScriptBasePathFromModulePath`, which normalizes separators.
Also drops the last of the external-repository wording from the descriptor's
module docs and from the `codebase` rejection a user can hit.
* feat(dbt): infer the run form locally, keep test-only retries from cascading
`windmill-parser-wasm-yaml` 1.770.0 carries `parse_dbt`, so the browser and the
CLI derive a dbt script's run arguments from its descriptor instead of waiting
for the deploy to hand back a schema. Pins bumped in both.
`generate-metadata` was rewriting a dbt script's `lock` field on every run: a
dbt lock comes from the dependency job on a worker, so nothing generates it
locally and the resolved `!inline` reference was left inlined into the metadata
or blanked. It is restored instead, and a push straight after
`generate-metadata` is a no-op again.
A retry now needs a failed node that materialises something. `dbt retry` builds
its graph from error, fail and skipped nodes, and with `test_behavior:
after_all` a failing test is what `run_results.json` ends up describing — so the
retry reran tests, wrote nothing, succeeded, and still dispatched every
deploy-time write.
The dbt badge's destination is deterministic: writers outrank readers, and among
several writers of one relation (which the backend permits) the smallest id
wins, rather than whichever write edge arrived last.
* feat(dbt): browse the project and read a run's per-node result
Two views a dbt user expects and that the generic script surfaces do not give.
**The project.** A dbt script's editor gains a Project tab beside its
descriptor: the module bundle as the tree dbt itself expects, each file
read-only with syntax highlighting. The existing module tab strip is a flat row
built for a couple of helper files and does not survive a real project; a
13-file fixture already overflows it. Directories sort before files so it reads
like the checkout on disk, and an empty bundle explains the `cp -r` instead of
showing a blank pane.
**The run.** `DisplayResult` renders a dbt invocation's per-node breakdown above
the raw payload: totals, then a table of node, kind, target relation, rows and
time, with failures and warnings sorted first and carrying their message. The
data was already structured; it was being shown as JSON to scroll and PASS/WARN
counts to find in the log. On a failed run the same JSON rides in the error
message after the exit-status line, so it is parsed back out — that is the case
worth rendering, since the failing node is what the user came for.
* docs(dbt): say that profile.resource is what buys the asset graph
The starter descriptor described `profile.resource` as the thing rendered into
profiles.yml, with the project's own file as an equal alternative. It is not
equal: the resource PATH is the warehouse's identity in the asset graph, so a
project bringing its own profiles.yml runs fine and silently gets no assets, no
lineage and no cascade. The deploy already says so in its log; now the
descriptor a user starts from says it too, before they choose.
* fix(dbt): authorize a resource used only for asset identity, clean up after failed installs
A descriptor setting both `profile.profiles_yml` and `profile.resource` took
its connection from the project's file but returned the resource path as the
graph's warehouse identity without ever reading it. A script editor could
therefore publish `table://<any resource>/...` writes, and wake that
warehouse's subscribers, while connecting somewhere else. The resource is now
read on that path too — reading is what authorizes it — so the combination
keeps working for the case that wants it (keep your own profiles.yml, still get
lineage) and fails closed otherwise.
Provisioning cleaned up its staging directory only on the paths someone
remembered, so a run of failed or cancelled first-use installs accumulated
venvs, tarballs and installer scripts until the worker's disk was gone. All
three engines now hold their scratch paths in a guard that removes them on
drop, which is the one exit every path takes, cancellation included.
Frontend: `partial success` is dbt's word for a node that built but whose tests
failed, counted in `totals.error` and redone by a retry, so it ranks with the
failures instead of rendering green with its message hidden. And the run panel
now keys off the worker's engine discriminator rather than `{nodes, totals}`,
which is a shape an ordinary script can return. Both pinned by unit tests on
the extracted `parseDbtRun` helpers.
* feat(dbt): show a run's models on the run page
The run page is where you land on a running job, and until now it showed a dbt
run as streaming text: the per-node table only renders once the job has
produced a result, and the graph that moves per model lived on the pipeline
page you had to navigate to. A Models section now sits above the result,
scoped to the running script's own relations and its `ref()` lineage, polling
while the job is in flight so nodes move as dbt walks the DAG.
No `resolveGraph`: that merges drafts and live editor buffers into the
persisted graph, and a run page has neither.
* feat(dbt): retry failed nodes automatically, and from any worker
**Node-level retry, in the job.** `retry_failed_nodes: {attempts, delay_seconds}`
rebuilds only what a failed build left failed or skipped, before the job reports
failure. dbt confines a failure to its own subtree and `dbt retry` resumes
exactly that set, so a transient warehouse error costs those nodes rather than
the project. Doing it in-job is what keeps the state question out of it: the
previous attempt's `run_results.json` is still in the job directory, so there is
nothing to persist and no worker to land back on. This is the granularity
astronomer-cosmos gets from one Airflow task per model, without the ~6x that
per-model tasks measured.
A retry's `run_results.json` names only the nodes it redid, so it overlays the
accumulated results rather than replacing them: the job's result has to be every
node the job touched, or the nodes that succeeded before the retry settle no
materializations. Pinned by a test.
**Durable retry state.** `run_results.json` is now saved to `dbt_run_state` as
well as the worker's local cache, so an explicit `dbt_command: retry` works from
any worker of the group rather than only the one that failed. Only the results
are stored: `dbt retry` also needs `manifest.json`, roughly sixty times larger
and growing with the project (732 KB against 12 KB on the six-node fixture), but
the manifest is a pure function of the project files, vars and env, all of which
the stored identity already pins, so a worker restoring from the database
re-derives it with a `dbt parse` of about a second.
* fix(dbt): restore the sqlx cache, make retries cancellable and path-aware
**SQLx cache.** A `cargo sqlx prepare` deleted 750 entries, including the
enterprise queries CI needs under `SQLX_OFFLINE=true`, and the check that was
supposed to catch it reported zero losses because it was run from `backend/`
with a `backend/`-prefixed path, so its baseline was empty and it failed open.
All 750 are restored; the branch now adds 19 and deletes none, and
`SQLX_OFFLINE=true cargo check` passes.
**Retry backoff observes cancellation.** `canceled_by` is only written by the
job poller, which does not run between attempts, so re-reading it reported the
state as of the failed attempt and missed every cancel issued during the wait
— the whole window the check exists to cover. The wait now reads
`v2_job_queue.canceled_by` each second, and the job's deadline is honoured
before starting another dbt process.
**Retry state follows its script.** `dbt_run_state` is path-keyed like the
manifest sidecar but, unlike it, nothing regenerates it: a rename moves the row
so a resumable failure survives, while archive and delete clear it, so a script
later created at that path cannot inherit a stranger's failure and its
arguments.
**CLI.** `table` joins ducklake and s3object in the local graph's auto-trigger
kinds, matching `is_auto_trigger_kind` and the frontend's set; without it a
local graph and the generated docs omitted a cascade edge the deploy has.
* fix(dbt): carry only the project files a bundle can hold, and say what it drops
Exploring real and edge-case projects surfaced three frictions, all in the
import path a user hits first.
**A binary file broke the push, opaquely.** dbt projects carry images under
`docs/`, stray `.DS_Store` files and occasionally a parquet seed. Read as text
they become mojibake, and a NUL among them is rejected by Postgres with
`unsupported Unicode escape sequence` — which `wmill sync push` then reported as
success, exiting 0 with the script never created. Binary files are now detected
the way `git` detects them, by a NUL in the first 8000 bytes rather than by
extension, and skipped with the reason.
**The size guard the docs promised did not exist.** Now it does: 5 MB per file,
which only ever catches a committed dataset. Real dbt code is about 500 bytes
median and 1.9 KB at p90.
**Skipped files became a permanent phantom diff.** The push dropped them while
the sync diff still offered them, so every push reported changes no push could
resolve. One predicate now answers for the push, the staleness hash and the
diff.
Verified on a project with unicode filenames and content, CRLF endings, an
empty model, an ephemeral model, a disabled model, a `.md` docs block, an
extensionless README, six levels of nesting, a 7.6 MB seed and a PNG: it
pushes, round-trips byte-for-byte through pull, deploys to 7 dbt nodes and 6
`table://` assets (ephemeral and disabled correctly absent), and runs green.
* fix(dbt): resolve dbt-core against the adapter, settle partial success, unify status
**Adapters could not be provisioned.** The 1.x engine pinned `dbt-core` to a
fixed version independent of the adapter, but several adapters cap below it:
`dbt-mysql` at `~=1.7`, `dbt-oracle` and `dbt-databricks` below 1.12, and
`dbt-salesforce` has no package at all (it exists only inside Fusion). Those
projects failed at provisioning with a uv resolver dump. The install now asks
for a range and lets the adapter choose, and records what the resolver picked so
the lock pins a version that adapter can take.
The floor is the CLI this runtime invokes: resolving down to dbt-core 1.7
produced a working venv that then failed with `No such option '--target'`, which
is worse than not resolving. An adapter with no release in range now fails
naming itself and pointing at `dbt-core-2x` or `fusion`, instead of a resolver
dump. Salesforce is refused up front with the reason.
**`partial success` left a model stuck on `Running`.** It is dbt's word for a
node that built and then failed its tests, and it was already treated as a
failure when counting totals and deciding a retry — but the two sites that
settle the RELATION fell through to "says nothing", so the tailer's `Running`
was never replaced and a finished job showed a model still building. Six status
comparisons had drifted apart, two folding case and four not, while dbt-core 1.x
echoes the author's casing and 2.x uppercases; they are now one classifier.
**Agent workers.** The durable retry state and the cancellation poll both need a
database, which an agent worker reaches only through the API. The automatic node
retry is refused there rather than running a wait it could not interrupt, and
the docs say "any worker with a database connection" instead of overclaiming.
Also clears `dbt_run_state` when a path stops being a dbt script, and moves
`run_identity`'s contract onto `run_identity` from the digest helper below it.
* feat(dbt): show the transform behind a model on the run graph
The run page's graph carried a node for the script itself and drew every
relation as a bare table. Both were wrong for that page: the graph there is
already scoped to one script, so a node standing for it distinguishes nothing
(on the pipeline page it separates one project from another, which is why it
exists), and dbt's own DAG node is the model — the SQL and the relation it
writes are one thing, so a graph of relations alone leaves out what a reader
came to see.
The script node is dropped, and selecting a model now shows its SQL underneath
the canvas with its file path and materialization, read-only, the same view the
pipeline details pane gives.
* feat(dbt): move the graph with the run
The worker has always recorded a state per relation as dbt walks the DAG —
`running` when a model starts, `materialized` or `failed` when it ends — but
nothing rendered it: the graph response carries what a relation IS, not what a
particular run is doing to it, so the canvas had nothing to show and a running
job looked identical to a finished one.
`assets/run_progress/{job_id}` returns that state for one job, the run page
polls it beside the graph, and the asset node carries a spinner or its outcome.
Errors and retries need nothing extra: a failed node writes `failed`, and an
in-job retry rewrites the same row, so the node returns to `running` and on to
its new outcome by itself.
`materialized_partition` holds a relation's CURRENT state keyed by relation, so
filtering on `job_id` returns exactly what this run last touched — which is the
question a run page asks, and why a superseded older run shows nothing.
* feat(dbt): a dbt project is not a data pipeline
Deploying a dbt script marked it `auto_kind = 'pipeline'`, which enrolled it
in pipeline membership: the folder became a Pipeline entry on the home page,
the script folded into it, and `/pipeline/<folder>` opened a canvas holding
the project's whole model DAG next to the pipeline's own scripts. A folder
holding both then read as two projects in one editor, and the pipeline editor
offered to author transforms that are in fact authored in a local `dbt run`
loop and pushed as the script's bundle.
A dbt script is now never a pipeline member, and the pipeline canvas drops the
dbt script node. Its models stay, with their `ref()` lineage: the relations are
what a downstream pipeline script reads, and dropping them would break the
cascade from a dbt run — the point of giving dbt models `table://` identity.
Also drops a screenshot committed to this branch by accident.
* fix(dbt): authorize run_progress through the job, drop dbt from the local graph
`run_progress` read `materialized_partition` through `user_db` on the
assumption that RLS would scope the rows. That table has RLS disabled and no
policies, so any workspace member could pass a job id and read that run's
relation paths, row counts and error text. It now joins `v2_job`, which does
carry per-user policies, so a caller who cannot see the job sees nothing —
the same pattern `v2_job_completed` reads need. Verified as a plain member:
the old query returned 6 rows for another user's run, the new one returns 0,
while the job's owner still sees all 6.
The CLI's local graph still forced `in_pipeline` on every dbt script, so
`pipeline docs --local` and `pipeline dev` kept presenting a dbt project as a
pipeline the deploy no longer enrolls. It now skips them, matching the server.
A dbt descriptor has no asset parser locally, so nothing is lost: its models
come from the manifest the deploy derives.
Declares `run_progress` in openapi.yaml so the frontend uses the generated
client instead of a handwritten fetch; the generated `status` union also
replaces a hand-rolled string mapping.
* fix(dbt): drop the dbt node from the CLI's deployed pipeline views too
`pipeline dev` and `pipeline docs` (without `--local`) read `/assets/graph`
directly. That endpoint is asset-usage driven rather than membership driven, so
it returns a dbt script like any producer — and both commands render every
runnable, so a dbt project still showed up as a pipeline script there after the
local builder stopped emitting one.
`hideDbtRunnables` mirrors the frontend's projection of the same payload. It is
generic over the graph shape so the bounded-cascade view (`BCGraph`, a narrower
type over identical JSON) passes through without a cast.
The relations stay: they are what a downstream pipeline script reads, and the
node is what attributes them to a producer for every other consumer of the
endpoint, so the filter belongs in the views rather than the query.
* fix(dbt): narrow a selective run's cascade, settle the finished run graph
Review-round fixes.
A `select`/`exclude` run builds part of the project, but asset dispatch reads
the deploy-time write set for the whole script, so a run selecting one model
woke the subscribers of every other. Dispatch now intersects that set with the
relations the run actually recorded as materialized, scoped to dbt because it is
the only producer whose write set is decided per run. A run that recorded
nothing still dispatches everything, so an agent worker whose reconciliation
failed cascades as before. Verified both ways: `select: [extra_model]` no longer
wakes the `fct_orders` subscriber, and a full run still does.
`hideDbtRunnables` keyed its removal set on path alone while the graph keys
runnables by `(usage_kind, path)`, so a flow sharing a path with a dbt script
lost its node, edges and triggers too. Both copies now key on the pair.
The run graph never took a final reading when a job finished, so the last state
shown was whatever the tick before completion saw. Only `dbt-core-1x` streams
node events; the other engines record every relation during end-of-run
reconciliation, so their finished graph showed nothing until a reload.
`DbtNodeOutcome::Inconclusive` collapsed statuses the tally has to tell apart,
so two sites re-lowercased the status beside the classifier and `no-op` landed
in `totals.error` — a clean run reporting an error in its own result. Split into
Warn / Skipped / NoOp / Unknown so every site falls out of one match; `no-op` is
kept out of the retry set, which dbt spells as error / fail / skipped.
Also: reattach two doc comments to the items they describe, and correct the
engine-distribution table — only dbt-core-2x is baked into the images, 1.x is a
per-adapter venv provisioned on first use, and the default is compiled in rather
than an instance setting.
* fix(dbt): make the model chip inert where its project node is not on the graph
The canvas passed `onDbtSelect` unconditionally, so the chip always rendered
`cursor-pointer` and hover-highlighted — but the owner map is empty on both
graphs this feature added, since the run page carries no runnables and the
pipeline page hides the dbt node. The chip advertised a click that resolved to
nothing. It now takes its handlers only when the relation has an owner on this
graph, so it stays live on the surfaces that do show the project node.
`classify_status` and `DbtNodeOutcome` were `pub` in a private module with no
caller outside the file, unlike every neighbour.
* fix(dbt): take the cascade's write set from the run's own result
The previous narrowing read `materialized_partition`, which was wrong twice.
That table keeps one row per relation and the newest writer takes `job_id`, so
two overlapping runs over the same model erase each other's claim to it: the
earlier job would dispatch a subset of what it built, or none of it.
And an empty row set was read as "recording failed, dispatch everything" when it
is also a real answer. A `select` matching no model, or one resolving to tests
only, exits 0 having built nothing — and then woke every consumer of every model
in the project, which is the opposite of what the narrowing exists to do and is
reachable by a typo in a run argument.
The run now reports the relations it materialized in its own result, which is
immutable and per job. Absent means the producer said nothing (a job from before
the field, a non-dbt producer) and the whole deploy-time set dispatches as
before; present-and-empty means it built nothing and dispatches nothing.
Verified on all three: an unmatched selector builds nothing and wakes nobody, a
selector naming one unsubscribed model wakes nobody, and a full run wakes the
subscriber.
Also indexes `materialized_partition (workspace_id, job_id)` -- the run page
polls that shape every 2s and no existing index leads with `job_id` -- corrects
the selective-cascade section of the design doc, which still described the old
deploy-time behavior, and reattaches `buildLocalPipelineGraph`'s doc comment.
* docs(dbt): attach the CLI JSDoc to its function, correct the index rationale
The `hideDbtRunnables` JSDoc ended up documenting the type declared beneath it —
made while fixing the same mistake one function down.
The migration's comment credited the cascade with a `job_id` lookup that the
same commit replaced with a read of the job's own result. The run page's poll is
the only reader keyed on that column.
* fix(dbt): refuse graph publication for a removed script, allow test-only retries
An archived or hard-deleted script could still republish its graph: the
publication guard filtered `deleted` but not `archived`, and treated a missing
row as "nothing newer exists" rather than "nothing left to publish for". A
dependency job or dynamic run finishing after the removal put the asset,
provenance and subscription rows back with nothing left to clear them.
`dbt_command: retry` refused a run whose only failures were tests, which is
precisely what `test_behavior: after_all` produces. That restriction existed
because a successful job dispatched its whole deploy-time write set, so a
test-only retry would have woken every consumer for relations no one touched —
the cascade now dispatches what the run reports materializing, so it wakes
nobody and the restriction only blocked a legitimate retry.
* fix(dbt): gate run progress behind the job-read check, not RLS alone
The endpoint joined `v2_job` so RLS would decide visibility, which it does — but
`require_job_read_access` adds two things RLS does not: a scoped token's
`if_jobs:filter_tags` restriction, and the app-embed cutoff that stops untrusted
app JS from inheriting the viewer's broader job access. A scoped or embed token
could therefore read relation names, statuses, row counts and errors for jobs
the ordinary job endpoints deny it.
That helper is private to `windmill-api`, which depends on `windmill-api-assets`
rather than the reverse, so the endpoint moves to the job routes instead of the
check being duplicated. It is job-scoped anyway:
`/w/{ws}/assets/run_progress/{job_id}` becomes
`/w/{ws}/jobs/run_progress/{id}`, and the frontend follows the generated client.
* feat(dbt): a dbt run does not trigger downstream runs
dbt orders its own DAG, so a cascade only ever adds one thing: waking a Windmill
script that reads a mart. That edge is narrow, and only half of it can even be
expressed — nothing outside dbt can declare a `table://` write, since
`// materialize` accepts DuckLake targets only, so an ingestion script cannot
wake a dbt project.
Against that, dispatching correctly is not cheap. A run's `select` can build any
subset of the project, so the deploy-time write set is not what ran; using it
wakes consumers of relations the run never touched, and narrowing it needs a
per-job record of what was built. The per-relation state table cannot supply one
(it keeps a single row per relation stamped with the last writer), and the
result field added for it made a run's own output carry the cascade's bookkeeping.
So `asset_dispatch` returns early for `ScriptLang::Dbt`, before the producer
gate. dbt still materializes, records per-model state and publishes its graph:
models, `ref()` lineage and live run progress are unchanged, and a
`# on table://<mart>` reader still renders beside the model it reads. It simply
does not fire. Wiring it up later means deciding what a selective run should
notify, which is the actual work.
Verified: a full run of a 6-model project succeeds and starts nothing, where it
previously triggered its subscriber; the run page still reports all 6 relations
and the folder graph still carries 14 tables and 8 ref() edges.
* fix(dbt): remove the cascade surface, settle stranded models, fix nested __mod
Stopping dispatch left its surface behind. `table://` was still an auto-trigger
kind, `persist_ingest` still derived subscriptions from a manifest's reads, and
the deploy still accepted `# on table://` — so the canvas drew cascade arrows
into scripts nothing could wake. All three are gone: the kind no longer derives,
the ingest only deletes rows earlier versions wrote, and the deploy refuses the
annotation with a message saying why rather than persisting a silent no-op.
`DescriptorTriggers` went with them; every field it parsed was cascade config.
A model marked `running` by the live tailer was never settled when the run did
not finish: reconciliation only revisits nodes `run_results.json` names, and a
cancelled or timed-out run has none for the model in flight, so the finished job
showed a relation building forever. It is now settled on every exit path.
Verified by cancelling a run mid-flight: 3 models `running` before, 3 `failed`
after, none stranded.
`getScriptBasePathFromModulePath` took the first matching suffix rather than the
outermost boundary, so `proj__dbt/models/legacy__mod/a.sql` resolved to
`proj__dbt/models/legacy`. dbt owns its directory names verbatim, so a folder
ending `__mod` is legal inside a project, and a module-only sync would have
looked for a descriptor that is not there and skipped the deploy.
* fix(dbt): colour a finished run's models from its own result
`materialized_partition` keeps one row per relation stamped with whichever job
wrote it last, so reopening a run showed only the models no later run had
touched since — down to none for an old run, which reads as a broken page rather
than as stale data. Reproduced: a 6-model run reported 6 relations, then a second
run rebuilt one shared model and the first reported 5.
A finished run already carries the answer. Its result lists every node with a
status, and the graph carries each asset's dbt `unique_id`, so the two join
directly — no path derivation, nothing stored twice, and nothing a later run can
overwrite. The endpoint stays for the live window, where the result does not
exist yet, and as the fallback for a run that never produced one (cancelled or
killed, whose relations the worker settles in the table instead).
`relationOutcome` mirrors the worker's `classify_status` so the colour drawn over
a record agrees with the record: `warn`, `skipped` and `no-op` leave the relation
untouched and stay uncoloured, as do tests and analyses, which match no asset.
Verified in the browser on the run whose model had been stolen: all six
relations green again, both sources correctly uncoloured.
* docs(dbt): record why only dbt-core 1.x has live per-model progress
`emits_node_events()` reads as "the Rust engines produce no node events", which
is false and would close off the option. They produce exactly the same events;
they put them on the console and ignore `--log-format-file json`, which both
accept. Measured on 2.0.0-alpha.5 and fusion 2.0.0-preview.202: 15 node events
each on stdout, 0 in the file log, for a three-model project.
Taking them means owning the job log's presentation to work around a flag that
is documented and simply unimplemented, so the note records the measurement, the
sample event, and that flipping the predicate is the whole change once either
engine honours it.
* fix(dbt): give HighlightCode a dialect-agnostic sql language
`npm run check` had three errors the fast check does not reach: `"sql"` is not a
value `HighlightCode` accepts. Every SQL dialect it knows maps to one grammar,
but a dbt model is compiled by whichever adapter the project targets, so naming
a dialect would be a guess — `sql` is now a value in its own right.
`langOf` was typed `string` and returned `markdown`, `python` and `text`, none
of which the component accepts either, so a dbt project's YAML and Python files
rendered unhighlighted. It now returns the component's own prop type, which is
what caught them, and `undefined` for what has no grammar rather than a name
that silently means the same thing.
Verified in the project panel: SQL 22 tokens, YAML 27, where YAML was plain.
* fix(dbt): stop failing no-op models, drop table triggers client-side, keep cross-selection edges
The sweep that settles a run's stranded relations was marking `no-op`, `warn`
and `skipped` models FAILED on successful runs: reconciliation reports those
nodes without settling their record, so they were indistinguishable from a model
the run never reached. It now excludes every relation the run accounted for, so
only the genuinely abandoned ones are settled.
`table` was removed from the backend's auto-trigger kinds but left in both
client mirrors, so the editor and `pipeline dev`/`docs` kept drawing cascade
arrows the deploy will not create.
`isModuleEntryPoint` scanned for the first `__mod/`, the same bug its sibling
just had: a `legacy__mod/script.ts` nested in a dbt project — dbt owns those
names verbatim — read as that script's entry point. Both now anchor on the
outermost boundary.
A script selecting a model whose parent another script builds dropped the parent
entirely, so no `dbt_edge` could reach it and the two relations sat on the graph
unconnected. The parent is now kept as an endpoint and recorded as a READ, since
this script does not build it — splitting a project across selections only
composes if the seam still draws.
* fix(dbt): don't double-run after-all tests, count only models a script builds
An `after_all` run whose test phase failed saves a `run_results.json` holding
tests alone. Retrying it reran exactly those tests — and then the test phase ran
the whole suite again, appending a second copy of every result: duplicate ids in
the run table, doubled totals. A retry whose saved results are tests alone IS
the test phase, so the suite is not run after it, and the two phases now merge
by node id rather than concatenating.
Keeping a selection's unselected parents as nodes made them count toward the
`×N` badge, whose tooltip says "materializes N models" — a script selecting one
mart claimed the staging models upstream of it, and the number grew with the
seam. The count now comes from the relations the script writes.
That change also made the cross-selection read block dead, with a comment
asserting the inverse of what now happens; it is removed, and the test that
covered it still passes on the new arm. The test I added landed between a
neighbouring test's comment and its `#[test]`, orphaning the attribute so that
test stopped running.
Two display fixes: the run page no longer shows a relation's SQL when the
provenance belongs to another project that materializes the same relation, and
the editor no longer draws an explicit `# on table://` arrow the deploy refuses.
The starter descriptor no longer promises the removed cascade.
* fix(dbt): clear untouched models, reject unknown descriptor fields
A `no-op` model was left `running` forever on a successful run. The previous
attempt at this stopped the sweep marking such models FAILED but gave them no
terminal state instead, so they simply never settled. Reconciliation now returns
what it settled and what the run reported but did not build, and the two get
opposite treatment: a relation the run left untouched has its row DELETED, which
is what the finished run's own result says about it (`relationOutcome` colours a
`no-op` nothing), so the live and settled views agree; only a relation the run
never reached at all is failed.
The descriptor accepted unknown fields, so `selcet:` was ignored and left an
empty selection — building the whole project — and a misspelled `target` fell
back to the profile's default. It rejects them now. That immediately caught two
of our own test fixtures still passing `repo:`, a field removed with the git
path, which is exactly the class of mistake it exists to stop.
`isDbtModulePath` matched `__dbt/` anywhere in a path, the third site with that
bug: `foo__mod/vendor/x__dbt/a.ts` read as a dbt project file, and the push then
looked for `foo.script.yaml` and could skip the edit.
A verbatim dbt bundle dropped any file named `*.lock` before it reached the
module map, so an authored `uv.lock` never deployed and the unmodified-project
round trip quietly lost it. The exclusion now applies only to `__mod` bundles,
where `.lock` really is the script's own lockfile — in the walker that hashes
modules too, or a change to such a file would not register as one.
Also: `langOf` fell back to `undefined`, which HighlightCode resolves to
TypeScript rather than to no highlighting, so seeds and Markdown were coloured
as code; and four comments still gave the removed cascade as the reason for
sharing an asset node, which is now lineage.
* fix(dbt): retry failed tests too, anchor the last __dbt path check
`retry_failed_nodes` only ran after the model phase, which fails before the
`after_all` test phase exists — so a project whose models built and whose tests
failed got no retry at all, exempting exactly the failure mode that separate
phase produces. The loop is now a function, called after both phases.
`isDbtGeneratedPath` matched `__dbt/` anywhere, the fourth site with that bug:
`foo__mod/vendor/x__dbt/target/a.ts` counted as generated dbt output, so
`ignoreF` excluded an ordinary module file and a module-only edit never deployed
its parent script.
`wmill sync push` still dropped an ADDED or DELETED `.lock` three branches
before the module arm, so the earlier fix only covered a first push: adding a
`uv.lock` to a deployed project was reported as a change forever and never
applied, and deleting one left it deployed. Editing worked, which is what made
the round trip look whole.
Also removes a duplicate `#[test]` that was double-registering a test and
detaching its neighbour's comment, and rewrites seven comments that still gave
the cascade as the reason for behaviour that now serves lineage only.
* fix(dbt): bound the excluded-file read, keep the retry budget job-wide
`isBundledModuleFile` read a file in full before deciding it was too big or
binary, so a project sitting next to a multi-gigabyte parquet seed loaded the
whole thing only to reject it. It now takes the size from `stat` and reads at
most the 8 KB the NUL check needs: a 191 MB file is rejected in 0.0ms at 82 MB
RSS.
Calling the retry helper after both phases gave each its own `attempts` budget,
so a job could spend double what the descriptor asked for — the bound exists
because every attempt is a real dbt invocation holding a worker slot. The budget
is now the job's, spent across whichever phases fail, and the field says so.
Extracting that helper had also placed it between `#[allow(clippy::
too_many_arguments)]` and `run_dbt`, taking the attribute off the 12-argument
function it was written for.
* fix(dbt): actually spend the retry budget
`retry_failed_nodes` looped on `while *remaining > 0` and never decremented it,
so a failing job reissued `dbt retry` — logging "attempt 1 of 3" each time —
until the job's deadline instead of `attempts` times. The decrement existed
briefly and was lost when the function was re-extracted by hand.
Claiming and counting are now one operation, `claim_attempt`, because keeping
them apart is exactly how the bound goes missing: the loop cannot iterate
without spending the budget.
Its test is bounded by its own `for` rather than by the function under test. An
earlier version collected `std::iter::from_fn(|| claim_attempt(..))`, which
against a non-spending `claim_attempt` is an infinite iterator — it allocated
until the machine died. A test for a loop bound must fail an assertion when the
bound regresses, not consume the host: it now reports `[1, 1, 1, …]` against
`[1, 2, 3]` in 0.00s.
* fix(dbt): ask before reading, not after
Bounding `isBundledModuleFile` did nothing for the bundle builder, which read
the whole file into memory and only then asked whether to keep it — so a
multi-gigabyte seed beside a project was still loaded in full just to be
skipped. The predicate is now consulted first, and the read happens only for
files the bundle actually carries.
* feat(dbt): animate the ref() edges feeding the model being built
The nodes moved during a run but the edges did not, so the graph showed where
dbt had got to without showing it flowing there.
Reuses the canvas's existing rule rather than adding a second one: an edge
animates when it touches what is happening. For a pipeline that is the running
script; for dbt the unit of work is the model, so a `ref()` edge animates while
its target builds. Same `animated` field, same visual language, no new styling.
Verified mid-run on a 7-model project: of six `ref()` edges only the two feeding
the model then building were animated, and none once the job finished.
* feat(dbt): show what each model wrote, and say when its SQL is another project's
Three things a reader wanted from the run graph and could not get.
Row counts: the worker already records one per relation and `run_progress`
already returned it, but the graph used only `status` and dropped the number. A
model that built green having emitted zero rows is the failure that looks like a
success, so the count is on the node.
The relation's fully-qualified name, copyable: there is no table browser to open,
so the next best affordance is the exact identifier to paste into a SQL client.
It is parsed with `splitRelation`, which honours quoting the way the worker's
`split_relation` does — splitting on every period renders
`"wh"."analytics.v2"."orders"` as a relation `orders` in a schema `v2`, which
does not exist.
And when two projects materialize one relation, the graph keeps a single
provenance winner, so the losing project's node carries the other's model. The
SQL was already suppressed there — correctly, it is not this run's code — but
silently, which reads as a dead click. It now says so.
* fix(dbt): a finished run's graph is the models it built, not today's project
`/assets/graph` is the current deploy, so an old run's graph drifted with the
project: a model added after it appeared as though the run had built it, and the
older the run the wronger the picture. A finished run's node set now comes from
its own result, which named exactly what it touched.
Sources survive the filter regardless — dbt never lists them in
`run_results.json` because it does not build them, but they are the upstream the
run read, and dropping them would leave the models hanging.
The graph is still the current deploy's, so a model renamed or deleted since
cannot be drawn at all. Rather than a silently shorter graph, the count is
stated above it.
Verified by adding a model after a run: the old run renders 7 models without it,
a fresh run renders 8 with it.
* feat(dbt): preview a model's rows with `dbt show`
There was no way to see the data behind a node — only its SQL and its row count.
`dbt show` selects from a model and returns rows, and every engine ships it, so
the preview needs no adapter code of ours: no connection path, no dialect-correct
quoting, no type coercion for ten warehouses. It runs against the profile the
run already renders.
It is a `dbt_command` rather than a new endpoint, so it inherits the whole job
path — authorization, isolation, cancellation, logs, engine provisioning — and
`limit` joins the run form beside it. That the allowlist can admit it at all is a
consequence of dropping the cascade: while a successful job dispatched its
deploy-time write set, a command that wrote nothing woke every consumer for
relations nothing had touched.
Read-only, and treated as such: no graph republish, no materialization records,
no retry state, no test phase. Captured rather than streamed, like `dbt ls` —
these rows are the result, not commentary, and the job-log writer is what
`NO_LOGS_AT_ALL` discards.
Verified: `{"dbt_command":"show","select":["stg_customers"],"limit":3}` returns
three rows; a preview leaves `materialized_partition` untouched (62 → 62, 0 rows
for the job); `clean` is still refused by the allowlist.
* feat(dbt): preview a model's rows from the graph, and keep our locks out of dbt projects
The run page could show a model's SQL and how many rows it wrote, but not the
data. Selecting a model now offers "Preview rows", which runs the script with
`dbt_command: show` and renders the result as a table.
Explicit rather than on-select: a preview is a job, so it costs a worker slot
and the engine's start-up, and previewing on every click would spend both on
mere navigation. Sources are excluded — dbt shows what a model SELECTs, and a
source is not one.
Also: `updateModuleLocks` was the one module helper that never learned about
verbatim bundles, so it walked a dbt project writing `foo.lock` beside `foo.sql`.
None of those files is a Windmill script needing a lockfile, and the bundle
promises to round-trip the project byte-for-byte — our artifacts have no business
in it.
Verified in the browser: selecting `stg_customers` and previewing returns the
columns `id`/`src` and five rows from the warehouse.
* fix(dbt): keep a run's models when another project owns their provenance
Scoping a finished run's graph to the ids it named dropped relations whose
provenance winner belongs to a different project — so a run of a project sharing
a schema showed 3 of the 6 models it had built. An id that was never this run's
package cannot be judged against its result, so it is kept: the relation IS one
the run wrote, and hiding it understates the run. The same rule applies to the
"no longer in the project" count, which otherwise reported deletions that were
only provenance collisions.
Previews are now cached per model and survive the selection moving. One was
thrown away whenever the reader clicked elsewhere, which for a job costing a
worker slot and an engine start-up meant re-running it to see it again — and the
run continues in the background, so leaving and returning finds the rows there.
The spinner also never span: `startIcon` takes the icon and its classes
separately, so the animation has to be passed alongside.
How long it took is shown with the rows. A preview is a job, and its cost should
not be something the reader has to guess at.
* fix(dbt): resolve argument references, clamp the show limit, flag renamed relations
`handle_dbt_job` cloned `job.args` where every other executor calls
`build_args_map`, so a `$var:` / `$res:` / `$encrypted:` argument reached dbt as
the literal string. A placeholder holding a schema or an `enabled` flag would
then build a different slice of the project than the caller asked for.
`--limit` took any positive i64, and the worker buffers the whole of dbt's
stdout to read the rows out of it — so a caller with only run permission could
make it hold an unbounded allocation. It is clamped to a ceiling now, extracted
as `show_limit` so the bound is pinned by a test rather than inline in an async
function nothing can reach.
And a model keeps its id when its alias or schema changes, so an old run's node
showed today's relation while the run wrote another — the page asserting it had
materialized a table that did not exist yet. The run's result carries the
relation each node actually wrote, so the drift is detectable without a graph
snapshot, and the count is stated above the graph. Rendering the run's own
lineage still needs a per-job snapshot; this stops the page claiming otherwise.
* fix(dbt): stop persisting resolved secrets, bound the preview by bytes
Resolving `$var:` / `$res:` / `$encrypted:` for dbt — added in the previous
commit — meant `save_run_state` wrote the resolved PLAINTEXT into
`dbt_run_state.args` and the worker's `state.json`. The row outlives the job, so
a secret stayed in the database and a later `dbt_command: retry` replayed it
after the grant was revoked or the value rotated. The invocation now carries the
args as submitted alongside the resolved ones, run state persists those, and the
restore path resolves them again under whoever is retrying.
Clamping `--limit` bounded the row COUNT, not the size: one column can hold a
megabyte, so a thousand rows is a thousand megabytes, and `run_capturing`
buffers all of it. The captured output has a byte ceiling now.
`limit` became a built-in argument without joining `RESERVED_ARG_NAMES`, so a
descriptor writing `{{ limit }}` was silently handed the preview control's
default instead of being told the name is taken.
Two display fixes: the relation-drift banner compared a canonicalized (lower
case) asset path against the warehouse's own spelling, so it fired on every
model of every finished Snowflake run; and caching a preview's failure left
`Preview rows` dead for that model until reload.
* feat(dbt): key the graph by script version so a run renders its own project
The dbt graph was keyed by path alone, so a deploy overwrote the only copy and a
run page could only ever show today's project — an older run rendered today's
models, SQL and `ref()` lineage no matter what it had run. My previous attempt
filtered that view to the ids the run named, which stopped it lying but could not
show what was gone: the data no longer existed.
`dbt_node` / `dbt_edge` now carry `script_hash` in their primary key, so each
deployed version keeps its own graph, and the run page passes the version its job
recorded. Per DEPLOY, not per run — ten thousand runs of one version share one
graph — and a composite FK to `script (workspace_id, hash)` with ON DELETE
CASCADE means a version's graph dies with it. Nothing pruned these before,
because there was one copy per path; they would otherwise have accumulated with
no sweep.
Two deploys of one path now write disjoint rows, so the graph can no longer be
lost to a race. `claim_graph_publication` remains only for what is still
path-keyed — the `asset` usage rows — and an older deploy finishing late records
its own graph before declining to touch those, where before it published nothing
at all.
A pinned request is scoped by the version's own nodes rather than by `asset`:
that table describes the current deploy, so scoping through it would filter a
model out of the very run that built it.
Verified end to end: deployed v1 (8 models), ran it, deployed v2 with four models
removed and one rewritten. The old run renders 8 models, 6 ref() edges and v1's
SQL; a new run renders 4 and the v2 rewrite.
* fix(dbt): scope graph cleanup to one version, bound the preview capture
Archive and delete both act on a single `hash`, but the graph cleanup they
called deleted every row for the path. Now that the graph is keyed per
version, archiving an old version erased the live one's models, SQL and
lineage, and nothing repaired it. Both callers have the path in hand, so the
by-hash wrapper is gone and they use the version-scoped clear directly.
`dbt show` checked its 8 MB ceiling after `wait_with_output` had already
buffered everything, so the ceiling could not bound what the worker held.
`run_capturing` now reads both pipes incrementally against a caller-supplied
limit and kills the child on overflow. The read buffers are heap-allocated:
as arrays they were baked into the future, which the job poller boxes several
layers deep, and that overflowed the worker thread's stack — a `dbt show` run
aborted the whole worker process.
A retry's `dbt parse` ran on the arguments as submitted while the build ran on
resolved ones, so a `$var:` shaping the graph parsed verbatim. The parse moves
to the caller, after resolution.
A run that names its own `select`/`exclude` now drops the descriptor's
`selector`: dbt resolves `--selector` instead of `--select`, so passing both
made a preview of one model return another's rows.
Also: log instead of silently swallowing a `modules` column that fails to
deserialize (pre-existing, but for dbt it means running with no project at
all); keep the model SQL reachable once a preview has landed; render which
node the rows came from; stringify object-valued cells; document
`dbt_script_hash` in the OpenAPI spec.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
* docs: record the per-worktree dev environment and the backend-run check
Three mistakes this guidance would have prevented, each of which cost a cycle:
A worktree has its own database and ports, but AGENTS.md stated the
single-checkout defaults as facts. Pointing `DATABASE_URL` at another
worktree's database makes `cargo sqlx prepare` fail on every query touching a
table your migrations added — and it deletes `.sqlx/` before it fails, so the
cache is gutted rather than merely stale. Starting a backend on the wrong port
leaves the UI up with every call 502ing, which reads as an application bug.
Both values are now discoverable with commands that work as written.
`prepare` is also documented as the wrong tool for a removal-only change: the
cache is already complete for CI, and the only residue is orphaned entries that
can be found by text-matching against the sources without a database.
Nothing told a reader that `cargo check` does not exercise a worker path. A
read buffer declared as an array inside an async block is baked into the
future, and once boxed by the job poller it overflows the worker thread's
stack — compiling and unit-testing clean while aborting the whole worker
process at runtime.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
* fix(dbt): scope the remaining path-wide reads and clears to one version
Four places still spoke for a whole path after the graph became per-version:
The relation-root drift check read `dbt_node` by path with an unordered
`LIMIT 1`, so with v1 at root A and v2 at root B it could answer with v1's
row, suppress the refresh v2 needed, and leave v2's graph naming relations the
run does not build. It now reads this job's version.
`dbt_dep`'s no-resource branch cleared the path, so a descriptor edited to
bring its own `profiles.yml` emptied every earlier version's graph and with it
every finished run's page. The ownership being given up is the path-keyed
`asset` usages cleared beside it; the graph clear is now this version's.
The graph was inserted before the publication claim checked the version was
still live. Archive and delete only soft-update `script`, so the foreign key
still accepted an in-flight dependency job's rows and the failed claim
committed them — and because pinned queries deliberately serve archived
versions, deleted model SQL became readable again. The write is now gated on a
`FOR UPDATE` liveness check.
`clear_dbt_run_state_by_script_hash` resolved a hash to a path and deleted the
path's saved run. `dbt_run_state` is keyed by path by design — one saved run
per script — so archiving one version discarded the live version's resumable
failure. It clears only once no live version of the path is left; `identity`
already refuses a resume whose project, warehouse or engine moved.
"Preview rows" ran `runScriptByPath` while the SQL beside it was pinned to a
hash, so an old run showed its own SQL over today's rows. Verified end to end:
with v3 deployed, the v2 run's preview runs v2's hash and returns v2's rows.
Also: keep the TAIL of a captured stderr, since dbt prints its summary last;
one `$derived` for the parsed result rather than five; collapse three
near-identical argument accessors onto one generic; fold the single-use
`copy_dir_command` into its caller; and give `parseDbtRun.ts` one status
classifier instead of spelling dbt's failure vocabulary twice.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
* refactor(dbt): one JobCtx down the executor, one table per adapter
Two changes aimed at the operations this code will keep having: adding a
phase, and adding a warehouse.
`JobCtx` already bundled the five values every phase needs, and nine functions
took it — but the top of the executor threaded the fields apart and rebuilt the
struct at each call, so the same literal appeared eight times and each new
phase meant five more parameters. It is now built once per entry point and
reborrowed. `prepare_project` goes from 21 parameters to 17, `retry_failed_nodes`
from 15 to 11, and `run_dbt` drops below the lint threshold. The two remaining
constructions are the worker boundary, where the pieces genuinely arrive apart.
`DbtAdapter` answered five questions with five parallel matches over the same
eleven variants, plus a sixth list of adapters kept by hand in a test. The
facts now live in one `AdapterSpec` per adapter, reached through one exhaustive
match, so adding a warehouse states its name, driver, package, port, database
key and licensing together and the compiler demands the arm. Each arm spreads
from a Postgres base, which makes the inheritance visible per adapter instead
of hidden in the `_ =>` defaults `default_port` and `database_key` used to
carry. `DbtAdapter::ALL` replaces the list the test kept separately.
Verified by dumping all seven facts for all eleven adapters before and after:
byte-identical.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
* fix(dbt): job-keyed run progress, and stop path-wide reads and clears
Six findings from the last round, in the order they bite.
The publication liveness gate refused on `archived`, but `create_script`
archives the parent on every redeploy — so deploying v2 while v1's dependency
job was still parsing left v1 without a graph, permanently, which is the exact
case the unconditional write existed to serve. It gates on `deleted` alone now;
an explicit archive is still covered by the `FOR UPDATE` ordering.
A project-owned `profiles.yml` trusted `profile.type` instead of reading the
file. The Rust engines carry every adapter, so a CE script could declare
`postgres` over a target that is `sqlserver` and have dbt connect with the
enterprise adapter. The file is read whichever way, and a descriptor that
disagrees with it is refused.
Renaming a dbt script, or editing one so its newest version is no longer dbt,
cleared the graph for the whole path — every older version's models, SQL and
lineage, which their own finished runs still render. Neither needs it: graph
queries join on `(path, hash)` through a `language = 'dbt'` CTE, so an old
version's rows cannot attach to whatever lives at that path next.
Live progress read `materialized_partition`, whose key is the relation and
whose `job_id` is only the last writer. Two runs of one project took rows from
each other. Progress now has its own job-keyed table; the relation table is
untouched, because one row per relation is right for the pipeline canvas and
fork defer. Verified with two overlapping builds: both keep 6 rows in the new
table, while the old one attributes 6 to one run and 0 to the other.
`Scratch::drop` removed a half-installed virtualenv synchronously from inside
the job future, blocking a runtime thread; it goes to `spawn_blocking`, with a
direct call when there is no runtime to hand it to.
The E2E list asked for a `# on table://` subscription the deploy now refuses.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
* feat(dbt): snapshot a dynamic descriptor's graph per run
A `{{ }}` placeholder in `vars` can enable a different set of models per run, so
those runs re-ingest the graph. Keyed by version alone, each re-ingest
overwrote the last: reopening an older run showed the newer run's project, and
a model only the older run built was gone entirely — no SQL, no lineage, and
nothing the saved result could colour, since it can only tint nodes that are
there.
`dbt_node` / `dbt_edge` gain `job_id`. A run of a dynamic descriptor writes its
own snapshot under its job id and its page reads it back; a static descriptor
writes the version's graph once, under a zero-UUID sentinel, and every run of it
reads that. The sentinel is a value rather than NULL because `job_id` is part of
the primary key and Postgres does not treat two NULLs as the same key, so each
re-ingest would add a row set instead of replacing one.
`/assets/graph` takes `dbt_job_id` and prefers a snapshot when one exists,
falling back to the version's graph otherwise — so a run page passes it
unconditionally and static descriptors are unaffected. Snapshots age out after
30 days, pruned by the runs that write them, so no background sweep has to learn
about these tables.
Verified end to end: one deploy, two runs of it with `extra=yes` and `extra=no`
gating a model's `enabled`. The version's graph holds 6 models, run 1's snapshot
7 including `opt_extra`, run 2's 6 without it; the endpoint returns each run's
own and falls back to the version's when the parameter is omitted.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
* perf(dbt): only snapshot a run whose graph differs, and prune from every run
Two costs the per-run snapshot carried, both found by measuring it rather than
by reading it.
A snapshot was written for every run of a dynamic descriptor, but marking one
dynamic is conservative: `graph_is_per_run` is true whenever `vars` holds a
`{{ }}` placeholder or `env` holds a `$var:`, which says the arguments reach dbt
and not that they change which models exist. The usual case is a date var, whose
graph is identical run after run, so the table filled with copies of an
unchanging picture — around 1 KB per model per run, which is a gigabyte or so a
month for a 200-model project on an hourly schedule. A row set now carries a
digest of its nodes, edges and relation root, and a run whose digest matches the
version's writes nothing; the read already falls back to the version's graph, so
those pages are unchanged. Only a run whose model set really differs pays.
The prune was hung off the progress reporter, which exists only for engines that
emit node events — so a Fusion or dbt-core-2x instance accumulated snapshots and
never deleted any. Retention that stops working because of an engine choice is
not retention; it runs detached from every dbt run instead.
Verified against a descriptor with a var-gated model: the version's graph holds
8 rows, a run that resolves to that same graph stores none at all, and a run
that enables the extra model stores its own 9. Both pages still render their own
project — 6 assets without the extra model, 7 with it.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
* fix(dbt): scope every dbt_node join to the chosen snapshot
`job_id` joined the key, but only the scoping CTE and `dbt_edge` were taught to
filter on it. The outer node SELECT and the parent/child joins in the edge query
were not, so each model came back once per retained snapshot plus once for the
version's graph, and each edge matched every combination of the two — the model
count multiplied and the edge join fanned out quadratically. Measured against
one stored snapshot: 17 node rows where 8 are wanted, and 28 edge pairs where 7
are. The response dedup hid the edge blow-up from the payload, not from the
plan, and the run page refetches the graph every two seconds.
The progress table gained writers it was missing. `terminalize_running_relations`
settled only the relation-keyed table, so a cancelled or killed run — the case
that function exists for, since it leaves no `run_results.json` — showed every
in-flight model still spinning on the run page for as long as the row lived. An
agent worker cannot write the new table at all, having no database of its own,
so the read falls back to the relation-keyed one when a job has no rows there.
Also: a wrapped string literal missing its backslash put eighteen spaces in the
middle of the profile-disagreement error; a comment still described concurrent
runs of one dynamic version overwriting each other's graph, which is what
keying by job removed; and the `materialized_partition` index justified itself
by a run-page poll that has since moved to another table, though the closing
sweep still earns it.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
* fix(dbt): give a graph snapshot a marker row, and scope what reads it
Five findings, four of which are the same mistake in different places: a
snapshot's identity was inferred from its contents.
Existence was inferred from a `dbt_node` row, so a dynamic run that disabled
every model — a legitimately empty graph — read as "no snapshot" and its page
showed the deployed models instead. The digest was a column repeated on every
node and read back with a `LIMIT 1` carrying no `job_id`, so a run could compare
itself against another run's digest and suppress a snapshot it needed. The
relation-root drift check read the same rows unscoped, so after a drift it could
find a previous run's root and conclude nothing had moved.
`dbt_graph_snapshot` holds one row per stored graph — path, version, job,
digest, timestamp. Existence is that row, the digest lives there once, the drift
check reads the deployed row explicitly, and the retention sweep deletes markers
first and then the rows no marker stands for. The digest is SHA-256 rather than
`DefaultHasher`, whose output is documented as unstable across Rust releases:
this value outlives the process that computed it, so a toolchain bump would have
silently stopped every comparison matching and quietly reinstated the duplicate
snapshots the digest exists to prevent.
`/run_progress` ignored the view token, so a share-link viewer got the graph and
was refused the progress that colours it.
A preview sent only its own three arguments, so a descriptor with a required
`{{ }}` var could not be previewed at all and an overridden one previewed a
different relation than the page was showing. The run's arguments go first now,
with the preview's three overriding.
Verified on a project whose only model is var-gated: the deploy stores a marker
with zero nodes, a run with the var set stores a marker with one, and the
endpoint answers 0 and 1 respectively rather than showing the deployed models
for both.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
* refactor(dbt): squash the runtime's migrations into one
Ten migrations reshaping the same three tables is a history no installation
ever had. `dbt_node` gained `script_hash`, then `job_id`, then `ingested_at`,
with its primary key rebuilt twice; `graph_digest` was added by one migration
and dropped by the next after the digest moved to its own table. On a fresh
database all of that replays to arrive at a shape the schema can simply state,
and this feature has never shipped, so there is no upgrade path to preserve.
One migration now creates `dbt_node`, `dbt_edge`, `dbt_graph_snapshot`,
`dbt_run_state` and `dbt_run_progress` in their final shape, carrying forward
the rationale each of the replaced migrations recorded. The enum additions stay
in `add_dbt_lang`, since a value cannot be added and used in one transaction,
and the `materialized_partition` index stays separate because it belongs to a
table this feature did not introduce.
Verified by rebuilding: dropped the five tables, replayed from the single
migration, and confirmed the result is identical — same primary keys, the same
two composite `script` foreign keys, the same seven indexes. Every `sqlx::query!`
in the workspace then compiled against it, which checks each column's name, type
and nullability, and a deploy plus run on the rebuilt schema produced 8 nodes,
7 edges, a snapshot marker and 6 progress rows.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
* fix(dbt): authorize snapshot reads, bound the prune, give the marker a lifecycle
`dbt_job_id` is caller-supplied and selected straight from `dbt_graph_snapshot`,
which carries no RLS — so a caller who could see the script could read any run's
model set and relation paths, which a dynamic alias or schema can encode. Both
graph queries now require the job itself to be visible, in the authed
transaction, the same gate `raw_code` already applies to the script that
produced it.
The drift check compared against the deployed graph alone, which misses the way
back: a run at root B republishes the path-keyed `asset` usages at B, and
returning the profile to A then matches the deploy and skips the refresh,
leaving those usages at B while dbt builds A. It reads the most recent ingest
for the version instead — the one that last wrote them — ordered rather than an
arbitrary `LIMIT 1`.
The prune anti-joined every non-deployed node and edge with no age predicate, so
each run scanned the whole retained sidecar and concurrent runs duplicated it.
All three deletes share one age bound again, with the sentinel spelled as a
literal so the partial indexes apply — a bound parameter cannot be proven to
match the index predicate.
`dbt_graph_snapshot` was the one dbt table nothing in the script lifecycle
deleted: no `script` foreign key and absent from both `clear_dbt_manifest*`
sites. A marker outliving its rows is read as a snapshot with no nodes, and its
digest still answers the suppression check, so an identical run would write
nothing and then render an empty graph. It cascades like the rows now and both
clears take it.
Also: the preview cleared `exclude` rather than inheriting it, since previewing
a model the run excluded reached dbt as `--select m --exclude m`; and
`terminalize_running_relations` no longer claims to cover a killed worker, which
never reaches it.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
* fix(dbt): quote profile names, record where usages were published, stop polling the graph
A profile name comes from the project's own `dbt_project.yml` and a target from
the descriptor, and both were interpolated into `profiles.yml` as bare YAML —
including as mapping keys. A name like `prod # hidden` truncates the mapping and
a newline opens a sibling key of the author's choosing. Both are rendered as
quoted scalars now, as are the BigQuery keyfile's keys, with a test that asserts
the document still parses to exactly the keys we wrote.
The drift check read the most recent ingest, which latches: a run that returns
to the deployed root re-ingests but stores no snapshot (its digest matches the
version's), so the moved run's rows stay newest and every later run pays an
extra parse and ingest. The publisher now records the root it published the
path-keyed usages at, which is the only thing that answers "where do the current
usages point" — the deploy's own root goes stale as soon as a run republishes.
The run page polled `/assets/graph` every two seconds alongside progress, so it
re-sent every node's SQL for the length of a run — hundreds of KB a tick on a
real project, for a graph that a dynamic descriptor re-ingests exactly once
before the build. It fetches once more shortly after mount and then polls
progress alone.
Node results carry `outcome` beside `status`. `status` stays dbt's own word, but
dbt owns that vocabulary — 1.x and 2.x differ on casing and `no-op` arrived in a
minor release — so publishing only it would force a break or a lie the first
time it moves. `outcome` is the stable half a downstream script branches on.
Also: the worker's dbt entry points are `pub(crate)`, since nothing outside the
crate calls them and they resolve secrets and launch processes; and the snapshot
gate records that it is RLS-only where `/jobs/run_progress` also honours a
share-link token, which is a gap in what a shared page shows rather than in what
it protects.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
* test(dbt): pin the graph storage invariants against a real database
Every defect review found in this area was DB-shaped — which row set a read
resolves to, which rows a clear takes, whether a snapshot exists at all — and
none of it is reachable from a unit test on a pure function. Four rounds
established these answers and nothing guarded them, which is why each round kept
finding another.
Six cases, on the harness the repo already uses for schema-shaped behaviour:
an identical run stores no snapshot and leaves no marker; a differing run keeps
its own while the version's is untouched; an empty run graph is still a snapshot
rather than an absent one; clearing one version leaves the others whole; the
path-wide clear takes the markers with it; and the sweep ages out run snapshots
while never touching a version's own graph.
`IngestedNode` gains `Default` so a test can state the two fields a case is
about rather than the eighteen it is not.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
* perf(dbt): bound a script's stored graphs by deploy count
Run snapshots expire on a clock, but a VERSION's graph could not: its reader is
every finished run of that version, and a run page is as old as its job. So
nothing reclaimed them — a deploy graph went only when its `script` row was hard
deleted, which Windmill does not routinely do. A CI deploying on every commit
added a full model set with SQL bodies per commit, forever: roughly 200 KB a
deploy for a 200-model project, which is gigabytes a year across an instance.
Bounded by COUNT instead of age, since age is the thing that cannot be right
here. The newest 50 deploys per path keep their graph and older ones are
reclaimed, making growth `versions x models` rather than unbounded in time.
Generous on purpose: reaching the bound empties that version's run pages, so it
exists to stop unbounded growth rather than to be hit in normal use. Ordered by
the script's own `created_at`, so a late-finishing job re-ingesting an old
version cannot promote it.
Pinned by a test that deploys past the bound and asserts both halves: the count
holds, and the newest version is always among the survivors.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
* fix(dbt): let the run page know when its snapshot has landed
The one-shot graph refetch was wrong: a dynamic descriptor's ingest happens
before the build but after cloning, dependency install and parse, so a fixed
delay either fires too early — and the run page then shows the deployed models
for the whole run, never that run's own — or keeps re-sending the whole graph
for the length of it. Neither is a timing problem to tune; the page had no way
to tell "the snapshot is not written yet" from "this run has none".
`/assets/graph` answers that directly: `dbt_snapshot_job` is the job the dbt half
resolved from, when one was asked for and found. The page polls the graph until
that is its own job, and stops. A static descriptor never snapshots, so an
attempt cap ends it there rather than polling for the run's duration.
`dbt_node.relation_root` is gone. The drift check moved to the marker's
`published_relation_root`, which left the column written on every node and read
by nothing.
`outcome` was published as the stable half of the result contract, but the
in-tree consumer still ranked and coloured from dbt's own word — so the field
existed and nothing used it. `statusRank` takes it, `DbtRunResult` passes it, and
`classifyStatus` is documented as the fallback for results that predate it and
for the live event stream, which carries dbt's word alone.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
* docs(dbt): record what a share-link viewer actually sees
The comment at the snapshot gate said the graph "falls back to the deployed
set", which is only the rarer half of it. A share link is an extra grant for a
logged-in user who lacks access to the job, so the usual case is no read on the
script either — and then the `live` CTE matches nothing and the whole dbt half
comes back empty. A blank Models panel over working progress rows, not a
fallback.
`docs/dbt-runtime.md` now carries the analysis a follow-up needs: that relaxing
this leaks nothing, because `v2_job_completed.result` already gives that viewer
every node's `unique_id` and `relation_name` — the graph's only incremental
exposure is `raw_code`, which is gated separately on seeing the script. And the
shape of the fix: `OptViewToken` and `validate_view_token` are self-contained
enough to move into `windmill-api-auth`, which `windmill-api-assets` already
depends on, after which the gate can honour a token for that job's snapshot
alone while `raw_code` stays where it is.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
* fix(dbt): keep model SQL behind the scripts:read scope, and unbreak CI
`/assets/graph` is authorized as `assets:read`, and RLS decides whether the
caller can see the script that produced a node — but RLS is not a scoped
token's grants. A token deliberately narrowed to `assets:read` could therefore
read model source and repository paths for scripts outside its `scripts:read`
paths. The same `build_scope_path_predicate` the macro endpoint already applies
now gates `raw_code` and `original_file_path`; the relation's shape is
unaffected, only its body is withheld.
`DbtAdapter::ALL` exists for the tests that must cover every adapter, so it is
dead in a release build and `-D warnings` failed all four backend checks on it.
It is `#[cfg(test)]` now.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
* fix(dbt): stop the graph poll at the ingest, and type `limit` in the schema
The poll's stop condition was a snapshot appearing, with a 40-attempt cap
behind it — so a STATIC descriptor, which never snapshots, took the cap every
time and re-fetched the whole graph forty times. That is most of what removing
the poll was meant to save, and static is the common case.
The ingest runs BEFORE the build, so the first model to report progress proves
it has already happened: a snapshot absent by then is one this run never
writes. Progress arriving is now the second exit, and the cap is only a
backstop for a run that reports none at all.
`limit` is declared `Typ::Int` but `dbt_arg_schema` had no integer arm, so the
run form and the generated clients saw an untyped default and offered no
numeric control for a value the worker clamps. Covered by the schema test.
`relationOutcome` still re-derived from dbt's word while `statusRank` had moved
to `outcome`; both read it now.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
* fix(dbt): make a retry prove its arguments still resolve the same
The saved arguments are the ones SUBMITTED, so a `$var:` in them is re-resolved
on retry. The identity did not cover the resolved values, so a variable that
changed between the failed run and the retry was accepted — and which graph the
retry then used depended on WHERE it landed: a worker holding the local
snapshot replays the saved manifest, while a database restore reparses with the
new value. Placement decided whether the resumed failures described the
relations being built.
The identity gains a digest of the resolved arguments, and is compared in two
halves because resolution happens between them. Project, warehouse, engine and
env are checkable up front; the arguments are not, because a retry request
carries only `dbt_command` and the ones to compare are the SAVED arguments after
this caller has re-resolved them. Comparing the whole string up front would have
refused every retry — which is what the obvious version of this fix does.
A row written before the digest existed has no last segment, and still restores
rather than being refused.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
* fix(dbt): keep pre-upgrade retries working, and stop losing a late snapshot
Splitting the identity on its last `|` read a pre-upgrade row's env digest as an
arguments digest and left only `<run_identity>` as the prefix, so every saved
failure on an upgraded instance became unretryable — a regression the previous
commit's own test missed by using an identity with no `|` in it at all, which is
not what an old one looks like. The digest is tagged (`|args=`) rather than
positional, and the test now uses a real pre-upgrade identity.
The graph poll gave up after a bounded number of tries, but provisioning and
`dbt deps` precede the ingest and can outlast that on a cold worker — and the
engines that emit no node events never produce the progress that ends it early.
A finished run now reloads the graph unconditionally, and the poll's own exit
issues one last load: progress proves the ingest happened, not that the previous
tick saw it, and dbt's compile window is wider than one tick.
A `dbt retry` restores the failed run's arguments inside the worker and they are
never written back to the retry job, whose own args are just
`{"dbt_command": "retry"}` — so previewing a row on a retry's page ran without
the vars the run used. The result now carries the invocation's arguments as
SUBMITTED, so a `$var:` stays a reference and no resolved value is published.
The deploy-count sweep ran instance-wide on every dbt run: `FROM script WHERE
language = 'dbt'` has no index to stand on, and both orphan deletes are the
complement of every partial index here. It is scoped to the running script's
`(workspace_id, path)` — which `index_script_on_path_created_at` serves — and
the orphan deletes only run when a marker actually went.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
* fix(dbt): hide dbt from module-less pickers, and stabilise the retry digests
`processLangs` feeds every language picker, including flow steps and app inline
scripts. Those are raw bodies with nowhere to carry a module bundle, and a dbt
script IS its bundle — so choosing dbt there produced a job that could only fail
once the worker looked for `dbt_project.yml`. Those two surfaces use
`processInlineLangs`, which drops the languages that need modules; a flow still
reaches dbt the way it reaches any script, by path to a deployed one.
`graph_digest` moved to SHA-256 because it is persisted and compared by a later
worker, and `DefaultHasher` is documented as unstable across Rust releases — but
the retry identity's own digests were left on it, and they are persisted in
`dbt_run_state.identity` for exactly the same comparison. A toolchain bump would
have refused every saved failure as a different project. All three go through
one `stable_digest`, length-prefixed so no split of the same bytes collides.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
* fix(dbt): enforce the tag scope on snapshot reads, reset state between runs
A tag scope is an orthogonal hard restriction: a token limited to some tags must
not read a job outside them however else it is authorized. The snapshot lookup
went through `v2_job` RLS alone, which knows nothing about tags, so such a token
could still retrieve a run's model set and its dynamic relation paths. The same
predicate `require_job_read_access` applies for the progress half of the page is
applied here — `get_scope_tags` is already public in `windmill-api-auth`, and it
is `None` for an unscoped caller, so a normal session pays nothing.
SvelteKit reuses the run graph between run ids, and `graphTries`, `polled` and
`raw` all describe the previous job: a spent retry count stopped the next run's
snapshot poll before it began, and stale progress coloured its models with
another run's statuses. All three reset when the graph key changes.
Also a wrapped string literal missing its backslashes, which put two ~22-space
runs in the middle of the retry-refusal message.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
* docs(dbt): put each digest helper's rationale on its own function
Inserting `stable_digest` above `split_identity` split that function's doc, so
five lines describing where the identity divides ended up introducing the
hasher. Each is back on the function it describes, stated once.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
* fix(dbt): read the run-pinned graph through the job, not the asset graph
Pinning the asset graph to one run is job-scoped data, but `dbt_job_id` sat on
`/assets/graph`, authorized as `assets:read`. The job-read contract —
`require_job_read_access` — is five parts that pull in opposite directions (tag
scope restrictive, `created_by` permissive, app-embed restrictive-overriding,
view token permissive, RLS underneath), so plain RLS is neither a stricter nor a
looser approximation of it. Restating the parts near the graph query kept leaving
one out: first the job check entirely, then the share-link asymmetry, then the
tag scope, and the app-embed restriction was still missing and failing open.
The helper cannot be called from `windmill-api-assets`, because `windmill-api`
depends on that crate. So move the read instead of the check: the run-pinned
graph is now `GET /w/{w_id}/jobs/dbt_graph/{id}` in `windmill-api`, on the same
gate as the `run_progress` it colours, and `/assets/graph` has no `dbt_job_id`
parameter at all.
- `asset_graph_for` takes the job as an argument from an already-authorized
caller; the route handler passes `None`.
- Extract the graph response into a `AssetGraph` component schema, now that two
paths return it.
- The run page fetches the job route when it has a job id.
* fix(dbt): charge assets:read on the run-graph route, trust the job gate in SQL
Round 13 findings on the route moved last commit.
The scope domain comes from the URL segment, so putting the read under `/jobs`
asked a scoped token for `jobs:read` alone while returning asset-graph data that
`/assets/graph` charges `assets:read` for. A token narrowed to polling run status
could read workspace topology, and the missing-job fallback made it cheaper still
— any random UUID skipped the job gate. Both scopes are now required: the job
gate reaches this run, `assets:read` reaches asset data at all.
The `chosen` CTE re-decided job visibility under plain RLS after the caller had
already passed `require_job_read_access`. It could only disagree, and did so
silently by falling back to the deployed graph — a share-link viewer entitled to
the run was shown a different run's model set. Dropped; the contract is that a
job reaching `asset_graph_for` is already authorized.
Also: the flow editor's `+` insert menu still offered dbt (the third
`processLangs` caller, missed when the other two moved to `processInlineLangs`),
the docs still described the deleted `dbt_job_id` parameter, and the new handler
had again been inserted between `get_run_progress`'s doc comment and its
function.
* fix(dbt): resolve a pinned run's version from the job row, not script RLS
Local codex review of the branch.
A share-link viewer is entitled to the run and usually has no grant on the
project — that is what the link works around. The graph's `live` CTE resolved the
version by selecting `script` inside the viewer's RLS transaction, so it answered
for their access to the project rather than for the run they were given: the
Models panel came back blank beneath working progress rows.
A pinned run now takes its path and hash from the job row the handler already
read after authorizing the job, so `live` does not consult `script` at all.
`raw_code` keeps its own `EXISTS` against `script`, so the model bodies stay
behind access to the project. Verified under RLS as an unprivileged role: the
shape query goes 0 rows -> 1, the `raw_code` gate stays 0.
Taking the version from the job also means a caller can no longer pin one
project's version while naming another's run, since `dbt_script_hash` is ignored
when a job is given.
The run page's graph fetch is a raw `fetch`, which bypasses the interceptor that
adds `X-View-Token` to generated-client calls, so a shared page was refused
before any of this mattered; it goes through `appendViewToken` now.
Also trims three comments to the AGENTS.md limit, dropping drafting-history
rationale that belongs in docs/dbt-runtime.md.
* fix(dbt): snapshot vars-overridden runs, carry the pinned version everywhere
Second local codex pass.
A `vars` run argument overrides the descriptor's, and vars drive `enabled`,
alias, schema, database and materialization — so such a run builds relations the
deployed graph does not describe. It now snapshots under its own job id, which
per-job keying makes safe: the version's graph stays for runs that did not
override. The old comment claimed gating on it would strand the override's graph
for the next default run, which was true only when the write went to the
deployed slot.
Two sites still read the caller's `dbt_script_hash` instead of the version
resolved from the job, so `/jobs/dbt_graph/{id}` without that redundant
parameter dropped models the run's version had and a later deploy removed.
The `dbt_snapshot_job` marker re-checked `v2_job` under RLS — the recheck the
graph query itself drops. A share-link viewer got the right graph and a null
marker, so the run page refetched it 40 times before giving up.
`wmill script preview` read the bundle with the generic `__mod` suffix and
script-module parsing, so previewing a `.dbt.yaml` omitted the project and failed
on the missing `dbt_project.yml`. It uses the same suffix and verbatim read as
deploy.
* fix(dbt): key retry state by principal, not by script path alone
`dbt_run_state` held one row per (workspace, script path), and a retry replaces
the caller's arguments with the saved ones. Anyone able to run the script could
therefore retry whoever ran it last, replaying that run's literal `select` and
`vars` against the warehouse and publishing them as their own job's
`invocation_args`. Running the script was already theirs to do; seeing another
principal's arguments was not.
`permissioned_as` joins the key, so a retry resumes only state written under the
same authority. Two runs sharing an authority can already act for each other, so
this is the boundary that matches the rest of the job model.
* test(dbt): pin what a caller without access to the project sees of its run
The share-link case had no regression guard, and every fix in this area touched
one of its two halves: the graph's SHAPE has to survive a caller who cannot read
the script, and the model SQL must not.
Two cases against a real database, calling `asset_graph_for` as a member with no
grant on the project's folder: pinned to a run, the models render and `raw_code`
is withheld; unpinned, the same caller sees nothing of it, so making the first
work did not relax the second.
Both assertions were checked by mutation — reverting the `live` bypass empties
the graph, and dropping the `raw_code` script gate leaks `select 1` — so neither
passes on the code it is meant to catch.
* fix(dbt): key the worker-local retry cache by principal too
Keying `dbt_run_state` by `permissioned_as` left its worker-local twin keyed by
workspace and script path alone, so the boundary held only where the database row
was consulted. An agent worker never reads that table — `Connection::Http` leaves
`latest_job` as `None` — so there the local cache was the whole boundary and it
had none: the next principal to retry the script on that worker restored the
previous one's `select` and `vars`.
Also records the sqlx `--all-targets` trap in the update-sqlx skill: it is needed
for queries inside tests, and in a CE checkout it aborts on `tests/otel.rs`
(EE-only `otel_ee`) after having already emptied the cache.
* fix(dbt): log a dropped retry-state save, correct the run-progress contract
Saving retry state is best-effort — losing it costs a retry, not the run that
just finished — but `.ok()` dropped the reason too. The only symptom was `dbt
retry` reporting nothing to resume, which reads as a bug in retry rather than a
failed write. Found by running a real failing build against a worker whose
binary predated the `permissioned_as` column: the insert violated NOT NULL and
said nothing.
The run-progress endpoint's OpenAPI description promised an empty list for a
caller who cannot see the job. It is refused instead; an empty list means the job
recorded nothing yet or is unknown here.
* fix(dbt): return the retry-state write failure the warning was added to report
`save_run_state` discarded the insert result, so the caller's warning could never
fire and a lost retry row stayed silent — the symptom being `dbt retry` finding
nothing on another worker.
The error is held rather than returned at once: the worker-local copy is what an
agent worker resumes from, so a failed insert must not cost that too. Every exit
after it surfaces it, including the ones that give up on the local save.
* fix(dbt): decide a retry's graph from its restored args, keep local state in step
Three from the seventh local review.
A retry submits only `dbt_command`, so the vars-override check ran against an
empty argument set and left `graph_is_per_run` false. The failed run's arguments
are restored afterwards, and those are what the retry builds with — an overridden
one wrote no snapshot for its own job and its page fell back to the deployed
graph, showing the wrong enabled models, aliases and schemas. The decision is
re-asked once the restore has happened.
A failed durable write no longer publishes the worker-local generation either.
`restore` accepts a local generation only when the database row names it, so
publishing one the database never recorded made this worker reject its own newest
state and resume the previous run's — its selection and vars, or "nothing to
retry" if that one had succeeded. An agent worker attempts no durable write, so
it keeps its local copy as before.
A rename that also converts away from dbt moved the old path's retry state onto
the new one, reinstating what the conversion had just cleared and leaving one
user's arguments and results under a path no dbt script occupies. It moves only
while the destination stays dbt, and clears the source otherwise.
* fix(dbt): drop retry state when a run produced none, let module pushes fail loudly
Three from the eighth local review.
A run that never wrote `run_results.json` — cancelled, timed out, or dead before
dbt got there — left the PREVIOUS run's state authoritative in both the database
and the local pointer, so a later `dbt retry` resumed that older invocation's
failed nodes. Producing nothing resumable now clears both copies, so neither can
answer for the other.
`wmill sync push` wrapped the descriptor lookup and its deployment in one
try/catch meant for a missing parent. Any API failure or invalid descriptor was
reported as "no parent found" and swallowed, so a module-only push exited zero
with the remote project unchanged. Only the lookup is tolerated now.
Also condenses a comment that narrated how earlier status comparisons behaved.
* fix(dbt): forget retry state on pre-build exits too, drop cascade claims
A dynamic run whose pre-build `dbt parse` or graph ingest fails returns before
the save that clears stale state, so the previous run stayed authoritative in
both the database and the local pointer and `dbt retry` resumed ITS failed nodes
— writing relations the run that just failed never touched. Both exits now
invalidate, through one helper shared with the no-artifact case.
Two frontend comments described dbt producer rows as driving cascade dispatch.
The executor returns before dispatch for every dbt job and deployment rejects
`table://` subscriptions, so they promised behaviour that cannot occur; they
describe the lineage and ownership that is actually retained.
* fix(dbt): let the database decide retry state where it is reachable
A SQL worker treated "no `dbt_run_state` row" as no opinion and accepted any
worker-local generation. But no row is the authoritative answer that the last
invocation left nothing resumable, so a local pointer that outlived it — an
unlink that failed, a process killed between the delete and the removal, a stale
cache — resurrected a replaced run and let `dbt retry` write relations it never
touched. An agent worker keeps accepting its local copy: it has no authority to
consult.
Invalidation failures are logged rather than dropped, since a silent one is
exactly what leaves the pointer behind.
* docs(dbt): record how to run an agent worker locally, keep archived graphs
Every step of standing one up fails as something else: a normal build cannot
start one at all, the server's routes need a separate feature, and all three
token mistakes surface as a bare 401 on the agent with the reason only in the
server log. Written down with the error each produces.
Also keeps a dbt script's graph when it is ARCHIVED rather than deleted. The
pinned read resolves versions through a CTE that already skips archived rows, so
clearing bought nothing and emptied the Models panel of every completed run of
the project. Deletion still clears it.
* feat(dbt): let an agent worker publish its graph, through one endpoint
An agent worker was refused any dbt script whose profile comes from a Windmill
resource — the common case — because it could neither read the stored relation
root to check for drift nor re-ingest a corrected one.
Those look like two needs but collapse into one: verification exists only to
decide whether the stored graph still describes reality, so a worker that can
PUBLISH never has to ask. It stores what it just parsed.
`POST /api/agent_workers/dbt_graph/{workspace_id}` is the whole addition. It
wraps the same `replace_dbt_manifest` the SQL path calls, so digest suppression,
the marker write and retention cannot drift between the two transports, and it
refuses a job the token's tags do not cover. `IngestedManifest`/`IngestedNode`
gain Deserialize to cross the wire.
Two guards go, both now false: the pre-build refusal, and the `Connection::Sql`
gate added earlier to stop a `vars` override grounding an agent run.
Live progress stays SQL-only — that is a per-model event stream, and routing it
through the API would mean a round trip per node.
* docs(dbt): warn that a differing cargo feature set swaps the shared binary
* docs(dbt): record the verified agent-worker behaviour and the tmpfs quota trap
An agent worker now runs a dbt job end to end, retries, and publishes its graph
— confirmed with a dynamic descriptor whose per-run snapshot came back through
the new endpoint. The doc said it was refused; that was true before the endpoint
existed.
Also `WINDMILL_DIR`: on a dev box the job dies with `Disk quota exceeded (os
error 122)` writing the project's files while `df` shows free space AND free
inodes, because /tmp is a tmpfs carrying a per-USER quota. Point the worker at a
real disk rather than trying to clean up beneath it.
* chore(dbt): pin the EE revision carrying the agent graph endpoint
* fix(dbt): bind the published graph to the job, break the completed-page poll loop
Five from the thirteenth local review.
The EE endpoint took `script_path` and `script_hash` from the payload and checked
only that the supplied job carried one of the agent's tags, so an agent holding
any matching-tag job could name another script and replace its graph. Both are
read from the verified queue row now and the request carries only the job id. A
raw preview has no version, so it no-ops rather than 422ing before dbt runs.
`IngestedManifest`/`IngestedNode` take `#[serde(default)]`: they were
serialize-only, and a field the serializer skips made the whole manifest
unparseable on the receiving side.
A completed run page fetched the graph forever — `load()` assigns `raw`, which
recomputes `settled`, which re-entered the same effect. The final fetch is keyed
to the job by a plain (non-reactive) variable, and `settled` is read untracked.
The pin now names a revision that compiles: the previous one still called
`authed.tags()`, a method that does not exist, because both that fix and the JSON
response landed after it was committed.
* fix(dbt): keep a run snapshot out of the script's deployed ownership
Everything `persist_ingest` writes after the manifest is keyed by PATH — one row
set per script, describing what is deployed there. A run snapshot was still
reaching it, so a one-off `vars` override republished that invocation's relations
as the script's ownership and the workspace graph stayed on the override's
schemas and aliases: an ordinary run of a static descriptor never ingests again
to correct it, so only a redeploy would. A snapshot now stops after recording its
own rows.
The row preview also selected a bare model name, which dbt resolves across every
installed package while `show` takes a single node — a project model sharing its
name with a package's was previewed wrongly or refused. It selects the
package-qualified FQN.
* fix(dbt): forget stale retry state when preparation itself fails
`prepare_project` runs before every path that could clear it, and it fails for
reasons unrelated to the saved run — a profile that stopped resolving, a
provision cancelled, packages that will not install. The invocation still left
nothing resumable, so the previous one must not stay authoritative: a repaired
project would otherwise let `dbt retry` rebuild an older run's selection and
write relations the latest invocation never reached. A retry is exempt, since it
is trying to use that state and failing to prepare says nothing about it.
Also corrects the runtime doc, which still described agent workers as unable to
run dynamic descriptors or Windmill-resolved profiles. They publish their graph
through the API now; what they do not get is live progress and a durable retry
row, and the doc says so.
* fix(dbt): spell the whole FQN for preview, bound retained retry generations
The FQN selector added last commit was `<package>.<name>`, but a dbt FQN is the
resource's path within its package and the matcher must consume the selector and
end on equal lengths — so it matched nothing for a model under `models/marts/`,
which is the layout most projects use and the one this repo's own complex fixture
has. The middle segments come from `original_file_path`, whose first element is
the resource root the FQN excludes. Without a path it falls back to the bare
name: ambiguous across packages, but a selector dbt resolves rather than rejects.
Tested on a nested model, which is the input that separates the three spellings.
Superseded retry generations were removed only when a later run published one,
and never inside the hour-long grace period — so a burst left a manifest and a
results copy per run with nothing afterwards to collect them. At most four now
sit in the grace window, oldest evicted first.
* fix(dbt): scope preview state to the run, seed the project on a language switch
Previews are keyed by `unique_id`, which is the same string for the same model in
every run, and the run-change effect reset only the graph and progress. Opening a
second run of one project therefore showed the previous run's rows immediately,
and `runPreview` treated them as cached and refused to fetch. A generation
counter also drops a preview that resolves after navigation, which the reset
alone cannot catch.
The dbt project was seeded only by the empty-script bootstrap, but dbt is in the
ordinary language picker: reaching it by switching a draft produced a script with
no `dbt_project.yml`, which the runtime refuses to deploy or run. Both entry
points seed now, and neither touches modules that already exist.
* chore(dbt): cache the agent graph endpoint's query for the EE offline build
* fix(dbt): publish the graph a moved profile built
A run snapshot stopped before everything `persist_ingest` keys by PATH, which is
right for a one-off `vars` override and wrong for the other two reasons a run
re-ingests. `graph_is_per_run` was one bool for all of them, and the profile
drift check both sets it and reads back what the publisher recorded: a profile
moved A->B was detected by every run forever, each paying a `dbt parse` for a
snapshot nobody reads while the asset rows went on naming schema A.
The reason is carried now (`GraphRefresh`), and it decides both writes. Drift is
the version's own move, so it rewrites the VERSION's graph and republishes the
ownership that ends the drift; a dynamic descriptor snapshots under its job id
and still publishes; anything the CALLER scoped — an overridden `vars`, a
narrowed `select` — snapshots and publishes nothing, so one invocation's subset
can neither stand as what the script owns nor drop the models it left out from
the version's graph. Where they meet the caller wins, and the next ordinary run
settles the drift.
A restore also rebuilt `run_results.json` by copying the generation directory a
second time, so a burst of saves pruning it mid-restore left `dbt retry` with
nothing to resume and a job that reported success. It is written from the bytes
the restore already read; a manifest that went the same way falls back to the
parse a database restore pays anyway, and a generation that vanished before
either read falls back to the database's row for that same run instead of
reporting there is nothing to retry.
* fix(dbt): select a row preview by package, not by file path
The preview built dbt's FQN by dropping one segment of `original_file_path`,
which assumes the model root is `models/`. A project setting
`model-paths: ["src/models"]` turned `src/models/marts/orders.sql` into
`pkg.models.marts.orders`, and dbt's matcher — equal lengths, compared from the
front — resolves that to nothing: the preview came back empty for every model in
the project.
It selects `<name>,package:<pkg>` instead. The comma is dbt's intersection
operator, so this names the node by its own name and the package it belongs to,
which is what the FQN was reaching for and needs no knowledge of the resource
root. Verified on dbt-core 1.12, dbt-core 2.0.0-alpha.5 and fusion
2.0.0-preview.202, including a package shipping a model whose name the root
project also uses.
* fix(dbt): refuse a lockfile version that is not one, keep a named selector
Two things a preview reaches that a deploy does not vouch for.
A raw preview submits its own `lock`, so `engine_version` arrives from the
caller and was interpolated straight into the engine cache path — `../..` in it
made the download, extraction and rename land anywhere the worker can write,
and provisioning runs on the host rather than inside the dbt jail. Both it and
`adapter_version` (a pip requirement) are now accepted only as a plain version
token.
`effective_selector` also read any submitted `select`/`exclude` as an override
of the descriptor's named selector. The generated run form posts a default back
for every field the caller left untouched, and a selector descriptor's `select`
default is `[]` — so pressing Test, saving a schedule or firing a webhook built
the WHOLE project instead of `--selector nightly`. An override is now one that
DIFFERS from the descriptor's own value; a run that wants the whole project
despite the selector asks with `["*"]`.
* fix(dbt): let a moved profile settle, from the runs that actually happen
Two ways the drift check could never come to rest, both verified against a real
run of a real project on a normal worker.
`add_caller_args` read any submitted `select`/`exclude` as a caller's narrowing.
The generated run form posts a default back for every field left untouched, so
every run from the UI, a schedule, a webhook or a flow step carried them and was
marked caller-scoped: with the profile moved A->B, each one stored its models
under its own job id and left the workspace graph — and the root the check reads
back — at A. Since no UI run omits the field, the "an ordinary run settles it"
escape hatch was unreachable. Both this and `effective_selector` now ask one
question, `selection_is_overridden`: DIFFERENT from the descriptor's, not merely
submitted.
The root was also recorded beside the path-keyed publication rather than beside
the graph it describes, so a version that cannot claim the path — an older one
run by hash, a deploy overtaken by a newer one — rewrote its graph at the moved
root and recorded nothing. Its next run then compared against a root that was
absent or two moves stale and skipped the refresh its own run page needed. It is
written wherever the deployed row's graph is.
Verified end to end: same UI-shaped arguments before and after, the moved
profile now republishes (asset rows and version graph both move to the new
schema), a second run detects nothing and re-parses nothing, and moving the
profile back settles it again.
`prune_dbt_run_graphs` also ran from runs alone, while a deploy writes a whole
node set of its own, `raw_code` per model included: a project redeployed on
every push by CI and run nightly kept one full graph per push until the next
run, and one deployed but never run kept them for good.
* fix(dbt): drop a self-dependent effect in the run graph
`previewGen` was `$state` written by the effect that also reads it, three lines
under a `finalLoadFor` that is a plain `let` for exactly that reason. Nothing
reactive reads it — the only reads are inside `runPreview`, a plain async
function — so it becomes a plain `let` too.
* fix(dbt): pin a retry to the engine versions it resolved
`run_identity` carried the engine KIND but not the version it resolved, nor the
dbt-core 1.x adapter's. Redeploy an unchanged project after a release and it
locks a newer dbt or adapter while the saved `run_results.json` still passes the
check, so `dbt retry` feeds one version's artifacts to another — the exact
reproducibility the lockfile exists to hold. Both resolved versions are in the
identity now; a real failure and retry still resumes.
Also drops three comments that outlived what they describe: two said `[]`
clears a descriptor's selector, which `selection_is_overridden` reversed, and
one pointed at an agent-worker guard that no longer exists — the agent path
reaches the ingest deliberately and publishes through the API.
* fix(dbt): discard a run graph the page has already navigated away from
The component is reused across runs, so a slow `/jobs/dbt_graph` or progress
response could land after the reset and put the previous run's models, statuses
and failure state on the current run's page, where nothing would fetch again to
correct it. Every response is now checked against the generation it was
requested under — the counter the preview path already used, renamed for what
it means.
The graph poll also backs off. Neither of its stops is reachable for a whole
class of runs — `dbt_snapshot_job` never matches a static descriptor, and
`polled` stays empty for the engines that emit no node events — so an ordinary
run walked to the cap, re-sending every model's SQL 40 times in two minutes.
* fix(dbt): forget the previous run when the durable save fails, keep quoting
`save_run_state` returns the database error when its upsert fails, which leaves
run N-1's row and local generation in place: same project, same arguments, so a
`dbt retry` matches them and resumes an older attempt's failed nodes against
this checkout — the outcome the no-results branch twelve lines above calls
`invalidate_run_state` to prevent, reached by another door. It now goes through
the same call. Best effort, since the delete goes to the database that just
refused a write, but the local pointer is what a retry landing back here reads.
The run page also rejoined a relation's parts after `splitRelation` stripped
their quotes, so the one name the button exists to paste —
`"wh"."analytics.v2"."Order Items"` — was copied as something no client
resolves. It copies `relation_name` verbatim.
And a source on a finished run was called another project's: the check that
guards against two projects claiming one relation asks whether this run executed
the node, and a run executes no sources — they appear in no `run_results.json`.
Nothing materializes a source, so that warning could never be true of one.
Docs: the `vars`-override paragraph still said such a run does not refresh the
graph, which the table above it contradicts — it refreshes under its job id and
publishes nothing.
* fix(dbt): keep the asset rows and the version's models describing one graph
The workspace graph takes an asset's relations from the path-keyed `asset` rows
and its models, SQL, tests and lineage from the version's `dbt_node`/`dbt_edge`.
A dynamic descriptor published the former while storing the latter under its own
job id, so a placeholder that moved an alias or a schema left the current graph
with assets no model stands behind — nothing dbt contributes to them survives.
Ownership is published exactly when the VERSION's graph was written now, which
is the only state in which the two agree. Two cases are settled elsewhere by
design: an override's relations are a one-off, and a dynamic descriptor at a
moved profile keeps the deploy's ownership until a redeploy — its runs each show
their own models and it re-parses regardless, so the undetected drift costs it
nothing it was not already paying.
An agent worker has no durable row, so its local `current` pointer is the whole
of what a retry reads — and every local publication failure returned success
with the PREVIOUS run's pointer still in place. Where a row exists that is
harmless (`restore` takes a local generation only when the row names it), so the
abandonment is scoped to the agent case.
A failed `dbt show` also cleared the retry state: the preparation-failure exempts
`retry` but not a read-only command, and the run page's row preview is exactly
that, run as the principal the state is keyed by — so a preview that could not
provision took the retry away from the run being looked at.
Frontend: the run-change reset left `loading` and `failed` behind, so the gap
before the next run's answer rendered "no models in the asset graph" — a claim
about the descriptor — over a project that is fine. And three derivations argued
from "the graph is the current deploy", which the pinned endpoint made untrue;
each is still needed, for the version-graph rewrite and retention reasons now
written down.
* fix(cli): let --skip-scripts cover a script's module files
The module shortcut in `elementsToMap` maps the file and `continue`s before
every skip filter, and a module is deployed as part of its parent script — so
`wmill sync push --skip-scripts` still pushed the script whenever one of its
modules changed, and pull still overwrote them locally. Harmless while a module
was a rare helper file; every file of a dbt project is one of these now.
* docs(dbt): a dynamic descriptor's ownership stays the deploy's
* fix(dbt): read the run out of a failure whose message has braces of its own
`parseDbtRun` anchored on the FIRST `{` in the error message and parsed
everything after it. The worker appends the structured result after the error
text, and dbt's errors carry braces — a Jinja template, the compiled SQL, an
adapter's own JSON — so the failures most worth reading were the ones whose
summary and per-node outcomes the run page dropped. Every brace is tried now,
bounded, and the first that parses as a run wins.
Pins the EE revision that gives the agent publish endpoint the deleted-version
guard the SQL path takes: deletion is soft, the foreign key still accepts graph
rows, and the pinned graph query serves non-live versions, so an agent finishing
during a delete put a deleted project's model SQL back on screen. The query is
byte-identical to `persist_ingest`'s, so the offline cache already covers it —
verified with a full-EE `SQLX_OFFLINE=true` check.
* fix(dbt): seed a project when a modular draft switches to dbt
`seedDbtProject` returned whenever the draft carried any module at all, so a
modular script holding a `helper.ts` reached dbt with none of what dbt needs:
the project view is read-only, and the worker refuses a bundle without
`dbt_project.yml`, so that draft could neither run nor deploy. Keyed on the
project file now, and the seed goes in under whatever is already there — the
previous language's helpers are inert to dbt and the user's to remove.
Also records this runtime's schema in `backend/summarized_schema.txt`: the
`table` asset kind, the `dbt` script language, the five dbt tables and the
`materialization_status` enum the progress table uses.
* docs(dbt): move the pipeline-membership rationale out of the deploy path
* fix(dbt): gate a pinned run's model SQL on the version it belongs to
The `EXISTS` against `script` is the only thing standing between a share-link
viewer and the project's source, and it matched the workspace and path alone.
`extra_perms` is a grant on a ROW: archive a version that granted someone
access, recreate the path with narrower permissions, and that stale grant
satisfied the probe while the query returned the NEW version's `raw_code`. Both
probes name the hash now. The regression test drives exactly that shape and
fails without it, returning `select 2` to a caller granted only on the archived
version.
* fix(dbt): keep a delimiter an identifier escaped by doubling
Every dialect these relations come from escapes its own delimiter by doubling
it, and both split functions closed the quoted section on the first half and
reopened on the second: `"schema"."a""b"` came out as `a.b`. The manifest keeps
the real spelling, so the run wrote its per-model status and row counts under an
asset path no graph node has — the node simply never moves, which is the failure
mode this splitter exists to prevent.
Fixed in the worker and in its frontend mirror, which have to agree, with a case
per delimiter on both sides.
* fix(dbt): key retry state by the caller, not only by the principal it runs as
An `on_behalf_of` script executes every caller's job as its owner, so
`permissioned_as` names one principal for all of them and the retry state — the
durable row and the worker-local generation both — collapsed onto a single
entry. After one caller's run failed, the next could submit `dbt_command: retry`
and resume it: their arguments replayed against the warehouse, and handed back
through `invocation_args`. Nothing else separated them, and on an agent worker
the local directory is the whole boundary.
`created_by` joins the key in both places. For an ordinary script it changes
nothing — `permissioned_as` is already that caller — and a run that was itself
superseded was never resumable anyway.
Includes the offline cache for the four changed queries and the three the
pinned-graph regression test added last commit, which had none: `prepare`
without `--all-targets` does not compile test targets, so CI's
`SQLX_OFFLINE=true ... --all-targets` would have failed on them.
* fix(dbt): compare the schema too when reporting a relation that moved
`relationDrift` compared the leaf name alone, and the move it exists to report —
a profile repointed at another schema, which a later run then writes into the
version's graph — leaves every model's name exactly where it was. So the one
case that reliably produces a graph naming relations this run did not write was
the one case the notice stayed silent for.
The schema segment joins the comparison, qualified against qualified: an
unqualified one means the target's own database, which the relation names
anyway, so comparing that would report a move on every node.
* fix(dbt): bound the retry state now that it is keyed per caller
Keying by `created_by` fixed one caller resuming another's run and created a
growth problem doing it: a shared `on_behalf_of` script kept one row and one
worker directory for everyone who had ever run it, and the generation prune only
bounds files INSIDE a directory.
Three bounds, none of them new machinery. A run with nothing failed or skipped
saves nothing — `dbt retry` builds from those nodes alone, so that state could
only ever be refused — while still clearing what the previous run left, since
its failures are no longer what last happened here. The rows expire on the same
30-day clock as a run snapshot, swept per path by the prune every dbt job
already spawns. And the worker-local directories are swept there too, by the age
of the pointer a save rewrites, because their digest names neither the script
nor the caller.
* docs(dbt): the retry state is worker-affine only on an agent worker
* fix(auth): only the server may set a token label that names a user
`create_token_internal` wrote `NewToken.label` verbatim, and the auth layer reads
some labels as an IDENTITY: `username_override_from_label` maps
`ephemeral-script-end-user-<name>` to exactly `<name>`, which then becomes
`created_by` on every job that token pushes. The label is free-form request
input, so any member could mint a token that speaks as somebody else — the shape
`require_job_read_access` already works around when it refuses to trust
`username_override` and falls back to an RLS probe, and the one that made dbt's
retry-state key (`created_by`) forgeable for an `on_behalf_of` script.
The labels are refused where request input enters: the member-facing
`tokens/create`, and `impersonate`, which names its subject in
`impersonate_email` and has no business renaming the caller too. The legitimate
producers are unaffected — a job's own token comes from `create_token_for_owner`
in the worker, and native triggers and app-embed tokens build their labels
themselves rather than accepting one.
`Ephemeral lsp token` stays allowed: its override is the fixed sentinel `lsp`,
not a name the caller chose, and the editor mints exactly that label through this
endpoint for its language server. The test pins the two lists together, so an arm
added to `username_override_from_label` that lets a label choose a name fails
until it is reserved too.
* Revert "fix(auth): only the server may set a token label that names a user"
This reverts commit
|
||
|
Ruben Fiszel
|
dda59767c2 |
feat: stamp webhook trigger_kind on token-driven job runs (#10431)
* feat: stamp ui vs webhook trigger_kind on direct job runs * fix: gate ui trigger kind on min worker version and dedupe display names * docs: state that the ui trigger kind attributes rather than proves * refactor: fold the trigger fallback into one trigger_or_fallback helper * feat: hold trigger_kind as a tolerant label on the worker paths * chore: refresh the sqlx offline cache for the trigger_kind label queries * chore: update ee-repo-ref to 7de7daff5eed410e0c815ad6b292d2b4303f02f2 This commit updates the EE repository reference after PR #700 was merged in windmill-ee-private. Previous ee-repo-ref: 974ab910d9a30c5565e1198ee312acc6d11239f3 New ee-repo-ref: 7de7daff5eed410e0c815ad6b292d2b4303f02f2 Automated by sync-ee-ref workflow. * fix: keep the API job structs tolerant of unknown trigger kinds too * chore: point ee-repo-ref at the merged EE main --------- Co-authored-by: windmill-internal-app[bot] <windmill-internal-app[bot]@users.noreply.github.com> |
||
|
Ruben Fiszel
|
02c4a9e515 |
fix: carry the token label into job-run audit rows (#10433)
* fix: carry the token label into job-run audit rows * docs: state the audit end-user precedence at the push signature * chore: point ee-repo-ref at the companion branch * docs: state the username/end_user split at the push signature * feat: keep the audit caller searchable when a token label takes end_user * fix: skip the caller parameter when it repeats the end user |
||
|
Ruben Fiszel
|
c69f08073a |
fix: add apps:run to the token scope picker and confine path-scoped app tokens (#10428)
* fix: expose apps:run in the token scope picker and let apps:write grant it Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix: confine path-scoped app run/write tokens to the app they name Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix: let apps:run read back its own app's S3 files, condense scope comments Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix: let apps:write mint apps:run and extend run read-back to app S3 display routes Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * docs: drop stale embed-token wording from the app S3 helper summary Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com> |
||
|
Ruben Fiszel
|
3716a71fd7 |
fix: credit the token owner instead of the token label in the audit trail (#10423)
* fix: credit the token owner instead of the token label in the audit trail Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix: address review findings on token-owner audit attribution Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix: carry token-label provenance explicitly instead of inferring it Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * chore: point ee-repo-ref at the companion branch Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix: trust only non-forgeable token labels to name the acting entity Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix: reject reserved system-token labels at token creation Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix: narrow the token-label guard to server-minted namespaces Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix: add the provenance field to the remaining ApiAuthed literals Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix: stop trusting the email- label, which no mint produces Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com> |
||
|
Ruben Fiszel
|
f9d5da11b7 |
feat: allow changing an account email in the superadmin settings (#10355)
* feat: allow changing an account email in the superadmin settings * fix: cover slack_email and usage rows, and scope job rewrites to the queue * fix: compare the destination email case-insensitively * fix: only warn about the consequences once the email is edited * docs: warn that changing an account email is a last resort * fix: repoint app policies and raw-email permissioned_as, reject self-change * fix: repoint folder default rules and guard the varchar(55) job column |
||
|
Ruben Fiszel
|
4a898247a2 |
fix(apps): let entitled viewers read pre-existing S3 files from deployed apps (#10245)
* fix(apps): let entitled viewers read pre-existing S3 files from deployed apps Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(apps): confine S3 viewer fallback to full unscoped sessions A scope-restricted token (e.g. apps:read:<app>, or an app-embed token) is allowed on apps_u/* but rejected by the route-scope middleware on job_helpers/*, so granting it the viewer fallback would be a new capability it cannot obtain directly. Gate the fallback on scopes.is_none() so only full sessions (which can already read via job_helpers) delegate; scoped and anonymous callers stay gated. Add a scoped-token isolation assertion. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(apps): treat filter-tags-only tokens as unscoped for S3 viewer fallback The scopes.is_none() guard wrongly denied the viewer fallback to tokens that are effectively unscoped (empty scope arrays and if_jobs:filter_tags:-only tokens), which the route-scope middleware treats as unrestricted and which can therefore read the same file via job_helpers directly. Reuse that semantics via a shared is_effectively_unscoped helper so the relaxation covers exactly the tokens that gain no new capability, while genuinely scoped tokens stay gated. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
|
Ruben Fiszel
|
68debab877 |
feat(triggers): add AMQP (RabbitMQ) trigger via lapin (#10230)
* feat(triggers): add AMQP (RabbitMQ) trigger using the lapin library Fixes WIN-2214 Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * chore(triggers): defer AMQP cross-workspace deploy pending utils-internal publish Revert the amqp_trigger additions to the shared windmill-utils-internal TriggerDeployKind and the frontend cross-workspace deploy adapter: the frontend installs the published npm package, which lacks the new kind until a release is cut. AMQP create/edit/delete/list/sync/capture are unaffected (they use local types); only cross-workspace deploy/merge of AMQP triggers waits on the package bump. Also document the at-most-once ack in the consumer loop. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(triggers): address AMQP review — at-least-once ack, workspace cascade, contracts - ack AMQP deliveries only after successful dispatch; nack+requeue on failure - add ON DELETE CASCADE workspace FK so amqp_trigger rows are cleaned on workspace deletion (and the listener stops) - fix the /amqp_triggers/test OpenAPI body and add amqp_trigger to WorkspaceDiffRow.kind - register AMQP in the generated workspace trigger tool (create_trigger) - drop banned $bindable defaults on optional props in the config section - add build_uri unit tests (encoding, ports, vhost) Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(triggers): stop AMQP poison-message loop and reconnect on transient drops Chaos testing against a live RabbitMQ broker showed the previous nack(requeue) + immediate re-poll spun a tight redelivery loop (~1000 critical-error reports/sec) on a poison message, and any connection blip permanently disabled the trigger (lapin has no built-in reconnect). - on dispatch failure: nack+requeue then stop consuming; the listener framework re-lists the trigger after its ping goes stale (~15s), backing redelivery off to that cadence instead of a tight loop (verified: rate dropped from ~1000/s to ~1 per ~26s, message preserved) - on connection/stream error: stop and let the framework reconnect instead of disabling; persistent failures are still disabled via get_consumer (verified: a forced connection close now auto-reconnects and resumes) - finish the AI create-trigger action wiring for AMQP: add amqp to CreatedResourceTriggerKind, the action-card registry, and the drawer registry so the result card renders and its "Open" action works Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(triggers): complete AMQP frontend registries and defer merge rows - add amqp to capturableTriggerTypes (so AmqpCapture mounts), the Runs jobTriggerKinds filter, and CLOUD_DISABLED_TRIGGER_TYPES - wire AMQP into global AI chat mode: TRIGGER_KINDS, the request union, writeTriggerSchema, triggerServices, and the draft adapter - stop emitting actionable AMQP fork-comparison rows (revert amqp_trigger from TRIGGER_OR_SCHEDULE_TABLES) since cross-workspace deploy is deferred until windmill-utils-internal is published — avoids a deploy that fails with "Unknown kind: amqp_trigger" - use design-system TextInput instead of raw <input> in the config section Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(triggers): complete AMQP session/draft registries and constrain prefetch - add amqp to the session-deploy, draft-compare, preview-router, and copilot workspace-item registries so AMQP drafts/deploys/nav/path resolution work - include amqp_count in the MoveDrawer attached-trigger rename warning - replace the raw prefetch <input> with a design-system TextInput bounded to an integer 1-65535 (backend u16) and block save on invalid values Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(triggers): make AMQP disconnect/reconnect consistent with the Kafka trigger lapin, like rdkafka, has no transparent reconnect, so the AMQP listener now mirrors the Kafka trigger's explicit reconnect loop instead of relying on the framework re-list (which disabled the trigger once get_consumer failed on a sustained outage): - get_consumer returns cheaply; consume owns a (re)connect loop that retries with a 30s backoff, reports a critical error every 10 failed attempts, and reports a recovered critical error once it reconnects — never disabling the trigger on a connectivity failure - a consumer/stream error breaks out to reconnect rather than disabling - dispatch failure still nacks+requeues (at-least-once) with a short backoff to avoid a tight poison-message loop, keeping the connection alive Verified against a live RabbitMQ broker: killing the broker keeps the trigger enabled and retrying (attempt N), and restarting it auto-reconnects (logs "reconnected after N attempts") and resumes dispatch. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(triggers): complete AMQP capture registries and constrain prefetch contract - add the 'amqp' case to triggerKindToTriggerType so opening the AMQP editor from a capture button no longer throws "Unknown TriggerKind: amqp" - register AmqpIcon in CaptureTable's icon map and add an AMQP entry to the script/flow CaptureButton menu - bound the OpenAPI prefetch_count to an integer 1-65535 (matches the Rust u16) and regenerate clients/prompts - require a non-empty exchange name when the exchange binding is enabled - build_uri: fall back to "/" on a blank vhost and bracket IPv6 hosts (+ tests) Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(triggers): wire AMQP into pipeline graph, git-sync, and preprocessor types - asset_graph: discover attached amqp_trigger rows and emit an AMQP TriggerEdge so AMQP triggers render (and can be opened/deleted) on the data-pipeline canvas - frontend pipeline graph: add amqp to NativeTriggerKind, the add-trigger menu, node presentation, event-trigger set, annotation keywords, and the editor/service registrations - git-sync: add the amqp_trigger include pattern (+ test) so an AMQP git-sync deployment stages only its .amqp_trigger.* file, not an unrelated same-path object - preprocessor starters: add the AMQP event to the generated TS/Python/PHP trigger event types (kind/payload/exchange/routing_key/queue_name/redelivered/ delivery_tag) Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(triggers): finish AMQP pipeline/parser wiring, prefetch validation, source lists - fix a stray edit that corrupted the pre-existing MqttTriggerEditor import ($lib/... path) in PipelineTriggerEditors.svelte - reject prefetch_count = 0 server-side in validate_config (RabbitMQ treats 0 as unlimited) and defensively skip basic_qos(0) in build_consumer (covers the capture path that bypasses CRUD validation) - recognize `// on amqp` in the canonical parser (TriggerSpec::Amqp) and add amqp to the CLI non-autorun/event-trigger sets so a pipeline cascade never runs an AMQP-only node as a manual root without an event - add amqp to the preprocessor intro lists and both pipeline AI instructions Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(triggers): reject zero AMQP prefetch in all paths and finish guidance lists - extract a shared validate_amqp_options used by both CRUD validate_config and build_consumer, so capture configs (which bypass CRUD validation) also reject prefetch 0 instead of silently connecting with an unlimited buffer (+ unit tests for 0/1/65535/None) - add AMQP to the main script-writing preprocessor-sources prompt and the CLI triggers-skill guidance list Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * docs(triggers): de-duplicate AMQP prefetch comment and fix GET response text - keep the zero-prefetch rationale only on the shared validate_amqp_options doc; drop the redundant call-site comments - correct the getAmqpTrigger OpenAPI 200 description ("deleted" -> "retrieved") Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * chore: update ee-repo-ref to aaa6cb89b05b76139252c64f057e53b94d12ac60 This commit updates the EE repository reference after PR #680 was merged in windmill-ee-private. Previous ee-repo-ref: 5da5fd65aca9594b2611837a52e4677b544b0380 New ee-repo-ref: aaa6cb89b05b76139252c64f057e53b94d12ac60 Automated by sync-ee-ref workflow. * chore(migrations): consolidate the four AMQP migrations into one The table and the three enum ADD VALUE statements (trigger_kind, job_trigger_kind, draft_kind) are one atomic feature. ALTER TYPE ... ADD VALUE runs inside the migration transaction on PG >= 14 (Windmill's minimum) since the amqp_trigger table doesn't reference those enum types, so they can share a single migration instead of four. Verified applying cleanly in a single transaction on a fresh DB. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Co-authored-by: windmill-internal-app[bot] <windmill-internal-app[bot]@users.noreply.github.com> |
||
|
Ruben Fiszel
|
fd51d40f12 | feat(pipelines): catalog declared measures and dimensions (#10190) | ||
|
Ruben Fiszel
|
4917f79935 |
fix(mcp): let MCP tokens call preview run tools (jobs:run scope) — Fixes GIT-920 (#10107)
* fix(mcp): let MCP tokens call preview run tools (jobs:run scope) The MCP proxy mints an internal JWT scoped to exactly `scope_for_route` for the endpoint it forwards to. For preview run routes (`run/preview`, `run/preview_bundle`, `run/preview_flow`, `run_wait_result/preview`, `run_wait_result/preview_flow`), `determine_kind_from_route` matched the `SCRIPT_JOBS` prefix `jobs/run_wait_result/p` (because "preview" starts with "p") and derived `jobs:run:scripts`. But the preview handlers run arbitrary request-supplied code with no deployed path and require the broad `jobs:run` scope, so `jobs:run:scripts` was rejected with 403 "Required scope: jobs:run". Preview/bundle routes now carry no runnable kind, so the derived scope is the broad `jobs:run` the handlers expect. This also aligns the route-level access check with the handler check for these routes. Fixes GIT-920 Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(mcp): anchor preview-route match to endpoint segment Address CI review: `route_path.contains("preview")` also matched by-path runs of a deployed runnable whose path contains "preview" (e.g. `jobs/run_wait_result/p/f/team/preview_report`). Since determine_kind_from_route also feeds check_route_access, such a route would derive the broad `jobs:run` and reject a legitimately kind-scoped `jobs:run:scripts:*`/`jobs:run:flows:*` token with 403. Anchor the exception to the actual preview endpoints (`jobs/run/preview*`, `jobs/run_wait_result/preview*`) so by-path runs keep their kind. Add regression tests for preview-named by-path paths, and trim the comments per AGENTS.md. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
|
Ruben Fiszel
|
04eb7ddd39 |
fix: clearer errors on auto-draft save failure (WIN-2157) (#10053)
* fix: clearer errors on auto-draft save failure (WIN-2157) When an autosave draft save fails, the cloud indicator now surfaces the backend reason on hover (native title tooltip) in addition to the existing click popover, so the cause is discoverable without a click. Backend now returns a clearer, actionable message: - `require_can_write_path` distinguishes a malformed path (unrecognized namespace prefix -> BadRequest) from a genuine permission denial, and the deny message spells out where the user *can* write. - `require_owner_of_path` no longer panics with an out-of-bounds index on a malformed single-segment path (e.g. a bare `u`/`f`); it returns a clear BadRequest instead. Covered by a regression test. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * chore: trim narrative comment to invariant in drafts.rs (WIN-2157) Address CI review (AGENTS.md: comments record constraints, not narration, ≤4 lines): keep the malformed-path invariant, drop the motivation tail. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix: don't let a malformed stored draft 400 the draft listing (WIN-2157) Address CI review (P1): require_can_write_path can now return BadRequest for a malformed path, and list_drafts propagated it — so a single malformed stored draft row (the draft table has no path constraint; legacy/admin-authored rows may be malformed) would make GET /drafts/list return 400. Treat BadRequest like NotAuthorized there: the row is simply not writable. Verified e2e on EE — listing returns 200 with can_write false for the malformed rows. Also trim "unchanged"/"still" drafting-history narration from the regression test comments (P2, AGENTS.md). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * chore: compress list_drafts comment to 4 lines (WIN-2157) Address CI review P2: keep the constraint (draft table has no path constraint) and the invariant (one malformed row must not 400 the listing) within the AGENTS.md ≤4-line limit. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
|
Ruben Fiszel
|
91e1b087a2 |
feat(auth): add runtime NO_AUTH mode for authentication bypass (#9962)
* feat(auth): add runtime NO_AUTH mode for authentication bypass Adds a runtime `NO_AUTH` env flag that makes every request resolve as the `admin@windmill.dev` superadmin with no login required, so self-hosted deployments can front Windmill with their own authenticating gateway without building a dedicated `oss` (compile-time `no_auth`) binary. - `NO_AUTH` is honored in any build but is force-disabled when `CLOUD_HOSTED` is set, so the managed cloud always enforces real auth. - The existing compile-time `no_auth` feature keeps its always-on behavior (`cfg!(feature = "no_auth") || *NO_AUTH`), so `oss` builds are unchanged. - `Tokened` now yields a synthetic token in no-auth mode so handlers that require it (e.g. global_whoami, called by the frontend on load) resolve. - A loud startup banner warns when the mode is on; `HIDE_NO_AUTH_BANNER` silences it once the operator has deliberately deployed behind a gateway. Fixes WIN-2131 Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(auth): dismissable NO_AUTH warning banner via global setting Replaces the HIDE_NO_AUTH_BANNER env flag with a UI warning banner that can be permanently dismissed for all users from within the running instance (not exposed in instance settings). - New `no_auth_banner_dismissed` global setting, only ever written by dismissing the banner itself. - `GET /api/settings/no_auth_banner` returns whether to show the banner (true only when NO_AUTH is active and it hasn't been dismissed). - NoAuthBanner.svelte renders a top-of-app warning in NO_AUTH mode; its dismiss button opens a confirmation modal, then writes the global setting via the existing setGlobal endpoint so it stays hidden for everyone. - The server still logs the startup NO_AUTH warning unconditionally. Fixes WIN-2131 Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(auth): resolve NO_AUTH in AuthCache so all_runnables works Codex/Pi review flagged that `/api/users/all_runnables` still failed in NO_AUTH mode: `get_all_runnables` extracts `Tokened` and re-validates the request token per workspace via `AuthCache::get_authed`, which rejected the fabricated `"no_auth"` token (no matching DB row) with a 400. Short-circuit `AuthCache::get_opt_job_authed` (the resolver behind `get_authed`) to the admin superadmin in no-auth mode, so any direct cache caller resolves without a real token. Single-source the mode check and the synthetic identity via `is_no_auth()` / `no_auth_admin_authed()` and reuse them across the extractor, resolver, and login paths. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * revert(auth): drop the NO_AUTH dismissable UI banner The in-app banner added a GET /api/settings/no_auth_banner request to every instance load for little benefit. The startup log warning already surfaces that auth is bypassed to operators, so drop the banner, its endpoint, and the no_auth_banner_dismissed global setting entirely. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
|
Ruben Fiszel
|
76a9523009 |
feat: use derived username instead of email for non-member superadmins (#9857)
* feat: use derived username instead of email for non-member superadmins Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix: address review - drop redundant username cache, guard whoami membership by email Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * refactor: use explicit non_member boolean instead of role string for superadmin banner Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix: resolve email from password table for non-member superadmin permissioned_as Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix: resolve non-member superadmin drafts via shared username->email resolver Adds resolve_username_to_email (usr, then super_admin password fallback for both derived-username and email modes) and uses it in get_email_from_permissioned_as and the drafts get/list endpoints, so a non-member superadmin's drafts resolve and no email leaks into the drafts payload. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * test: superadmin-not-in-workspace schedule uses derived username as permissioned_as Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix: resolve non-member superadmin identity in draft owner-circles, username_to_email, and home filter Applies the password-fallback username resolution to the script/flow/app/draft owner-circle subqueries and the username_to_email endpoint (was an admins-workspace 'username == email' hack), and switches the home items-list user-folder filter to the non_member flag instead of the now-broken username-contains-@ heuristic. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix: backfill non-member superadmin favorites from email to derived username Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix: propagate DB errors in username resolution instead of leaking email (CI review) Addresses cubic-dev-ai P2: get_instance_username_or_fallback_to_email now returns Result and only falls back to the email for a genuine 'no derived username'; a query error propagates so callers fail closed rather than leaking the raw email as the acting username. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix: clarify non-member superadmin popover (username used + admin permissions) Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix: keep username_to_email endpoint member-only to not disclose non-member superadmin email (CI review) Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix: forbid disabling automate_username_creation once usernames assigned (CI review) Makes the setting effectively one-way once instance-wide usernames exist, so the global-uniqueness invariant that keeps stored u/<username> identities (schedules/triggers/drafts/superadmin ownership) unambiguous can never be dropped back to workspace-local uniqueness. Re-saving false on an already-disabled instance stays a no-op. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
 centdix
|
9d61e4e59e |
feat: self-host docs search for chat, mcp, cli; drop inkeep (#9772)
* feat: self-host docs search for chat, mcp and cli; remove inkeep
Embed a vendored docs snapshot (llms.txt/llms-full.txt) in the backend and
serve ranking + page rendering from GET /api/docs/{search,page}. The AI chat,
the MCP searchDocs/readDocsPage tools, and 'wmill docs' all consume it, so docs
search works with no runtime egress and is no longer EE-gated. Removes the
inkeep proxy. EE companion deletes inkeep_ee.rs (ee-repo-ref bumped).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* refactor: name read_docs_page param `url` instead of `path`
search_docs returns each hit's `Source` URL, so the read tool now takes a
`url` argument to match — the AI/MCP loop reads "search gives a Source URL,
read takes that url" rather than copying a `Source:` URL into a `path` slot.
A bare `/docs/...` path is still accepted and canonicalized before lookup.
Regenerated openapi-deref, the MCP endpoint tools, and the frontend client.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* ci: add scheduled workflow to refresh the vendored docs snapshot
The backend embeds docs_snapshot/*.gz at build time, so the in-product docs
corpus is otherwise only as fresh as the last manual fetch.sh run. This adds a
weekly (and manually dispatchable) job that re-runs fetch.sh, sanity-checks the
result against truncation/garbage, and opens a PR via the internal app when the
snapshot changed — so a human reviews the docs diff before it rides into the
next release build.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* refactor: make docs tool-result strings caller-neutral
The search/page endpoints back three differently-named consumers (the AI chat
`read_docs_page` tool, the MCP `readDocsPage` tool, and the `wmill docs` CLI),
so the shared rendered text shouldn't name one of them. Refer to "the docs
page-reading tool" and its `url` argument instead, and add tests pinning the
caller-neutral follow-up guidance.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* chore: point ee-repo-ref at inkeep-removal companion rebased on EE main
The companion branch now carries only the inkeep_ee.rs deletion on top of EE
main (was based on the native-job-retry EE line, which polluted the EE PR diff).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* fix(docs): expose docs:read in token catalog; precompute lowercased corpus
Addresses two review nits on the self-hosted docs PR:
- docs:read was enforced (ScopeDomain::Docs) but missing from the token scope
catalog (token.rs ALL_SCOPES), so it couldn't be selected when creating a
standard scoped token in the UI — leaving scope-restricted CLI/MCP docs use
effectively ungrantable. Add a read-only "Documentation" group (no write
surface) and a test asserting it is exposed.
- search ran page.body.to_lowercase() on the whole corpus per query. Lowercase
body/title/description once at parse time (into the OnceLock corpus) and scan
the precomputed copies instead.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* chore: update ee-repo-ref to 27a4f41b8e5603d6e444efcfc420bd1c44a07eed
This commit updates the EE repository reference after PR #630 was merged in windmill-ee-private.
Previous ee-repo-ref: c7ec3a0c2fa38d4cb5e50bf0265eef4710de4860
New ee-repo-ref: 27a4f41b8e5603d6e444efcfc420bd1c44a07eed
Automated by sync-ee-ref workflow.
---------
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Co-authored-by: windmill-internal-app[bot] <windmill-internal-app[bot]@users.noreply.github.com>
|
||
|
hugocasa
|
88fca6a8c1 |
fix: enforce containment of python module dir for preview jobs (#9704)
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
|
hugocasa
|
043c2c05b7 |
fix: forbid superadmin job tokens from global user and token management (#9715)
* fix: forbid superadmin job tokens from global user and token management Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix: extend superadmin job token guard to offboard and export routes Apply forbid_superadmin_job_token to offboard_global_user and export_global_users, the remaining global user-management routes that were gated only by require_super_admin. Offboarding can delete a user along with their tokens, password, invites and instance-group membership, and export returns every user's password_hash, so both must be unreachable by a superadmin job token. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
|
hugocasa
|
2879cbb65a |
feat(apps): opt-in sandbox isolation for published & raw apps (alpha) (#9420)
* feat(apps): sandbox published & raw apps with a scoped embed token Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * chore: point ee-repo-ref at embed-token EE commit Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(apps): allow top-navigation from the sandboxed app iframe Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(apps): share app localStorage across apps via the embedder Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(apps): publisher disable-sandbox option with per-version viewer consent Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * chore(sqlx): cache for disable-sandbox queries Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * chore: bump ee-repo-ref to disable-sandbox EE commit Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(apps): always sandbox the served raw-app wrapper + viewer fixes The raw-app wrapper served by get_raw_app_data now always carries `CSP: sandbox`. The publisher "disable sandbox isolation" opt-out is applied entirely on the viewer side, which (after per-version consent) builds its own same-origin blob wrapper — so the backend-served document stays isolated regardless of how it is reached, never via a relaxed real-origin URL. Also: - CORS on the global /apps_u mount so the opaque viewer can load custom-path public apps cross-origin. - Reject runnable-bridge messages unconditionally until the iframe is bound. - Relay the viewer's in-app hash up to the embedder address bar so deep links stay shareable (hash only; embedder keeps its own pathname). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * feat(apps): render public raw apps single-iframe (drop embed token) Public raw apps now render directly on the real origin with a single opaque bundle iframe and the page credential, instead of the opaque viewer + scoped-token indirection. The author bundle stays isolated in its own opaque iframe (CSP-sandboxed); low-code apps, whose code runs in the viewer frame, keep the opaque viewer + scoped token. embed_token now reports raw_app and skips minting a token for raw apps; the access check still gates visibility. Also set disable_sandbox: None in the remaining Policy constructors so the full feature build (all_sqlx_features, enterprise, license) compiles. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * chore: bump ee-repo-ref to single-iframe raw-app EE commit Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * feat(apps): grandfather existing apps as legacy-unsandboxed + authed-only consent Existing apps are stamped by migration as `legacy_unsandboxed` so they keep running same-origin on upgrade — no breakage and no consent prompt. New apps are sandboxed by default; re-deploying an app clears the flag. The publisher `disable_sandbox` consent prompt is now shown only to authenticated viewers — an anonymous viewer has no session to expose, so the prompt was meaningless friction. embed_token reports `legacy_unsandboxed` and `authed`. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * chore: bump ee-repo-ref to legacy-unsandboxed EE commit Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * feat(apps): deploy-time migration prompt for legacy-unsandboxed apps On the first re-deploy of a grandfathered (legacy-unsandboxed) app, the publisher must explicitly choose: enable sandbox isolation (the flag is cleared → the app becomes sandboxed) or keep running without isolation (→ disable_sandbox, with per-version viewer consent). updatePolicy() no longer carries the legacy flag through a deploy, so the choice is what sticks. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * fix(apps): disable the sandbox-isolation toggle until the app is deployed The Deploy-drawer "Disable sandbox isolation" toggle called setPublishState() — which updates the app by path — even before the app was first deployed, when the path is empty, throwing an error. Guard it with disabled={!savedApp}, matching the adjacent visibility toggle. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * feat(apps): sandbox the in-workspace low-code app viewer in an opaque iframe Extend the opaque-origin iframe isolation to the logged-in /apps/get viewer. /apps/get becomes an embedder that keeps the workspace chrome + Edit button and renders the app inside a cookieless, chrome-less /app_embed viewer route, handed a scoped embed token minted from the member's session. The app frame runs in an opaque origin (no allow-same-origin), so it cannot reach the member's session cookie or window.parent. - apps.rs: get_app_embed_token_for_path (authed, by-path, scope + RLS gated); mint_app_embed_token grants a path-scoped apps:read:{path} so the viewer can load its own app definition and no other - lib.rs: CORS on /apps (bearer-token only, no cookies) for the opaque viewer's by-path reads - new /app_embed/[workspace]/[...path] viewer route (private analog of /public) - PublicAppFrame: viewerUrl prop to point the opaque iframe at the viewer route Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(apps): unify in-workspace app viewers on the shared sandboxed path Route every in-workspace app display (low-code and raw) through the same PublicAppFrame -> PublicApp machinery as the public viewer, so the sandbox / legacy-unsandboxed / disable-sandbox-consent behavior is identical on every page. - new InWorkspaceAppViewer renders both app types via PublicAppFrame; /apps/get and /apps_raw/get become thin wrappers over it - /apps_raw/get previously rendered RawAppPreview directly (always isolated, with no legacy-grandfathering or consent handling); now consistent with the rest - retire the legacy same-origin raw viewer /apps/get_raw/[version] and re-point the apps-list row to /apps_raw/get; remove the dead /apps_raw/[ws]/[version] route - load the raw bundle secret in the shared viewer (getAppByPath doesn't return it) Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(apps): address PR review feedback (scope + policy hardening, nits) - require handler-level apps:read on list_apps / list_search_apps so a scoped embed token cannot read app definitions through the list endpoints. The route layer treats apps:run as satisfying read; the handler check (which does not) closes the gap. - treat legacy_unsandboxed as backend-owned: strip any client-provided value in create/update so it can only be set by the grandfather migration, not the API. - document mint_app_embed_token's caller-verifies-access contract. - use Button's declared onClick prop for the consent action (was onclick, which fell into the rest-spread and bypassed the component's click handling). - test: lock that the embed scopes cannot satisfy domain-level apps:read. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * docs(apps): document embed-token endpoints in openapi + fix doc nit Second-round review nits: - add the three app embed-token endpoints (apps/embed_token/p/{path}, apps_u/embed_token/{secret}, and the EE apps_u/embed_token_by_custom_path) plus the EmbedTokenResponse schema to openapi.yaml; note .html on get_data - mint_app_embed_token doc: "Both" -> "All" (it lists three call sites) Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(apps): bound embed-token scopes to the caller's own The embed-token mint now enforces ensure_scopes_within_caller, so the minted scope set is always within the calling credential's own scopes (a no-op for regular unscoped sessions). Adds a unit test locking the boundary and documents the contract on mint_app_embed_token. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(apps): raw-app ctx in external embeds + page credential in direct render - RawAppPreview: engage the storage relay only in opaque frames (probe Web Storage instead of just window.parent), so a public raw app embedded in an external iframe hydrates ctx/storage directly; add a relay-timeout fallback so an unresponsive parent can never stall the ctx handshake. - PublicAppFrame: in direct render, expose the page's own bearer credential through the AuthToken context (JWT public URLs), matching the previous route behavior; opaque-viewer mode keeps the embed token. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(apps): sandbox isolation UI polish + COI embed support for raw apps - Deploy drawer: move the sandbox toggle out of "Public URL" into its own "Sandbox isolation" section (the setting applies to every viewing surface, not just the public URL), with positive phrasing, visible helper text, and state-aware alerts (warning when disabled, info for pre-isolation apps). Toggling it now toasts its own message instead of the login-mode one. - Extract the deploy-time migration prompt into a shared LegacySandboxMigrationModal built on the common Modal component, and wire it into the raw app editor header too (it previously had no prompt, so re-deploying a pre-isolation raw app silently changed behavior). updateRawAppPolicy now also drops the backend-owned legacy flag, matching the low-code updatePolicy. - Viewer consent prompt: use the common ConfirmationModal and show the app path (new appPath prop) instead of the route pathname, falling back to "this app" when the path isn't known yet. - COI embeds: propagate the wm_coep opt-in to the raw-app wrapper document and have the backend assert COEP require-corp on it when the flag is present — required for the bundle iframe to load when the public app page is embedded inside a cross-origin-isolated page. Previously this only worked in dev because the Vite proxy injects the header; the production response lacked it. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(apps): app navigation parity across sandboxed and direct viewers - Navbar component: same-app items relay query + hash to the embedder page (which mirrors them onto the root URL, keeping its own pathname and transport params), app items navigate the top page through a validated wm_embed_navigate relay instead of the cookieless viewer iframe, and external items keep opening a new tab. Selected-item detection now recognizes the /app_embed viewer route and ignores transport params. - Frontend-script `goto` and button `onSuccess: gotoUrl`: same-window navigation goes through a shared appNavigateSameWindow helper that relays to the embedder inside the opaque viewer (same-origin paths SPA-navigate, http(s) URLs do a full load, other schemes rejected) and keeps plain window.location everywhere else. - /apps/get and /apps_raw/get: key the viewer by workspace/path so in-route navigation fully remounts it — previously the URL changed but the app (and in sandbox mode its path-scoped token) did not follow. - wm_embed/wm_embedder_origin added to the reserved query params so they no longer leak into the app's ctx.query. - Raw apps: drop the sandbox attribute entirely for the unsandboxed (grandfathered/consented) blob path, matching the pre-isolation viewer exactly — the attribute added no isolation there and sandboxed popups (e.g. OAuth flows). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(apps): preserve grandfathered policy across updates + in-workspace viewer parity Round of compatibility hardening so pre-existing apps behave exactly as before on every surface: - `legacy_unsandboxed` is now preserved across app updates unless the payload explicitly clears it (`false`, sent by the editor's migration prompt and the sandbox toggle). Unrelated update paths — CLI / git-sync redeploys, publish-mode toggles, cross-workspace promotion — no longer silently drop the grandfathering. Clients still can never SET the flag. - The embed-token endpoints (secret, path, EE custom-path) read only the sandbox-decision policy fields, leniently, and no longer mint a token for raw / legacy / disable_sandbox renders: the token is only consumed by the sandboxed low-code render, and minting for the others wrote a useless token row per view and could fail the render for scope-restricted callers. - In-workspace viewer parity with the pre-sandbox `/apps/get`: new `inWorkspace` mode on PublicApp (no "Powered by Windmill" badge / user overlay, no HTML-result approval gate, column flex wrapper, `hideRefreshBar` honored again), and the page's query/hash are forwarded into the opaque viewer so `ctx.query` / `ctx.hash` reach the app. - Raw apps: `window.ctx` is always `{ctx, workspace}` again (anonymous viewers of pre-existing bundles rely on `ctx.workspace`), and the runnable bridge's job-id scoping now applies only to sandboxed renders (`gateJobIds`) — an unsandboxed bundle holds the same credential as the bridge, so gating there only broke pre-existing apps polling persisted or runnable-returned job ids. - Document `disable_sandbox` / `legacy_unsandboxed` in the openapi Policy schema; add a unit test for the lenient policy read. - bump ee-repo-ref to the matching EE commit. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(apps): keep share-link viewer credentials out of the isolated app context The JWT path segment of authenticated share URLs is an embedder-side credential, consumed only to mint the scoped embed token. Two transport channels still copied it into the isolated frame where app-authored code runs: - the opaque viewer iframe src defaulted to window.location.href — the public and custom-path routes now pass a sanitized viewerUrl (JWT segment stripped, query/hash preserved, captured once so the hash relay does not reload the iframe); - document.referrer on the same-origin iframe navigation carried the full embedder URL — both app iframes now set referrerpolicy="no-referrer" (sandboxed renders only for the raw bundle iframe, keeping exact legacy parity; nothing reads the referrer). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * chore(frontend): drop unused import inherited from main merge `slide` import in AssistantMessage.svelte (from #9539) turns `npm run check` red on this branch. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(apps): redirect the removed raw-app viewer path to the unified viewer The old same-origin raw-app viewer route (/apps/get_raw/{version}/{path}) was removed in favor of the sandboxed unified viewer. Re-add a thin client route at the old path that redirects stale bookmarks to /apps_raw/get/{path}, preserving query + hash (the pinned version is dropped — the unified viewer shows latest). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(apps): narrow embed-token scopes and base consent on browser session - Embed token: resource access is metadata-only (list/type/exists) via a `resources:run` marker — resource values (get/get_value/get_value_interpolated/ list_search) are no longer reachable. Job reads are by-id only: an `app_embed` sentinel blocks the workspace-wide job enumeration/export routes (jobs/list, list_filtered_uuids, queue/list, completed/list, queue/export) while by-id result polling keeps working. - disable_sandbox consent now gates on whether the browser holds any Windmill session (cookie-only whoami) rather than workspace-scoped auth, so a viewer logged into a different workspace is still prompted before a same-origin render. - db-explorer: resolve the MySQL database name server-side (the metadata query already falls back to DATABASE()) instead of reading the resource value client-side; getTablesByResource derives the default db from the schema. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * chore(apps): trim embed-scope and consent comments Reduce duplication — state the resource/job route exclusions and the workspace-session-vs-cookie rationale once at their source and reference them elsewhere; drop contrast/justification phrasing. No behavior change. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(apps): make app sandbox isolation opt-in (alpha) Replace the disable_sandbox + legacy_unsandboxed policy pair and the per-version viewer consent with a single positive `sandbox` opt-in flag. Apps are unsandboxed by default (same-origin, full session — the pre-isolation behavior), so existing apps are unchanged and no migration is needed. Publishers opt an app into isolation from the deploy drawer, flagged alpha. - Policy.sandbox: Option<bool>; EmbedTokenResponse -> {token, expiration, raw_app, sandbox}; mint an embed token only for sandboxed low-code apps. - Drop the legacy-unsandboxed migration and the deploy-time migration prompt; remove the consent modal and the browser-session probe. - Deploy drawer: a single "Sandbox isolation" toggle (alpha), off by default, shared by the low-code and raw editors. - Bump ee-repo-ref to the companion EE commit. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(apps): confine embed token to its intended user/folder/job routes The embed token's broad read scopes spanned whole domains while the matching routers are CORS-enabled for the opaque app iframe: - users:read / folders:read were domain-wide, so the token could reach users/list, users/list_usage, users/username_to_email/*, folders/list, etc. Restrict to an app_embed-sentinel allowlist: only users/whoami and folders/listnames; deny the rest of those domains. - jobs:read allowed jobs/completed/export, missed by the job denylist. Add it alongside jobs/queue/export. Extend the embed-scope allow/deny test matrix to cover all of these. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * docs(apps): align sandbox comments with the opt-in model The consent prompt, deploy-time migration, and legacy-unsandboxed grandfathering were removed when sandbox isolation became an opt-in policy flag; update the comments that still described them so they match the two-state (default-unsandboxed / opt-in-sandboxed) reality. Comments only, no behavior change. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(apps): confine embed-token job reads to runs the app launched App component jobs are stamped `created_by = the viewer`, so an embed token reads its own runs via the launched-by-viewer fast path. The token then also inherited the viewer's broader job access (share links, folder ACLs, admin RLS), letting user-authored app JS reuse it to read unrelated jobs by id. Stop embed tokens at the fast path: only jobs the viewer launched, never those merely visible to them. Return NotFound so the untrusted app can't probe existence. Regression test: an embed token reads its own launched job but is denied the foreign job (result/logs/getupdate) an admin viewer's normal token can read. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(apps): allowlist embed-token apps/jobs routes + scope run to the app The embed token's apps:run/jobs:read reached more than a running app needs. Replace the job denylist with strict per-domain allowlists on the app_embed sentinel: - Apps: only the app's own definition (apps/get/p/<path>) and the public app-serving endpoints (apps_u/*). Denies workspace app inventory (exists, custom_path_exists, list, list_paths*). - Jobs: only the by-id poll routes the frontend JobLoader uses. Denies job counts and the job_signature/resume_urls capability-minting routes (the by-id reads remain confined to the app's own runs). Drop unqualified apps:run from APP_EMBED_SCOPES; mint apps:run:<path> instead and authorize apps:run:<requested path> first in execute_component, so the token can only run its own app's components, not another app's. Extend the embed-scope route matrix and add a path-scoped run unit test. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * docs(apps): clarify the sandbox toggle vs the on-behalf-of model The deploy-drawer sandbox copy leaned on "session" in a way that collided with the on-behalf-of permissioning right above it. Reword it to say the toggle governs what the app's browser-side code can reach in the viewer's browser — distinct from who its runnables execute as — and rename the label to "Isolate the app from the viewer's browser session". Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(apps): path-scope embed-token S3 download to its own app The apps_u/* allowlist also admitted apps_u/download_s3_file/<path>, whose handler authorized any authenticated caller — so an embed token minted for app A could download app B's S3 files via B's on-behalf policy. Add the same path-scoped guard execute_component uses: download_s3_file_from_app now checks apps:read:<path> first, confining the token to its own app. Other path-taking apps_u routes are already covered (writes lack apps:write; embed_token/p path-checks; public_resource is type-constrained). Extend the path-scoping unit test to cover apps:read (download) alongside apps:run (execute). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(apps): path-scope public-app-by-secret read to the embed token's app The apps_u/* allowlist admitted apps_u/public_app/<secret>, whose handler only checked the viewer's read access — so an embed token minted for app A could read app B's definition by secret (confused deputy via the viewer's identity). get_public_app_by_secret now binds a scoped caller to the resolved app with check_scopes(apps:read:<path>), confining it to its own app; unscoped sessions and anonymous access are unchanged. get_raw_app_data needs no binding (pure secret capability, no caller identity). Document the full set of app-resolving handlers the path-scoped read covers. Bump ee-repo-ref for the companion custom-path fix. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(apps): preserve pre-sandbox behavior for db-explorer, edit link, jwt Three behavior-parity fixes for non-sandboxed (existing) apps that the sandbox-isolation refactor changed incidentally: - DB-explorer MySQL table picker: when the connection can see multiple non-system schemas, label the default db's tables unprefixed again. The resource-value read was removed globally, so identify the default db from the introspection script's `DATABASE() AS default_db_name` (carried on SQLSchema.defaultDb) instead of guessing "the single schema key". Equivalent to the prior resource.database match; editor-only (table picker). - In-workspace Edit button: restore `?nodraft=true` on both /apps/get and /apps_raw/get, so opening the editor from the viewer loads the deployed version, not a draft. - Custom-path (/a) viewer: restore the "could not authenticate user with jwt token" toast when a path JWT fails to resolve a user, instead of silently falling through. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(apps): confine embed-token S3 downloads to the app's own keys/outputs download_s3_file_from_app authorized any authenticated caller for any S3 key (opt_authed.is_some() bypass). A sandboxed app's embed token carries the viewer's identity, so app-authored JS could fetch arbitrary S3 keys readable by the on-behalf identity, beyond the app's own declared keys or outputs. Route app embed tokens through the same allowlist as anonymous viewers — the app's declared allowed_s3_keys, or files produced by this app's own component runs — instead of the authed bypass. The produced-files check is parameterized by created_by (the embed viewer for a token, else anonymous) so a sandboxed app's own S3 outputs still render while arbitrary keys are denied. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(apps): let embed tokens cancel their own jobs; gate cancel to launcher A sandboxed low-code app supersedes an in-flight component run on re-run by canceling it, but the embed token only had jobs:read, so cancellation silently failed and prior jobs ran to completion. - Permit the by-id jobs_u/queue/cancel POST for app_embed tokens at the route layer (the only write reachable through the existing by-id allowlist). - Gate cancel_job_api: an app_embed token may cancel ONLY jobs it launched (created_by == viewer). cancel_job_api had no other per-job ownership check, so this also confines the token instead of letting it cancel any job by id. - /app_embed now sets workspaceStore so cancellation targets the right workspace instead of an empty/stale one in the cookieless iframe. Add a shared has_app_embed_sentinel helper; cover cancel in the route matrix and the jobs_read_auth integration test (own job cancelable, foreign denied). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(apps): drop get_root_job_id from the embed-token job allowlist Audit of the embed token's reachable job routes: get_root_job (jobs_u/ get_root_job_id) has no access check in its handler at all — it returns any job's root-job id by id — and the app runtime never calls it. Remove it from the by-id allowlist so the embed token can't probe a foreign job's flow lineage; add a denied-route assertion. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(apps): scope sandboxed-app localStorage per app Sandboxed apps shared one localStorage store (one key on the real origin), so an app could read or clobber another app's keys — and, with job ids stashed there, reuse its embed token to read another app's job. Scope the backing store per app. The embed-token endpoints now return the resolved app_path (EmbedTokenResponse; not a new disclosure — the viewer already receives the path when it loads the app). PublicAppFrame (low-code) and RawAppPreview (raw) key their backing store by it: wm_apps_localstorage:<app_path>. Same app shares one store across its public and in-workspace surfaces; different apps are isolated. Unsandboxed apps are unaffected (real same-origin localStorage, as before). Bump ee-repo-ref for the companion custom-path change. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(apps): scope embed access checks to embed tokens + key app storage by workspace - Apply the path-scoped read/run checks on the public-by-secret read and the component run path only when the caller is an app embed token, so other caller types keep their prior access. - Key the sandboxed app's backing client storage by workspace + path instead of path alone, and return the resolved workspace from the embed-token endpoints so the custom-path viewer can derive it. - Show a clear message instead of an indefinite loader when the viewer route is opened outside its embedder. Bumps ee-repo-ref to 5b8476b. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(apps): mint embed tokens only from the trusted embedder caller An app embed token must not reach the embed-token mint endpoints; refresh minting stays with the embedder session/JWT. Enforced at the scope route layer and at the mint chokepoint, with a route-matrix regression test. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * fix(apps): support S3 upload and frontend-script S3 download in sandboxed apps Sandboxed apps run with a scoped embed token (no cookie). Let the app's S3 file-input upload and the frontend-script download({s3}) helper work in that context: upload is reachable with apps:run and re-checked per-app at the handler; the script download routes through the app-scoped apps_u endpoint with the embed token instead of the cookie-authed job_helpers path. Default (unsandboxed) apps are unchanged. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * chore: update ee-repo-ref to b0cb761bf9852974e571b2978032d310cc998517 This commit updates the EE repository reference after PR #600 was merged in windmill-ee-private. Previous ee-repo-ref: e673c714a4618fdb72353a475f49c748e6016642 New ee-repo-ref: b0cb761bf9852974e571b2978032d310cc998517 Automated by sync-ee-ref workflow. --------- Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Co-authored-by: windmill-internal-app[bot] <windmill-internal-app[bot]@users.noreply.github.com> Co-authored-by: Ruben Fiszel <ruben@windmill.dev> |
||
|
centdix
|
6f4017d694 |
feat(ai-chat): workspace AI chat skills (SKILL.md upload + read_skill tool) (#9648)
* feat(ai-chat): workspace ai_skill table + CRUD API Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(ai-chat): AI Skills workspace settings tab with SKILL.md upload Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(ai-chat): advertise skills in global system prompt + read_skill tool Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * refactor(ai-chat): move custom skills into AI settings (paste or folder) Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(ai-chat): cap folder import (depth<=3, max 50 skills, confirm dialog) Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * style(ai-chat): give import folder its own labeled subsection Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(ai-chat): resolve svelte-check never-narrowing in skills preview Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix: address ai skills review issues * fix: validate ai skills and reload workspace list * fix(ai-chat): spec-align skill validation and cap skills per workspace Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(ai-chat): reject duplicate skill uploads, audit skill names Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(ai-chat): sync deref openapi specs with skill validation rules Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
|
hugocasa
|
e19594df2a |
fix: re-enforce scoped API token boundaries across handlers (#9712)
* fix: re-enforce per-path token scope on store rename, delete and interpolation Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix: enforce token scope on workspace export and resume-url minting Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix: enforce per-item and runnable scope on trigger create paths Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix: enforce app write scope before persistence and on rename Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix: enforce scope containment on mcp oauth approval Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix: scope mcp endpoint-proxy jwt to the proxied route Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix: treat resource-linked variables and resources as covered by resource scope Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix: only require variables:read for plaintext-secret workspace export Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix: handle singlestepflow resume, reject empty mcp grant, scope var-skipped tarball Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
|
hugocasa
|
e8e0701a36 |
feat(api): add endpoint to update token label (#9474)
* feat(api): add endpoint to update token label Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(api): prevent renaming the session token label Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(api): restrict token-label edits to user tokens, not just session Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(frontend): edit token label in the edit modal instead of inline Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(api): reject relabeling tokens to reserved system-token names Centralize the is_user_token classifier in windmill-common and reuse it to reject labels colliding with system-token namespaces (ephemeral*, debugger-token, mcp-oauth-*), not just session. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(api): match ephemeral label case-insensitively and cap label length Align the canonical is_user_token, the SQL guard and the frontend mirror on a case-insensitive `ephemeral` match (so a token can't be relabeled to a casing the backend allows but the UI hides), reject labels over the VARCHAR(1000) column limit with a 400, and add unit tests for is_user_token. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
|
Ruben Fiszel
|
2ddf93de96 | fix(auth): enforce monotonic privilege on user token lifecycle endpoints (#9371) | ||
|
Ruben Fiszel
|
b5a0d46695 |
fix(auth): filter resource/variable listings by token scope (WIN-1981) (#9302)
A token scoped to a single resource (e.g. `resources:read:u/alice/foo`)
could call `GET /api/w/{w}/resources/list_search` and receive `path` and
`value` for unrelated resources in the workspace. Route-level scope
checks only validate `domain:action`; per-resource handlers do a
`check_scopes` against the path, but the listing endpoints did not —
leaking integration credentials, API keys, and other secrets stored as
resource values to narrowly-scoped tokens.
Add `build_scope_path_predicate` to `windmill-api-auth` (mirrors
`check_scopes` semantics but parses the token's scopes once, suitable
for filtering many rows). Apply it to `list_search_resources`,
`list_resources`, `list_names` (resources) and `list_variables`
(non-secret value leak), so a scope-restricted token only ever sees the
paths it is authorized to read. Unscoped tokens and tokens whose only
scopes are `if_jobs:filter_tags:*` are unaffected.
Includes regression tests covering: unscoped, tag-filter-only,
single-resource, wildcard, wrong-domain, and write-implies-read.
Fixes WIN-1981
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
|
||
|
Ruben Fiszel
|
7003998a57 |
fix(auth): tighten token-owner fallback for unscoped tokens (WIN-1978) (#9293)
* fix(auth): reject unscoped tokens with cross-workspace forged owners (WIN-1978) An unscoped token (workspace_id IS NULL) whose `owner` field references a user, group, or unprefixed value that is not present in the target workspace must not authenticate. The previous fallback in the `u/<username>` branch granted `(is_admin=false, is_operator=true)` when no `usr` row matched in the target workspace, letting a token holder who could mutate the `token` table cross workspace boundaries with operator privileges. The `g/<groupname>` branch likewise silently accepted any group name as a "group user", and the no-prefix branch granted operator state from arbitrary owner strings. Both are now rejected unless the owner matches a real user/group membership in the target workspace. Adds an integration regression covering all three forged-owner shapes. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * chore: drop integration regression for auth fallback The test added in the previous commit relies on a sqlx::query! that requires offline-cache regeneration; removing per code-review preference to keep this PR scoped to the auth-layer fix. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com> |
||
|
Ruben Fiszel
|
d666e8431c |
feat: read-only flag on API tokens (#9144)
* feat: read-only flag on API tokens, orthogonal to scopes Add a per-token `read_only` boolean set at creation time. When true, the token can only call HTTP methods classified as Read (GET/HEAD/OPTIONS). Mutating methods and job-run actions are rejected with 403, regardless of which scopes are attached. Surfaced as a prominent toggle in the standard token-creation flow and a discreet `2xs` toggle in MCP mode (where users often want write access, so we don't bias them toward enabling it). MCP enforcement: read-only tokens hide all script/flow/hub tools from `list_tools` and only see endpoint tools whose method is GET, and the runner rejects `call_tool` on anything mutating. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * fix: review fixes for read-only token flag - Exempt /api/mcp/* and /mcp/* paths from the read-only middleware check. MCP transport runs over POST (streamable HTTP / SSE), so otherwise the middleware would 403 every MCP request before the runner could enforce read-only at the tool-call level. - Tighten is_endpoint_read_only to GET only, matching the read_only_hint that create_endpoint_annotations actually emits. - Add unit test for check_read_only_for_route covering GET/HEAD/OPTIONS, mutating methods, and run paths. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * chore: bump ee-repo-ref to read-only-trigger-toggle Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * fix(frontend): make read-only toggle discreet in both modes Match the MCP-mode treatment in standard mode: text-tertiary, 2xs, shared "Read-only" label. The tooltip switches per mode so the explanation still fits the context. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * fix(frontend): gate read-only toggle behind Limit token permissions The read-only toggle now only shows when the user has limited the token's scopes (standard mode) or in MCP mode (which always picks an MCP scope). Turning the limit off also resets read-only so it doesn't silently stick. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * feat(frontend): hide incompatible MCP tools when read-only is on When the read-only toggle is on in MCP mode: - Endpoint badges and the custom-mode endpoint MultiSelect filter to GET. - Already-selected non-GET endpoints are pruned from the scope. - The scripts/flows preview is replaced with a note explaining they're hidden (the runner already rejects script/flow runs for read-only). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * fix(frontend): place read-only toggle at top of limited scope area The previous gate required at least one scope to be picked before the read-only toggle appeared, which made it look missing while the user was still building their scope list. Move the toggle inside ScopesPicker: - Standard mode: sits directly under the "Limit token permissions" toggle whenever Limit is on, before the scope selector. - MCP mode: sits at the top of the MCP scope block. readOnly is now $bindable on ScopesPicker so CreateToken still owns the value. The auto-reset on un-limit moves into ScopesPicker too. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * fix(frontend): nest read-only toggle inside the scope list card Place the read-only toggle at the top of the scope list (between the Selected Scopes summary and the bordered domain list) via a new optional topSlot snippet on ScopeSelector. Keeps ScopeSelector decoupled from read-only specifics; ScopesPicker fills the slot. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * chore: update ee-repo-ref to 9bc8160be50b3e57a60daf4e1b71c389a6e02b8a This commit updates the EE repository reference after PR #571 was merged in windmill-ee-private. Previous ee-repo-ref: f53d26e6685dfd60bfa67686fbd7358169cfd130 New ee-repo-ref: 9bc8160be50b3e57a60daf4e1b71c389a6e02b8a Automated by sync-ee-ref workflow. * fix: address CI review for read-only token flag - P1 (Codex): narrow the MCP middleware exemption from "any /api/mcp/*" to just the streamable HTTP transport endpoints (/api/mcp/gateway, /api/mcp/w/{ws}/{mcp,sse,list_tools}). Without this, a read-only token could POST /api/mcp/gateway/oauth/server/approve and mint a follow-on non-read-only MCP token via the OAuth code/token exchange. - P2 (Claude/cubic): fix test comment/assertion mismatch — the run-path assertion now exercises GET (which is what the RUN_PATH_ACTIONS elevation comment describes) in addition to POST. Add a regression assertion for /api/mcp/gateway/oauth/server/approve. - P2 (cubic): short-circuit script/flow/hub-script/resource fetches in MCP list_tools when read_only is on — they would only be discarded below, so skipping the DB and resource fan-out is pure win. - P2 (cubic): when scopes are pre-supplied via the CreateToken prop, the ScopesPicker isn't rendered, which previously hid the read-only toggle entirely. Render it next to the pre-supplied scopes display. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com> Co-authored-by: windmill-internal-app[bot] <windmill-internal-app[bot]@users.noreply.github.com> |
||
|
hugocasa
|
ee3d82f01f |
fix(native-triggers): serialize Google channel renewal across replicas (#9060)
* fix(native-triggers): serialize Google channel renewal across replicas `sync_all_triggers` runs every 5 minutes on every windmill-app replica with no leader election. Multiple replicas were each rotating the webhook token, creating a new Google watch channel, and racing the trigger UPDATE — leaving the loser's new token (in `token`) and channel (in Google) orphaned. Cloud was accumulating ~5 leaked tokens/week without the silent best-effort `delete_token_by_hash` ever logging a warning. Wrap each per-trigger renewal in a transaction and acquire the row with `SELECT … FOR UPDATE SKIP LOCKED`. Contending replicas skip the row instead of duplicating the work. The lock spans `rotate_webhook_token` → Google API call → `update_native_trigger_service_config` and is only released on commit. Re-checks `should_renew_channel` after acquiring the lock so a replica that committed seconds earlier doesn't trigger a duplicate renewal. The pattern matches existing batch-cleanup paths in `monitor.rs` (job-retention sweep) and other `FOR UPDATE SKIP LOCKED` call sites. Also logs at `debug!` when `delete_token_by_hash` finds no matching row, so future investigations can distinguish "deleted" from "not found" without changing the `Ok(false)` contract. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * fixup! fix(native-triggers): serialize Google channel renewal across replicas * fixup! fix(native-triggers): serialize Google channel renewal across replicas fixup! fix(native-triggers): serialize Google channel renewal across replicas Address claude review: - #5: per-skip log info -> debug (expected outcome under SKIP LOCKED) - #2: warn moved out of delete_token_by_hash to the call site that knows the expected state (try_renew_channel_locked); other callers are race-prone and shouldn't warn - #3: NULL service_config now warns (anomalous case) - #4: post-Google-API DB-update + commit failures log distinctly so the channel-orphan case is grep-able Plus: add 14d expiry to Google webhook tokens via ServiceName::webhook_token_expiration, mint fresh ephemeral-webhook-{service}-{rd5} labels at create + rotate so the existing 'ephemeral-' filter excludes them from user-token email/critical-alert paths (no filter changes in 3 places). Orphans now self-clean via the existing expiry sweep in monitor.rs. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * fixup! fix(native-triggers): serialize Google channel renewal across replicas fixup! fix(native-triggers): serialize Google channel renewal across replicas Address second-round review: - Claude #1 (P2): username_override_from_label now strips the 'ephemeral-' prefix for ephemeral-webhook-* labels, so created_by stays webhook-{service}-{rd5} instead of changing to label-ephemeral-webhook-... (preserves audit/job-list filter compatibility) - Codex (P2): updated renew_channel doc — labels are no longer copied; rotate mints fresh ephemeral-webhook-google-{rd5} with 14d expiration - Claude #3 (optional): test_rotate_webhook_token now asserts the rotated Google token has an ephemeral-webhook-google-* label and a populated expiration Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * fixup! fix(native-triggers): serialize Google channel renewal across replicas fixup! fix(native-triggers): serialize Google channel renewal across replicas Reconsider the previous fixup: stripping the 'ephemeral-' prefix made created_by no longer match token.label exactly, defeating the linking purpose. Just allowlist 'ephemeral-webhook-' alongside the other recognized webhook/email/ws prefixes — created_by becomes ephemeral-webhook-google-XXXXX, matching token.label exactly. The 'ephemeral-' substring also informs operators that this is a system-managed auto-expiring token vs a user-managed webhook trigger. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com> |
||
|
hugocasa
|
66db873651 |
fix: surface scope errors as 403 and show real message in CLI (#8953)
* fix: surface scope errors as 403 and show real message in CLI Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * fix: address review feedback on scope error PR - Backend: also patch handler-level check_scopes (lib.rs:223) — without this, endpoints using check_scopes (scripts, flows, jobs, …) still returned 401 for scope failures, which the CLI would render as the misleading auth message. - CLI: strip backend file refs and the duplicated "Permission denied:" / "Not authorized:" prefix from the surfaced error body. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com> |
||
|
hugocasa
|
d6c642b170 |
feat: add Azure Event Grid triggers (#8888)
* feat: add Azure Event Grid triggers (EE)
Introduces a new enterprise trigger kind `azure` that supports three
modes via a single unified trigger type:
- basic_push: Azure Event Grid basic — custom topics, system topics
(Storage, Resource Manager, Key Vault, etc.), domains (push only)
- namespace_push: Event Grid Namespace topics (CloudEvents over HTTP push)
- namespace_pull: Event Grid Namespace topics (HTTP pull with lock-token
ack/reject for dead-lettering)
Auth uses a Service Principal resource (tenant_id, client_id,
client_secret, subscription_id). Subscriptions are created in
CloudEvents 1.0 schema so the push webhook handler and the pull listener
share one payload parser.
Backend
- New crate `windmill-trigger-azure` (OSS stubs + EE impl symlinked from
windmill-ee-private)
- Migration `azure_trigger` table with CHECK constraints enforcing
mode/columns coherence
- `TriggerKind::Azure`, `JobTriggerKind::Azure`,
`DeployedObject::AzureTrigger` variants
- Push route `/api/azure/w/{workspace}/*path` handles classic
Event Grid SubscriptionValidation handshake and CloudEvents 1.0
abuse-protection OPTIONS handshake
- Optional inbound JWT validation (audience check only for v1)
- Feature flag `azure_trigger` propagated through windmill-api,
windmill-store (resource helper), and added to ee_core
Frontend
- `triggers/azure/` editor with mode toggle (basic/namespace-push/
namespace-pull) and per-mode config (topic ARM id / namespace +
topic name / subscription / filters / push auth / pull options)
- Registered in icon map, display names, save functions, badge,
wrapper, editor, add-trigger menu
OpenAPI
- `AzureTrigger`, `AzureTriggerData`, `AzureMode`,
`AzureSubscriptionMode`, `AzureDeliveryConfig`, `TestAzureConnection`
schemas; `/azure_triggers/*` endpoints; client regenerated
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* chore: update ee-repo-ref to eaa7c3a9cb37a9ccc93f10a2535d929365acd2d8
This commit updates the EE repository reference after PR #541 was merged in windmill-ee-private.
Previous ee-repo-ref: 9689014e8c12c36c1059fd8fa5758d550b8b8bc9
New ee-repo-ref: eaa7c3a9cb37a9ccc93f10a2535d929365acd2d8
Automated by sync-ee-ref workflow.
* feat(azure-trigger): secret-auth push, ARM discovery, capture isolation, CLI + parity
Frontend:
- Split mode selector into Namespace/Basic + Pull/Push
- ARM resource dropdowns (namespaces, Basic topics, namespace topics)
populated from the service principal; cascade with stale-selection
reset on SP / edition change
- Remove stale authenticate toggle + audience input (server-managed
push_auth_config has replaced them)
- Azure listing page: "Create from template" button; "Also delete Azure
subscription" toggle in the delete modal; simplified trigger label
falling back to path
- AzureCapture.svelte: "Test subscription name" with -wm-capture suffix
- CompareWorkspaces.svelte: wire Azure for fork/compare
- Drop Trigger-deployed/event-loss warning (capture subscription is
isolated with -wm-capture)
Backend:
- Shared-secret push auth (see EE crate for detail)
- JSONB push_auth_config column (renamed from delivery_config), #[serde(skip)]
so clients/CLI/exports never see it
- Drop redundant enabled column; mode supersedes
- Azure capture infra: AzureTriggerConfig + set_azure_trigger_config +
azure_payload route + TriggerKind::Azure arm; PT15M queue TTL on
capture subscriptions so they bound storage after tab close
- Granular ACLs, users offboarding, trash, git-sync deployed-object:
all include azure_trigger
CLI:
- Add azure to TRIGGER_TYPES, pushObj dispatch, getTypeStrFromPath,
trigger commands (get/update/create/list/template), sync delete
switch + regex; e2e test for `trigger new --kind azure`
- system_prompts: SCHEMA_MAPPINGS + schema_names include AzureTrigger;
auto-generated/* regenerated
Skill:
- .claude/skills/adding-a-trigger/ checklist covering every file that
needs editing when wiring a new trigger type (learned from this PR)
ee-repo-ref bumped to b0e490cbf3724b7b64c6a5b010e3bdf24acd873c.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* fix(azure-trigger): ci — ShareModal Kind + regenerated system_prompts
- frontend/src/lib/components/ShareModal.svelte: add 'azure_trigger'
to the Kind type so the listing page's "Permissions" action compiles
(ts2345 — caught by npm_check on CI, missed by fast-check locally).
- system_prompts/auto-generated/: regenerate to drop the stale
delivery_config / AzureDeliveryConfig fields from the Azure schema
(check-freshness on CI).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* refactor(azure-trigger): use workspace constant_time_eq crate
Drop hand-rolled constant-time compare in favour of the workspace
constant_time_eq crate (same one used by http_trigger_auth).
ee-repo-ref bumped to 9659382d47286e7f7f66d01b6f5dd8d4ed34848b.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* fix(azure-trigger): pass placeholder + disabled via inputProps
`TextInput`'s `placeholder` and `disabled` go through its `inputProps`
prop — CI's `npm run check` caught the stale top-level passing that
`npm run check:fast` missed. Align with the DefaultEmailConfigSection
pattern.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* fix(azure-trigger): correct LATEST_GIT_SYNC_SCRIPT_PATH version to 28213
The hub deploy of the azure-aware sync-script is version 28213, not
28214. Backend was pinning a non-existent hub script, which broke the
git_sync_e2e suite (every deploy's sync step 404'd).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* fix(azure-trigger): add azure_triggers to token scope selector + skill
- windmill-api/src/token.rs: `build_trigger_scope_domains` was missing
`("azure_triggers", "Azure Event Grid")`, so the CreateToken UI's scope
selector didn't surface azure_triggers:read/write. Backend already had
`ScopeDomain::AzureTriggers` wired (scopes.rs), this just exposes it.
- .claude/skills/adding-a-trigger/SKILL.md: capture both scope-related
files under the hardcoded-arrays section so future triggers don't miss
the UI surface.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* docs(adding-a-trigger-skill): clarify token.rs scope effect
Not a regression — nothing was working before. Skipping TRIGGER_DOMAINS
just means the scope works via API/CLI but has no UI checkbox.
* docs(adding-a-trigger-skill): trim token.rs bullet
* fix(azure-trigger): regen openapi-deref + swap textarea for TextInput
- Run build_openapi.sh to regenerate openapi-deref.{yaml,json} with the
12 azure_triggers paths + schemas. These files are served by the
runtime (include_str! in windmill-api/src/lib.rs) to external SDK
consumers; without this regen the new endpoints wouldn't be advertised.
- Replace the raw <textarea> for event type filters with the
design-system TextInput in textarea mode (frontend/CLAUDE.md bans raw
HTML elements).
Addresses cubic + claude PR review items.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Co-authored-by: windmill-internal-app[bot] <windmill-internal-app[bot]@users.noreply.github.com>
|
||
|
centdix
|
aea74445a3 |
fix: add flow conversation token scope (#8903)
* fix: add flow conversation token scope Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * refactor: make flow conversations scope plural Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * fix: update flow chat service import Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com> |
||
|
Ruben Fiszel
|
42d3e8c789 |
fix: enrich OTEL log records with per-request LogContext (#8812)
* fix: enrich OTEL log records with per-request LogContext Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * chore: add otlp_smoke example for manual OTEL log bridge verification Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * chore: update ee-repo-ref to 5d6b713b74fc46735807f5c32883002e8d976fbc This commit updates the EE repository reference after PR #529 was merged in windmill-ee-private. Previous ee-repo-ref: 45959d063bc941c567488d330b5819601cdd2d3d New ee-repo-ref: 5d6b713b74fc46735807f5c32883002e8d976fbc Automated by sync-ee-ref workflow. * refactor: store LogContext in ArcSwap instead of Mutex Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * chore: pin ee-repo-ref to ArcSwap branch commit * chore: update ee-repo-ref to be2f3d4d11bb7110200524d7157caab3aac53996 This commit updates the EE repository reference after PR #530 was merged in windmill-ee-private. Previous ee-repo-ref: 45b4d7963a9ebcd583d1a87abe7d07d3d521584a New ee-repo-ref: be2f3d4d11bb7110200524d7157caab3aac53996 Automated by sync-ee-ref workflow. --------- Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com> Co-authored-by: windmill-internal-app[bot] <windmill-internal-app[bot]@users.noreply.github.com> |
||
|
hugocasa
|
d938625785 |
feat: add download all logs button for flow jobs (#8748)
* feat: add download all logs button for flow jobs Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: use recursive CTE to include all nested flow jobs in log download Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: start iteration index at 1 and interleave children with parents Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: distinguish branch vs loop iteration in log section headers Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: include flownode and singlestepflow kinds in branch/iteration labels Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: improve branch labels (branchone: default/1/2, branchall: 1/2) Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: resolve module types from flow_node table for nested structures Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: use full path in iteration/branch labels and show step kind name Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: show iteration index for simple module forloop optimized jobs Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: handle aiagent jobs as intermediate flow jobs with tool call children Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * refactor: reuse existing get_logs_from_store/disk instead of duplicating Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * sqlx * sqlx --------- Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com> |
||
|
Ruben Fiszel
|
f5fc9f8485 |
fix: require mcp: scope for MCP endpoints instead of blanket bypass (#8597)
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com> |
||
|
Ruben Fiszel
|
3959fe8297 |
feat: add workspace-level service accounts (#8560)
* feat: add workspace-level service accounts (EE) Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * sqlx * sqlx * chore: update ee-repo-ref Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com> |
||
|
Ruben Fiszel
|
0389d9601c |
chore: upgrade axum 0.7 to 0.8 (#8539)
* chore: upgrade axum 0.7 to 0.8 and related dependencies Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * test: add route reachability tests for ~80 previously untested endpoints Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: switch feature-gated trigger handlers from axum::async_trait to async_trait crate Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: update new trash routes to axum 0.8 path syntax Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * chore: update ee-repo-ref to latest EE commit Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * test: upgrade route tests to assert 2xx responses with proper data setup Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * test: restore npm_proxy and ai_routes tests using local echo servers Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: gate workspace fork test behind enterprise feature flag Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * test: add ~40 more endpoint tests (jobs authed, health, favorites, ACLs, reachability) Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: address review findings from axum 0.8 upgrade - Use cookie value_trimmed() instead of value() for cookie 0.18 compat - Update comments still referencing old :workspace_id syntax Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * chore: update ee-repo-ref Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * chore: update ee-repo-ref to 61ae055ea31481f1899953e9d5f65566b8c707b1 This commit updates the EE repository reference after PR #486 was merged in windmill-ee-private. Previous ee-repo-ref: 0059d175a6fdddf52998b183bf91059b224704ac New ee-repo-ref: 61ae055ea31481f1899953e9d5f65566b8c707b1 Automated by sync-ee-ref workflow. * test: add test for new get_imports endpoint Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: remove unused import in raw_apps test Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com> Co-authored-by: windmill-internal-app[bot] <windmill-internal-app[bot]@users.noreply.github.com> |
||
|
hugocasa
|
efb4a27d51 |
fix: replace email with permissioned_as for triggers/schedules (#8439)
* refactor: replace email with permissioned_as for triggers/schedules
Add a new `permissioned_as` column (format: `u/{username}`, `g/{group}`,
or raw email) to all trigger tables and schedule. This value is used
directly for job permission checks, removing the need for email lookups
when creating/updating triggers.
- Migration: add permissioned_as to all 9 trigger tables + schedule,
drop email from trigger tables (schedule keeps it for backwards compat)
- Backend: resolve_email() (async, DB) -> resolve_permissioned_as() (sync)
- Email cache: get_email_from_permissioned_as() with quick_cache for
places that still need email (fetch_api_authed, schedule backwards compat)
- Frontend: rename email/preserve_email -> permissioned_as/preserve_permissioned_as
in deploy data and OpenAPI schemas
- Tests updated for new field names and u/{username} format
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* fix sqlx/build
* update ee ref
* refactor: simplify resolve_edited_by to always use authed username
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* fix compile + migration
* update ref
* test: add trigger trait method tests for permissioned_as queries
Add tests that call TriggerCrud and Listener trait methods directly
to verify dynamic SQL correctly references the permissioned_as column.
Covers get_trigger_by_path, list_triggers, set_trigger_mode, and
fetch_enabled_unlistened_triggers for all trigger types.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* update sqlx
* fix: use permissioned_as directly for schedules and fix audit RLS for groups
- Schedule: permissioned_as only set on create, not on edit/set_enabled
- Schedule: stop reading email column, use get_email_from_permissioned_as
- Triggers: use fetch_api_authed_from_permissioned_as instead of edited_by
- Triggers: rename listener fields for clarity (username -> edited_by)
- Fix audit author username for group permissioned_as (g/test -> group-test)
to match session.user, preventing RLS policy violations on audit_partitioned
- OpenAPI: remove permissioned_as/preserve_permissioned_as from EditSchedule
- Add backwards-compat comments for schedule email writes
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* chore: regenerate system prompts for permissioned_as field
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* fix build
* refactor: generalize onBehalfOf naming, add permissioned_as to EditSchedule
- Frontend: rename onBehalfOfPermissionedAs -> onBehalfOf with comments
explaining it carries emails for flows/scripts and permissioned_as for
triggers/schedules
- Frontend: rename getOnBehalfOfEmail -> getOnBehalfOf,
getOnBehalfOfPermissionedAsForDeploy -> getOnBehalfOfForDeploy,
customOnBehalfOfEmails -> customOnBehalfOf
- Backend: add optional permissioned_as/preserve_permissioned_as to
EditSchedule with COALESCE (only updates when provided)
- Backend: add on_behalf_of audit log for schedule edit
- Backend: remove unused resolve_on_behalf_of_permissioned_as
- Tests: remove email assertions from schedule update test (email is
just backwards compat, only permissioned_as matters)
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* fix: preserve email column when permissioned_as is preserved on schedule edit
Derive email from the preserved permissioned_as via cache lookup instead
of always writing authed.email. This keeps the email column consistent
with the old behavior for backwards compat with old workers.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* fix: update deploy UI labels from "edited by" to "run as" for triggers
Triggers now use permissioned_as (not edited_by) for permissions, so
update the deploy UI wording to reflect this. Also update wm_deployers
group description to mention schedules and permissioned_as.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* fix: use u/username format for custom trigger/schedule deploy selection
When picking a custom user for trigger/schedule deployment, store
u/${username} (permissioned_as format) instead of the email. Flows/scripts
continue to use email format for on_behalf_of_email.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* fix: show u/username format for "me" option in trigger deploy selector
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* refactor: simplify OnBehalfOfSelector to return the right format per kind
OnBehalfOfSelector now handles the email vs permissioned_as format
internally based on kind:
- triggers: returns u/username, displays u/username in all options
- flows/scripts/apps: returns email, displays username
The onSelect callback now takes (choice, value?) where value is already
in the correct format. Parent components just store it directly without
needing to know about the format difference.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* fix: always show u/username format in OnBehalfOfSelector for all kinds
Display is now consistent: all kinds show u/username in the selector.
The returned value still differs (email for flows/scripts, u/username
for triggers) since the backend APIs expect different formats.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* fix: replace email with permissioned_as in http_trigger test insert
The email column was dropped from trigger tables in the migration.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* fix: review fixes — migration, app policy, capture cleanup, naming
- Migration: remove DEFAULT '', use nullable → populate → SET NOT NULL
- App policy: set both on_behalf_of and on_behalf_of_email for all choices
- OnBehalfOfSelector: return OnBehalfOfDetails {email, permissionedAs} instead of ambiguous value
- Remove unused email field from Capture struct and query
- Rename getSourceEmail/getTargetEmail → getSourceOnBehalfOf/getTargetOnBehalfOf
- Rename test functions from preserve_email to preserve_permissioned_as
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* fix: add permissioned_as to all test schedule INSERTs
Since the migration no longer uses DEFAULT '', all INSERTs must
explicitly provide permissioned_as. Updated test fixtures and
schedule_push tests.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* fix: strip permissioned_as from exports/sync, fix OpenAPI required field
- Add permissioned_as to workspace export strip list (like edited_by)
- Add permissioned_as to CLI TriggerFile Omit list
- Fix TriggerExtraProperty.required: email → permissioned_as
- Regenerate frontend and CLI types
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* fix: remove accidentally committed generated files
These directories are gitignored and should not be tracked.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* chore: regenerate system prompts for permissioned_as schema changes
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* fix: remove permissioned_as from CLI TriggerFile Omit list
Already stripped in workspace export, no need to also omit from the type.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* fix: optimize email cache key and revert TriggerFile Omit change
- Use single concatenated string for cache key instead of (String, String) tuple
- Remove permissioned_as from CLI TriggerFile Omit (already stripped in export)
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* fix: zero-allocation email cache lookups using Equivalent trait
Use a borrowed EmailCacheKey(&str, &str) for cache lookups via
quick_cache's Equivalent support. Only allocates (String, String)
on cache miss for insert. This is called on every trigger fire
and schedule push.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* fix: add permissioned_as to Schedule required fields in OpenAPI spec
The backend always returns permissioned_as (non-optional String),
so the schema should reflect that.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* fix: handle group- prefix in migration UPDATE statements
edited_by can be 'group-{name}' for group-owned triggers/schedules.
The migration now correctly maps these to 'g/{name}' format instead
of incorrectly producing 'u/group-{name}'.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* Revert "fix: handle group- prefix in migration UPDATE statements"
This reverts commit
|
||
 Alexander Petric
|
51957f7d92 |
feat: mcp oauth gateway (#8443)
* feat: extract McpScopeSelector into reusable component Extract scope selection UI from CreateToken.svelte and mcp_authorize page into a shared McpScopeSelector.svelte component to reduce duplication. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * feat: add MCP gateway endpoint for workspace-agnostic access Add /api/mcp/gateway endpoint that allows MCP clients to connect without knowing the workspace ID upfront. During OAuth, the user picks their workspace on the consent page. The token is then scoped to that workspace. This enables a single URL for the Anthropic connectors directory. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * fix: address PR review nits - Use onClick prop instead of legacy on:click directive in McpScopeSelector - Remove unused catch variable in workspace loading Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * refactor: deduplicate gateway OAuth handlers into shared inner functions Extract build_oauth_metadata, build_protected_resource_metadata, oauth_authorize_inner, and oauth_approve_inner so gateway handlers are thin wrappers. Also revert formatting-only changes in auth.rs. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * fix: gate run_inline test helpers behind cfg(feature = "run_inline") Imports and helper functions were not gated, causing unused-import and dead-code errors when compiling without the run_inline feature. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * Update SQLx metadata --------- Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com> Co-authored-by: windmill-internal-app[bot] <windmill-internal-app[bot]@users.noreply.github.com> |
||
|
Ruben Fiszel
|
f2be625348 |
feat: store hashed tokens instead of plaintext (#8217)
* feat: store hashed tokens in the token table instead of plaintext
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* fix: address review issues in token hash migration
- Update all base.sql fixtures to include token_hash/token_prefix columns
- Keep plaintext token for webhook tokens (needed for URL reconstruction)
- Restore get_token_by_prefix to query DB for webhook tokens
- Fix down migration to delete NULL-token rows before restoring NOT NULL
- Update parser fixture standalone schema
- Update EE dedicated_worker_ee.rs to use token_hash/token_prefix
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* fix: restore sqlx offline cache (only add new query files)
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* refactor: keep writing plaintext token column for backward compat
Write to token column alongside token_hash until MIN_VERSION_SUPPORTS_TOKEN_HASH
(1.649.0) is reached. This ensures older workers can still authenticate
during rolling upgrades. Remove the separate UPDATE in new_webhook_token
since create_token_internal now writes plaintext directly.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* feat: branch on MIN_VERSION to write plaintext token or null
Check MIN_VERSION_SUPPORTS_TOKEN_HASH at runtime: write plaintext to
token column while old workers exist, switch to NULL once all workers
are >= 1.649.0.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* fix: set MIN_VERSION_SUPPORTS_TOKEN_HASH to 1.650.0
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* fix: use token_hash for email lookup and expiry notifications
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* refactor: rotate webhook tokens instead of recovering plaintext from DB
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* refactor: use token_hash for native trigger token lookups and deletes
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* sqlx
* refactor: drop webhook_token_prefix from native_trigger table
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
* fix: backward compat for token rotation and make webhook_token_hash NOT NULL
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
* fix: prevent panic on short superadmin secret token prefix
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
* fix: prevent panic on short superadmin secret token prefix
Replace all `token[0..TOKEN_PREFIX_LEN]` slicing with
`token.get(..TOKEN_PREFIX_LEN).unwrap_or(token)` to prevent
panics when a token shorter than 10 chars is provided (e.g.
malformed Authorization header, short superadmin secret).
Co-authored-by: hugocasa <hugocasa@users.noreply.github.com>
* fix: prevent panic on short token prefix slicing
Replace all `token[0..TOKEN_PREFIX_LEN]` with safe
`token.get(..TOKEN_PREFIX_LEN).unwrap_or(token)` to prevent panics
on malformed tokens shorter than 10 characters.
Co-authored-by: hugocasa <hugocasa@users.noreply.github.com>
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* Revert "fix: prevent panic on short superadmin secret token prefix"
This reverts commit
|
||
|
hugocasa
|
8667329110 |
fix: skip token expiry notifications for debugger and mcp-oauth tokens (#8316)
* fix: skip token expiry notifications for debugger and mcp-oauth tokens Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * fix: update frontend isUserToken to match backend filter Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * chore: add cross-reference comments to token filter functions Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com> |
||
|
hugocasa
|
e56ccd200b |
feat: token expiration notifications (#8190)
* feat: add token expiration notifications via email, critical alerts, and webhooks - Monitor loop checks for tokens expiring within 7 days and sends email notifications to token owners. Tracks notification state via new `expiry_notified` column on the token table to avoid duplicates. - When tokens expire and are deleted, owners are also notified. - Critical alerts (in-app UI) are gated behind a new instance setting `critical_alerts_on_token_expiry` (off by default); emails are always sent regardless of the setting. - Add TokenExpiringSoon and TokenExpired webhook message variants for workspace webhook integrations. - Frontend: show expiration badges and a warning banner on the tokens table for tokens expiring within 30 days. - Exclude session and ephemeral tokens from all notifications. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * refactor: use separate token_expiry_notification table for dedup - Replace `expiry_notified` column on token table with a dedicated `token_expiry_notification` table (token, expiration) - Insert notification row on token creation via shared `register_token_expiry_notification()` helper - Delete notification row atomically when sending the notification - Clean up orphaned rows in `delete_expired_items()` - No FK constraint to avoid cascade overhead on token deletions - Add index on expiration column for efficient range queries Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * fix: calendar-based expiration badge and move notification cleanup - Fix daysUntilExpiration to compare calendar dates instead of time diff - Move notification row cleanup from delete_expired_items to check_expiring_tokens to keep it off the hot path - Use simple expiration <= now() index scan instead of NOT EXISTS join Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com> |
||
 Pyra
|
baf2bcf14d |
feat: make WM_END_USER_EMAIL display users from different workspaces (#8208)
Signed-off-by: pyranota <pyra@duck.com> |
||
|
Ruben Fiszel
|
bc672555a7 |
fix: delete non-session tokens on workspace archive and reject token creation for archived workspaces (#8082)
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com> |
||
|
Ruben Fiszel
|
894d8a94f8 |
reuse existing transaction in push to reduce pool pressure (#7858)
* fix: reuse existing transaction in push instead of acquiring new connection In push_inner, fetch_authed_from_permissioned_as was acquiring a new connection from the pool to fetch job permissions, even though a transaction was already open. Use fetch_authed_from_permissioned_as_conn with the existing transaction instead, reducing pool pressure when many jobs are pushed concurrently. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * improve contention * improve contention --------- Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com> |
||
|
Ruben Fiszel
|
9ff8a85af6 |
refactor: extract windmill-api into subcrates for parallel compilation (#7845)
* refactor: extract windmill-api into 4 subcrates (api-auth, store, api-sse, api-jobs) Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * refactor: eliminate refresh_token OnceLock bridge in windmill-store Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * refactor: eliminate FromRequestParts OnceLock bridge in windmill-api-auth Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * refactor: wire subcrates into workspace and clean up unused re-exports Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * fix: resolve cargo check --all-features errors in subcrate wiring Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * sqlx * all * chore: update ee-repo-ref for warning fixes Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * refactor: extract windmill-trigger crate and expand windmill-api-jobs Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * refactor: extract windmill-trigger-kafka crate from windmill-api Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * refactor: extract windmill-trigger-postgres crate from windmill-api Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * refactor: extract windmill-trigger-websocket and windmill-trigger-mqtt crates Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * refactor: extract windmill-trigger-nats, sqs, gcp, and email crates Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * refactor: extract windmill-trigger-http crate from windmill-api Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * refactor: move token creation and permission helpers to windmill-api-auth Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * refactor: extract windmill-native-triggers crate from windmill-api Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * sqlx * all * refactor: extract windmill-api-embeddings crate and fix CI warnings Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * fix: resolve type mismatch in oauth2_oss and remaining warnings Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * fix: use correct HTTP_CLIENT config in embeddings crate (30s timeout, cert override) Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * all * fix: gate oauth_refresh_ee on oauth2 feature to fix warnings Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * all --------- Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com> |